Class LZ4


  • public class LZ4
    extends java.lang.Object
    Native bindings to LZ4, a lossless compression algorithm, providing compression speed at 400 MB/s per core, scalable with multi-cores CPU. It features an extremely fast decoder, with speed in multiple GB/s per core, typically reaching RAM speed limits on multi-core systems.

    Speed can be tuned dynamically, selecting an "acceleration" factor which trades compression ratio for more speed up. On the other end, a high compression derivative, LZ4_HC, is also provided, trading CPU time for improved compression ratio. All versions feature the same decompression speed.

    The raw LZ4 block compression format is detailed within lz4_Block_format.

    To compress an arbitrarily long file or data stream, multiple blocks are required. Organizing these blocks and providing a common header format to handle their content is the purpose of the Frame format, defined into lz4_Frame_format. Interoperable versions of LZ4 must respect this frame format.

    • Method Detail

      • LZ4_versionNumber

        public static int LZ4_versionNumber()
        Returns the version number.
      • nLZ4_versionString

        public static long nLZ4_versionString()
        Unsafe version of: versionString
      • LZ4_versionString

        public static java.lang.String LZ4_versionString()
        Returns the version string.
      • nLZ4_compress_default

        public static int nLZ4_compress_default​(long src,
                                                long dst,
                                                int srcSize,
                                                int dstCapacity)
        Unsafe version of: compress_default
        Parameters:
        srcSize - max supported value is MAX_INPUT_SIZE
        dstCapacity - size of buffer dst (which must be already allocated)
      • LZ4_compress_default

        public static int LZ4_compress_default​(java.nio.ByteBuffer src,
                                               java.nio.ByteBuffer dst)
        Compresses srcSize bytes from buffer src into already allocated dst buffer of size dstCapacity.

        Compression is guaranteed to succeed if dstCapacitycompressBound(srcSize). It also runs faster, so it's a recommended setting.

        If the function cannot compress src into a more limited dst budget, compression stops immediately, and the function result is zero. As a consequence, dst content is not valid.

        This function is protected against buffer overflow scenarios (never writes outside dst buffer, nor read outside src buffer).

        Returns:
        the number of bytes written into buffer dest (necessarily ≤ maxOutputSize) or 0 if compression fails
      • nLZ4_decompress_safe

        public static int nLZ4_decompress_safe​(long src,
                                               long dst,
                                               int compressedSize,
                                               int dstCapacity)
        Unsafe version of: decompress_safe
        Parameters:
        compressedSize - is the exact complete size of the compressed block
        dstCapacity - is the size of destination buffer, which must be already allocated
      • LZ4_decompress_safe

        public static int LZ4_decompress_safe​(java.nio.ByteBuffer src,
                                              java.nio.ByteBuffer dst)
        If destination buffer is not large enough, decoding will stop and output an error code (negative value).

        If the source stream is detected malformed, the function will stop decoding and return a negative result.

        This function is protected against malicious data packets.

        Returns:
        the number of bytes decompressed into destination buffer (necessarily ≤ dstCapacity)
      • LZ4_COMPRESSBOUND

        public static int LZ4_COMPRESSBOUND​(int isize)
      • LZ4_compressBound

        public static int LZ4_compressBound​(int inputSize)
        Provides the maximum size that LZ4 compression may output in a "worst case" scenario (input data not compressible).

        This function is primarily useful for memory allocation purposes (destination buffer size). Macro COMPRESSBOUND is also provided for compilation-time evaluation (stack memory allocation for example).

        Note that compress_default compresses faster when dstCapacity is ≥ compressBound(srcSize)

        Parameters:
        inputSize - max supported value is MAX_INPUT_SIZE
        Returns:
        maximum output size in a "worst case" scenario or 0, if input size is incorrect (too large or negative)
      • nLZ4_compress_fast

        public static int nLZ4_compress_fast​(long src,
                                             long dst,
                                             int srcSize,
                                             int dstCapacity,
                                             int acceleration)
        Unsafe version of: compress_fast
      • LZ4_compress_fast

        public static int LZ4_compress_fast​(java.nio.ByteBuffer src,
                                            java.nio.ByteBuffer dst,
                                            int acceleration)
        Same as compress_default, but allows selection of "acceleration" factor.

        The larger the acceleration value, the faster the algorithm, but also the lesser the compression. It's a trade-off. It can be fine tuned, with each successive value providing roughly +~3% to speed. An acceleration value of "1" is the same as regular compress_default. Values ≤ 0 will be replaced by ACCELERATION_DEFAULT (currently == 1, see lz4.c).

      • LZ4_sizeofState

        public static int LZ4_sizeofState()
      • nLZ4_compress_fast_extState

        public static int nLZ4_compress_fast_extState​(long state,
                                                      long src,
                                                      long dst,
                                                      int srcSize,
                                                      int dstCapacity,
                                                      int acceleration)
        Unsafe version of: compress_fast_extState
      • LZ4_compress_fast_extState

        public static int LZ4_compress_fast_extState​(java.nio.ByteBuffer state,
                                                     java.nio.ByteBuffer src,
                                                     java.nio.ByteBuffer dst,
                                                     int acceleration)
        Same as compress_fast, just using an externally allocated memory space to store compression state.

        Use sizeofState to know how much memory must be allocated, and allocate it on 8-bytes boundaries (using malloc() typically). Then, provide it as void* state to compression function.

      • nLZ4_compress_destSize

        public static int nLZ4_compress_destSize​(long src,
                                                 long dst,
                                                 long srcSizePtr,
                                                 int targetDstSize)
        Unsafe version of: compress_destSize
        Parameters:
        srcSizePtr - will be modified to indicate how many bytes where read from source to fill dest. New value is necessarily ≤ old value.
      • LZ4_compress_destSize

        public static int LZ4_compress_destSize​(java.nio.ByteBuffer src,
                                                java.nio.ByteBuffer dst,
                                                java.nio.IntBuffer srcSizePtr)
        Reverse the logic: compresses as much data as possible from src buffer into already allocated buffer dst of size targetDstSize.

        This function either compresses the entire src content into dst if it's large enough, or fill dst buffer completely with as much data as possible from src.

        Parameters:
        srcSizePtr - will be modified to indicate how many bytes where read from source to fill dest. New value is necessarily ≤ old value.
        Returns:
        nb bytes written into dest (necessarily ≤ targetDestSize) or 0 if compression fails
      • nLZ4_decompress_fast

        public static int nLZ4_decompress_fast​(long src,
                                               long dst,
                                               int originalSize)
        Unsafe version of: decompress_fast
        Parameters:
        originalSize - is the uncompressed size to regenerate. Destination buffer must be already allocated, and its size must be ≥ originalSize bytes.
      • LZ4_decompress_fast

        public static int LZ4_decompress_fast​(java.nio.ByteBuffer src,
                                              java.nio.ByteBuffer dst)
        This function is a bit faster than decompress_safe, but it may misbehave on malformed input because it doesn't perform full validation of compressed data.

        This function is only usable if the originalSize of uncompressed data is known in advance. The caller should also check that all the compressed input has been consumed properly, i.e. that the return value matches the size of the buffer with compressed input. The function never writes past the output buffer. However, since it doesn't know its src size, it may read past the intended input. Also, because match offsets are not validated during decoding, reads from src may underflow. Use this function in trusted environment only.

        Returns:
        the number of bytes read from the source buffer (== the compressed size). If the source stream is detected malformed, the function stops decoding and return a negative result. Destination buffer must be already allocated. Its size must be ≥ originalSize bytes.
      • nLZ4_decompress_safe_partial

        public static int nLZ4_decompress_safe_partial​(long src,
                                                       long dst,
                                                       int compressedSize,
                                                       int targetOutputSize,
                                                       int dstCapacity)
        Unsafe version of: decompress_safe_partial
      • LZ4_decompress_safe_partial

        public static int LZ4_decompress_safe_partial​(java.nio.ByteBuffer src,
                                                      java.nio.ByteBuffer dst,
                                                      int targetOutputSize)
        This function decompress a compressed block of size compressedSize at position src into destination buffer dst of size dstCapacity.

        The function will decompress a minimum of targetOutputSize bytes, and stop after that. However, it's not accurate, and may write more than targetOutputSize (but always ≤ dstCapacity).

        This function never writes outside of output buffer, and never reads outside of input buffer. It is therefore protected against malicious data packets.

        Returns:
        the number of bytes decoded in the destination buffer (necessarily ≤ dstCapacity)

        Note: this number can also be < targetOutputSize, if compressed block contains less data. Therefore, always control how many bytes were decoded. If source stream is detected malformed, function returns a negative result. This function is protected against malicious data packets.

      • LZ4_createStream

        public static long LZ4_createStream()
        Allocates and initializes an LZ4_stream_t structure.
      • nLZ4_freeStream

        public static int nLZ4_freeStream​(long streamPtr)
        Unsafe version of: freeStream
      • LZ4_freeStream

        public static int LZ4_freeStream​(long streamPtr)
        Releases memory of an LZ4_stream_t structure.
      • nLZ4_resetStream

        public static void nLZ4_resetStream​(long streamPtr)
        Unsafe version of: resetStream
      • LZ4_resetStream

        public static void LZ4_resetStream​(long streamPtr)
        An LZ4_stream_t structure can be allocated once and re-used multiple times. Use this function to start compressing a new stream.
      • nLZ4_loadDict

        public static int nLZ4_loadDict​(long streamPtr,
                                        long dictionary,
                                        int dictSize)
        Unsafe version of: loadDict
      • LZ4_loadDict

        public static int LZ4_loadDict​(long streamPtr,
                                       @Nullable
                                       java.nio.ByteBuffer dictionary)
        Use this function to load a static dictionary into LZ4_stream_t.

        Any previous data will be forgotten, only dictionary will remain in memory. Loading a size of 0 is allowed, and is the same as reset.

      • nLZ4_compress_fast_continue

        public static int nLZ4_compress_fast_continue​(long streamPtr,
                                                      long src,
                                                      long dst,
                                                      int srcSize,
                                                      int dstCapacity,
                                                      int acceleration)
        Unsafe version of: compress_fast_continue
      • LZ4_compress_fast_continue

        public static int LZ4_compress_fast_continue​(long streamPtr,
                                                     java.nio.ByteBuffer src,
                                                     java.nio.ByteBuffer dst,
                                                     int acceleration)
        Compress src content using data from previously compressed blocks, for better compression ratio.

        dst buffer must be already allocated. If dstCapacitycompressBound(srcSize), compression is guaranteed to succeed, and runs faster.

        Important: The previous 64KB of compressed data is assumed to remain present and unmodified in memory! If less than 64KB has been compressed all the data must be present.

        Special:

        1. When input is a double-buffer, they can have any size, including < 64 KB. Make sure that buffers are separated by at least one byte. This way, each block only depends on previous block.
        2. If input buffer is a ring-buffer, it can have any size, including < 64 KB.
        Returns:
        size of compressed block or 0 if there is an error (typically, cannot fit into dst). After an error, the stream status is invalid, it can only be reset or freed.
      • nLZ4_saveDict

        public static int nLZ4_saveDict​(long streamPtr,
                                        long safeBuffer,
                                        int maxDictSize)
        Unsafe version of: saveDict
      • LZ4_saveDict

        public static int LZ4_saveDict​(long streamPtr,
                                       java.nio.ByteBuffer safeBuffer)
        If last 64KB data cannot be guaranteed to remain available at its current memory location, save it into a safer place (char* safeBuffer).

        This is schematically equivalent to a memcpy() followed by loadDict, but is much faster, because LZ4_saveDict() doesn't need to rebuild tables.

        Returns:
        saved dictionary size in bytes (necessarily ≤ maxDictSize), or 0 if error
      • LZ4_createStreamDecode

        public static long LZ4_createStreamDecode()
        Creates a streaming decompression tracking context.

        A tracking context can be re-used multiple times.

      • nLZ4_freeStreamDecode

        public static int nLZ4_freeStreamDecode​(long LZ4_stream)
        Unsafe version of: freeStreamDecode
      • LZ4_freeStreamDecode

        public static int LZ4_freeStreamDecode​(long LZ4_stream)
        Frees a streaming decompression tracking context.
      • nLZ4_setStreamDecode

        public static int nLZ4_setStreamDecode​(long LZ4_streamDecode,
                                               long dictionary,
                                               int dictSize)
        Unsafe version of: setStreamDecode
      • LZ4_setStreamDecode

        public static boolean LZ4_setStreamDecode​(long LZ4_streamDecode,
                                                  java.nio.ByteBuffer dictionary)
        An LZ4_streamDecode_t context can be allocated once and re-used multiple times. Use this function to start decompression of a new stream of blocks.

        A dictionary can optionnally be set. Use NULL or size 0 for a reset order. Dictionary is presumed stable: it must remain accessible and unmodified during next decompression.

        Returns:
        1 if OK, 0 if error
      • LZ4_decoderRingBufferSize

        public static int LZ4_decoderRingBufferSize​(int maxBlockSize)
        In a ring buffer scenario (optional), blocks are presumed decompressed next to each other up to the moment there is not enough remaining space for next block (remainingSize &lt; maxBlockSize), at which stage it resumes from beginning of ring buffer. When setting such a ring buffer for streaming decompression, provides the minimum size of this ring buffer to be compatible with any source respecting maxBlockSize condition.
        Returns:
        minimum ring buffer size, or 0 if there is an error (invalid maxBlockSize)
        Since:
        version 1.8.2
      • nLZ4_decompress_safe_continue

        public static int nLZ4_decompress_safe_continue​(long LZ4_streamDecode,
                                                        long src,
                                                        long dst,
                                                        int srcSize,
                                                        int dstCapacity)
        Unsafe version of: decompress_safe_continue
      • LZ4_decompress_safe_continue

        public static int LZ4_decompress_safe_continue​(long LZ4_streamDecode,
                                                       java.nio.ByteBuffer src,
                                                       java.nio.ByteBuffer dst)
        These decoding functions allow decompression of consecutive blocks in "streaming" mode.

        A block is an unsplittable entity, it must be presented entirely to a decompression function. Decompression functions only accept one block at a time. The last 64KB of previously decoded data must remain available and unmodified at the memory position where they were decoded. If less than 64KB of data has been decoded, all the data must be present.

        Special: if decompression side sets a ring buffer, it must respect one of the following conditions:

        • Decompression buffer size is at least decoderRingBufferSize(maxBlockSize). maxBlockSize is the maximum size of any single block. It can have any value > 16 bytes. In which case, encoding and decoding buffers do not need to be synchronized. Actually, data can be produced by any source compliant with LZ4 format specification, and respecting maxBlockSize.
        • Synchronized mode: Decompression buffer size is exactly the same as compression buffer size, and follows exactly same update rule (block boundaries at same positions), and decoding function is provided with exact decompressed size of each block (exception for last block of the stream), then decoding & encoding ring buffer can have any size, including small ones ( < 64 KB).
        • Decompression buffer is larger than encoding buffer, by a minimum of maxBlockSize more bytes. In which case, encoding and decoding buffers do not need to be synchronized, and encoding ring buffer can have any size, including small ones ( < 64 KB).

        Whenever these conditions are not possible, save the last 64KB of decoded data into a safe buffer where it can't be modified during decompression, then indicate where this data is saved using setStreamDecode, before decompressing next block.

      • nLZ4_decompress_fast_continue

        public static int nLZ4_decompress_fast_continue​(long LZ4_streamDecode,
                                                        long src,
                                                        long dst,
                                                        int originalSize)
        Unsafe version of: decompress_fast_continue
      • LZ4_decompress_fast_continue

        public static int LZ4_decompress_fast_continue​(long LZ4_streamDecode,
                                                       java.nio.ByteBuffer src,
                                                       java.nio.ByteBuffer dst)
      • nLZ4_decompress_safe_usingDict

        public static int nLZ4_decompress_safe_usingDict​(long src,
                                                         long dst,
                                                         int srcSize,
                                                         int dstCapacity,
                                                         long dictStart,
                                                         int dictSize)
        Unsafe version of: decompress_safe_usingDict
      • LZ4_decompress_safe_usingDict

        public static int LZ4_decompress_safe_usingDict​(java.nio.ByteBuffer src,
                                                        java.nio.ByteBuffer dst,
                                                        java.nio.ByteBuffer dictStart)
        These decoding functions work the same as a combination of setStreamDecode followed by LZ4_decompress_*_continue(). They are stand-alone, and don't need an LZ4_streamDecode_t structure.

        Dictionary is presumed stable: it must remain accessible and unmodified during next decompression.

      • nLZ4_decompress_fast_usingDict

        public static int nLZ4_decompress_fast_usingDict​(long src,
                                                         long dst,
                                                         int originalSize,
                                                         long dictStart,
                                                         int dictSize)
        Unsafe version of: decompress_fast_usingDict
      • LZ4_decompress_fast_usingDict

        public static int LZ4_decompress_fast_usingDict​(java.nio.ByteBuffer src,
                                                        java.nio.ByteBuffer dst,
                                                        java.nio.ByteBuffer dictStart)
        See decompress_safe_usingDict.
      • nLZ4_resetStream_fast

        public static void nLZ4_resetStream_fast​(long streamPtr)
        Unsafe version of: resetStream_fast
      • LZ4_resetStream_fast

        public static void LZ4_resetStream_fast​(long streamPtr)
        Use this, like resetStream, to prepare a context for a new chain of calls to a streaming API (e.g., compress_fast_continue).
        Note

        Using this in advance of a non- streaming-compression function is redundant, and potentially bad for performance, since they all perform their own custom reset internally.

        Differences from resetStream:

        When an LZ4_stream_t is known to be in a internally coherent state, it can often be prepared for a new compression with almost no work, only sometimes falling back to the full, expensive reset that is always required when the stream is in an indeterminate state (i.e., the reset performed b resetStream).

        LZ4_streams are guaranteed to be in a valid state when:

        • returned from createStream
        • reset by resetStream
        • memset(stream, 0, sizeof(LZ4_stream_t)), though this is discouraged
        • the stream was in a valid state and was reset by resetStream_fast
        • the stream was in a valid state and was then used in any compression call that returned success
        • the stream was in an indeterminate state and was used in a compression call that fully reset the state (e.g., compress_fast_extState) and that returned success

        When a stream isn't known to be in a valid state, it is not safe to pass to any fastReset or streaming function. It must first be cleansed by the full resetStream.

      • nLZ4_compress_fast_extState_fastReset

        public static int nLZ4_compress_fast_extState_fastReset​(long state,
                                                                long src,
                                                                long dst,
                                                                int srcSize,
                                                                int dstCapacity,
                                                                int acceleration)
      • LZ4_compress_fast_extState_fastReset

        public static int LZ4_compress_fast_extState_fastReset​(java.nio.ByteBuffer state,
                                                               java.nio.ByteBuffer src,
                                                               java.nio.ByteBuffer dst,
                                                               int acceleration)
        A variant of compress_fast_extState.

        Using this variant avoids an expensive initialization step. It is only safe to call if the state buffer is known to be correctly initialized already (see above comment on resetStream_fast for a definition of "correctly initialized"). From a high level, the difference is that this function initializes the provided state with a call to something like resetStream_fast while compress_fast_extState starts with a call to resetStream.

      • nLZ4_attach_dictionary

        public static void nLZ4_attach_dictionary​(long working_stream,
                                                  long dictionary_stream)
        Unsafe version of: attach_dictionary
      • LZ4_attach_dictionary

        public static void LZ4_attach_dictionary​(long working_stream,
                                                 long dictionary_stream)
        This is an experimental API that allows for the efficient use of a static dictionary many times.

        Rather than re-loading the dictionary buffer into a working context before each compression, or copying a pre-loaded dictionary's LZ4_stream_t into a working LZ4_stream_t, this function introduces a no-copy setup mechanism, in which the working stream references the dictionary stream in-place.

        Several assumptions are made about the state of the dictionary stream. Currently, only streams which have been prepared by loadDict should be expected to work.

        Alternatively, the provided dictionary stream pointer may be NULL, in which case any existing dictionary stream is unset.

        If a dictionary is provided, it replaces any pre-existing stream history. The dictionary contents are the only history that can be referenced and logically immediately precede the data compressed in the first subsequent compression call.

        The dictionary will only remain attached to the working stream through the first compression call, at the end of which it is cleared. The dictionary stream (and source buffer) must remain in-place / accessible / unchanged through the completion of the first compression call on the stream.

      • LZ4_DECODER_RING_BUFFER_SIZE

        public static int LZ4_DECODER_RING_BUFFER_SIZE​(int mbs)
        For static allocation; mbs presumed valid.