Package org.lwjgl.util.zstd

Class Zdict

  • public class Zdict
extends java.lang.Object
    Native bindings to the dictionary builder API of Zstandard (zstd).

    • Method Detail

      • nZDICT_trainFromBuffer

        public static long nZDICT_trainFromBuffer​(long dictBuffer,
                                          long dictBufferCapacity,
                                          long samplesBuffer,
                                          long samplesSizes,
                                          int nbSamples)
        Unsafe version of: trainFromBuffer

      • ZDICT_trainFromBuffer

        public static long ZDICT_trainFromBuffer​(java.nio.ByteBuffer dictBuffer,
                                         java.nio.ByteBuffer samplesBuffer,
                                         PointerBuffer samplesSizes)
        Train a dictionary from an array of samples.

        Redirect towards optimizeTrainFromBuffer_cover single-threaded, with d=8 and steps=4.

        Samples must be stored concatenated in a single flat buffer samplesBuffer, supplied with an array of sizes samplesSizes, providing the size of each sample, in order.

        The resulting dictionary will be saved into dictBuffer.

        Note: ZDICT_trainFromBuffer() requires about 9 bytes of memory for each input byte.

        Tips:

        • In general, a reasonable dictionary has a size of ~ 100 KB.
        • It's possible to select smaller or larger size, just by specifying dictBufferCapacity.
        • In general, it's recommended to provide a few thousands samples, though this can vary a lot.
        • It's recommended that total size of all samples be about ~x100 times the target size of dictionary.
        Returns:
        size of dictionary stored into dictBuffer (≤ dictBufferCapacity) or an error code, which can be tested with isError.

      • nZDICT_getDictID

        public static int nZDICT_getDictID​(long dictBuffer,
                                   long dictSize)
        Unsafe version of: getDictID

      • ZDICT_getDictID

        public static int ZDICT_getDictID​(java.nio.ByteBuffer dictBuffer)
        Extracts dictID.
        Returns:
        zero if error (not a valid dictionary)

      • nZDICT_isError

        public static int nZDICT_isError​(long errorCode)

      • ZDICT_isError

        public static boolean ZDICT_isError​(long errorCode)

      • nZDICT_getErrorName

        public static long nZDICT_getErrorName​(long errorCode)

      • ZDICT_getErrorName

        @Nullable
public static java.lang.String ZDICT_getErrorName​(long errorCode)

      • nZDICT_trainFromBuffer_cover

        public static long nZDICT_trainFromBuffer_cover​(long dictBuffer,
                                                long dictBufferCapacity,
                                                long samplesBuffer,
                                                long samplesSizes,
                                                int nbSamples,
                                                long parameters)
        Unsafe version of: trainFromBuffer_cover

      • ZDICT_trainFromBuffer_cover

        public static long ZDICT_trainFromBuffer_cover​(java.nio.ByteBuffer dictBuffer,
                                               java.nio.ByteBuffer samplesBuffer,
                                               PointerBuffer samplesSizes,
                                               ZDICTCoverParams parameters)
        Train a dictionary from an array of samples using the COVER algorithm.

        Samples must be stored concatenated in a single flat buffer samplesBuffer, supplied with an array of sizes samplesSizes, providing the size of each sample, in order.

        The resulting dictionary will be saved into dictBuffer.

        Note: ZDICT_trainFromBuffer_cover() requires about 9 bytes of memory for each input byte.

        Tips:

        • In general, a reasonable dictionary has a size of ~ 100 KB.
        • It's possible to select smaller or larger szie, just by specifying dictBufferCapacity.
        • In general, it's recommended to provide a few thousands samples, though this can vary a lot.
        • It's recommended that total size of all samples be about ~x100 times the target size of dictionary.
        Returns:
        size of dictionary stored into dictBuffer (≤ dictBufferCapacity) or an error code, which can be tested with isError.

      • nZDICT_optimizeTrainFromBuffer_cover

        public static long nZDICT_optimizeTrainFromBuffer_cover​(long dictBuffer,
                                                        long dictBufferCapacity,
                                                        long samplesBuffer,
                                                        long samplesSizes,
                                                        int nbSamples,
                                                        long parameters)
        Unsafe version of: optimizeTrainFromBuffer_cover

      • ZDICT_optimizeTrainFromBuffer_cover

        public static long ZDICT_optimizeTrainFromBuffer_cover​(java.nio.ByteBuffer dictBuffer,
                                                       java.nio.ByteBuffer samplesBuffer,
                                                       PointerBuffer samplesSizes,
                                                       ZDICTCoverParams parameters)
        The same requirements as trainFromBuffer_cover hold for all the parameters except parameters.

        This function tries many parameter combinations and picks the best parameters. *parameters is filled with the best parameters found, dictionary constructed with those parameters is stored in dictBuffer.

        • All of the parameters d, k, steps are optional.
        • If d is non-zero then we don't check multiple values of d, otherwise we check d = {6, 8, 10, 12, 14, 16}.
        • If steps is zero it defaults to its default value.
        • If k is non-zero then we don't check multiple values of k, otherwise we check steps values in [16, 2048].

        Note: ZDICT_optimizeTrainFromBuffer_cover() requires about 8 bytes of memory for each input byte and additionally another 5 bytes of memory for each byte of memory for each thread.

        Returns:
        size of dictionary stored into dictBuffer (≤ dictBufferCapacity) or an error code, which can be tested with isError. On success *parameters contains the parameters selected.

      • nZDICT_finalizeDictionary

        public static long nZDICT_finalizeDictionary​(long dictBuffer,
                                             long dictBufferCapacity,
                                             long dictContent,
                                             long dictContentSize,
                                             long samplesBuffer,
                                             long samplesSizes,
                                             int nbSamples,
                                             long parameters)
        Unsafe version of: finalizeDictionary

      • ZDICT_finalizeDictionary

        public static long ZDICT_finalizeDictionary​(java.nio.ByteBuffer dictBuffer,
                                            java.nio.ByteBuffer dictContent,
                                            java.nio.ByteBuffer samplesBuffer,
                                            PointerBuffer samplesSizes,
                                            ZDICTParams parameters)
        Given a custom content as a basis for dictionary, and a set of samples, finalize dictionary by adding headers and statistics.

        Samples must be stored concatenated in a flat buffer samplesBuffer, supplied with an array of sizes samplesSizes, providing the size of each sample in order.

        Notes:

        • dictContentSize must be ≥ CONTENTSIZE_MIN bytes.
        • maxDictSize must be ≥ dictContentSize, and must be ≥ DICTSIZE_MIN bytes.
        • ZDICT_finalizeDictionary() will push notifications into stderr if instructed to, using notificationLevel>0.
        • dictBuffer and dictContent can overlap.
        Returns:
        size of dictionary stored into dictBuffer (≤ dictBufferCapacity) or an error code, which can be tested with isError.