Return a tuple, (total_input, total_output), the items are int objects.
Parameters input_stream: Input stream that has a .readinto(b) method. output_stream: Output stream that has a .write(b) method. If use callback function, this parameter can be None. zstd_dict: A ZstdDict object, pre-trained zstd dictionary. option: A dict object, contains advanced decompression parameters. read_size: Input buffer size, in bytes. write_size: Output buffer size, in bytes. callback: A callback function that accepts four parameters: (total_input, total_output, read_data, write_data), the first two are int objects, the last two are readonly memoryview objects.compress_stream(input_stream, output_stream, *, level_or_option=None, zstd_dict=None, pledged_input_size=None, read_size=131072, write_size=131591, callback=None) ---- Compresses input_stream and writes the compressed data to output_stream, it doesn't close the streams.
If input stream is b'', nothing will be written to output stream.
Return a tuple, (total_input, total_output), the items are int objects.
Parameters input_stream: Input stream that has a .readinto(b) method. output_stream: Output stream that has a .write(b) method. If use callback function, this parameter can be None. level_or_option: When it's an int object, it represents the compression level. When it's a dict object, it contains advanced compression parameters. zstd_dict: A ZstdDict object, pre-trained zstd dictionary. pledged_input_size: If set this parameter to the size of input data, the size will be written into the frame header. If the actual input data doesn't match it, a ZstdError will be raised. read_size: Input buffer size, in bytes. write_size: Output buffer size, in bytes. callback: A callback function that accepts four parameters: (total_input, total_output, read_data, write_data), the first two are int objects, the last two are readonly memoryview objects.Internal function, get zstd frame infomation from a frame header.get_frame_size(frame_buffer) ---- Get the size of a zstd frame, including frame header and 4-byte checksum if it has.
It will iterate all blocks' header within a frame, to accumulate the frame size.
Parameter frame_buffer: A bytes-like object, it should starts from the beginning of a frame, and contains at least one complete frame.Internal function, get CParameter/DParameter bounds.decompress(data, zstd_dict=None, option=None) ---- Decompress a zstd data, return a bytes object.
Support multiple concatenated frames.
Parameters data: A bytes-like object, compressed zstd data. zstd_dict: A ZstdDict object, pre-trained zstd dictionary. option: A dict object, contains advanced decompression parameters.True when both the input and output streams are at a frame edge, means a frame is completely decoded and fully flushed, or the decompressor just be initialized.
This flag could be used to check data integrity in some cases.decompress(data, max_length=-1) ---- Decompress data, return a chunk of decompressed data if possible, or b'' otherwise.
Parameters data: A bytes-like object, zstd data to be decompressed. max_length: Maximum size of returned data. When it is negative, the size of output buffer is unlimited. When it is nonnegative, returns at most max_length bytes of decompressed data.A streaming decompressor, accepts multiple concatenated frames. Thread-safe at method level.
EndlessZstdDecompressor.__init__(self, zstd_dict=None, option=None) ---- Initialize an EndlessZstdDecompressor object.
Parameters zstd_dict: A ZstdDict object, pre-trained zstd dictionary. option: A dict object that contains advanced decompression parameters.A bytes object. When ZstdDecompressor object stops after a frame is decompressed, unused input data after the frame. Otherwise this will be b''.If the max_length output limit in .decompress() method has been reached, and the decompressor has (or may has) unconsumed input data, it will be set to False. In this case, pass b'' to .decompress() method may output further data.True means the end of the first frame has been reached. If decompress data after that, an EOFError exception will be raised.decompress(data, max_length=-1) ---- Decompress data, return a chunk of decompressed data if possible, or b'' otherwise.
It stops after a frame is decompressed.
Parameters data: A bytes-like object, zstd data to be decompressed. max_length: Maximum size of returned data. When it is negative, the size of output buffer is unlimited. When it is nonnegative, returns at most max_length bytes of decompressed data.A streaming decompressor, it stops after a frame is decompressed. Thread-safe at method level.
ZstdDecompressor.__init__(self, zstd_dict=None, option=None) ---- Initialize a ZstdDecompressor object.
Parameters zstd_dict: A ZstdDict object, pre-trained zstd dictionary. option: A dict object that contains advanced decompression parameters.A compressor use rich memory mode. It is designed to allocate more memory, but faster in some cases.
RichMemZstdCompressor.__init__(self, level_or_option=None, zstd_dict=None) ---- Initialize a RichMemZstdCompressor object.
Parameters level_or_option: When it's an int object, it represents the compression level. When it's a dict object, it contains advanced compression parameters. zstd_dict: A ZstdDict object, pre-trained zstd dictionary.compress(data) ---- Compress data using rich memory mode, return a single zstd frame.
Compressing b'' will get an empty content frame (9 bytes or more).
Parameter data: A bytes-like object, data to be compressed.The last mode used to this compressor object, its value can be .CONTINUE, .FLUSH_BLOCK, .FLUSH_FRAME. Initialized to .FLUSH_FRAME.
It can be used to get the current state of a compressor, such as, data flushed, a frame ended._set_pledged_input_size(size) ---- *This is an undocumented method, because it may be used incorrectly.*
Set uncompressed content size of a frame, the size will be written into the frame header. 1, If called when (.last_mode != .FLUSH_FRAME), a RuntimeError will be raised. 2, If the actual size doesn't match the value, a ZstdError will be raised, and the last compressed chunk is likely to be lost. 3, The size is only valid for one frame, then it restores to "unknown size".
Parameter size: Uncompressed content size of a frame, None means "unknown size".flush(mode=ZstdCompressor.FLUSH_FRAME) ---- Flush any remaining data in internal buffer.
Since zstd data consists of one or more independent frames, the compressor object can still be used after this method is called.
Parameter mode: Can be these 2 values .FLUSH_FRAME, .FLUSH_BLOCK.compress(data, mode=ZstdCompressor.CONTINUE) ---- Provide data to the compressor object. Return a chunk of compressed data if possible, or b'' otherwise.
Parameters data: A bytes-like object, data to be compressed. mode: Can be these 3 values .CONTINUE, .FLUSH_BLOCK, .FLUSH_FRAME.A streaming compressor. Thread-safe at method level.
ZstdCompressor.__init__(self, level_or_option=None, zstd_dict=None) ---- Initialize a ZstdCompressor object.
Parameters level_or_option: When it's an int object, it represents the compression level. When it's a dict object, it contains advanced compression parameters. zstd_dict: A ZstdDict object, pre-trained zstd dictionary.Internal function, finalize a zstd dictionary.Internal function, train a zstd dictionary.The content of zstd dictionary, a bytes object, it's the same as dict_content argument in ZstdDict.__init__() method. It can be used with other programs.ID of zstd dictionary, a 32-bit unsigned int value.
Non-zero means ordinary dictionary, was created by zstd functions, follow a specified format.
0 means a "raw content" dictionary, free of any format restriction, used for advanced user.Zstd dictionary, used for compression/decompression.
ZstdDict.__init__(self, dict_content, is_raw=False) ---- Initialize a ZstdDict object.
Parameters dict_content: A bytes-like object, dictionary's content. is_raw: This parameter is for advanced user. True means dict_content argument is a "raw content" dictionary, free of any format restriction. False means dict_content argument is an ordinary zstd dictionary, was created by zstd functions, follow a specified format.Intentionally not supporting pickle.__init__ method is called twice.dwindowLogMaxdcompressionLevelewindowLogfhashLoggchainLoghsearchLogiminMatchjtargetLengthkstrategy�enableLongDistanceMatching�ldmHashLog�ldmMinMatch�ldmBucketSizeLog�ldmHashRateLog�contentSizeFlag�checksumFlag�dictIDFlag�nbWorkers�jobSize�overlapLog�@�Unable to allocate output buffer.�Unspecified error codeNo error detectedUnknown frame descriptorVersion not supportedUnsupported frame parameterCorrupted block detectedUnsupported parameterParameter is out of boundContext should be init firstDictionary is corruptedDictionary mismatchSrc size is incorrectFrame index is too largeDestination buffer is wrongSource buffer is wrongError (generic)Frame requires too much memory for decodingRestored data doesn't match checksumAllocation error : not enough memoryworkSpace buffer is not large enoughOperation not authorized at current processing stagetableLog requires too much memory : unsupportedUnsupported max Symbol Value : too largeSpecified maxSymbolValue is too smallCannot create Dictionary from provided samplesDestination buffer is too smallOperation on NULL destination bufferAn I/O error occurred when reading/seeking���$���L���L���L���L���L���L���L���L������L������L������L�������L���L���L����L�����L���L���L���L���L���L���L�����L���܂��L���Ԃ��L���L���L���L���L���̂��L���Ă��L�������L�������L�������L���L���L���L���L���L���L���L���L���L���L�������L�������L�������L�������L���L���L�������L���|���L���t���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���L���l���L���d���L���\���T���H���H���8���-���"������������Ȏ��Ȏ��������������������������ɕ��ɕ����������������������|�����������q���e���Y���M���A���5�����������x���m���b���W���L���A�������������������ܧ��ѧ��(D#$w��ֵ���'�`O��'=���1.5.2k8 ���pd`� �q0�?�������?���������������?�����������������������?�������t�����Q����t��z�����6��� ��������������������h���h���h���h���h��ph��`h��Ph���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f��@h��0h�� h��h���g���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���f���g���g��h��f���e���e���e���e���e���e���e���e��|e��f��$f��������������������������$�p�p������� �������������������������������������������������������p������������������������������������������������������|�d�����<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<���������l�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�<�T�L����`�H�0������H���p�����
�������Total samples size is too large (%u MB), maximum size is %u MB Total number of training samples is %u and is invalid.Total number of testing samples is %u and is invalid.Training on %u samples of total size %u Testing on %u samples of total size %u Failed to allocate scratch buffers Constructing partial suffix array WARNING: The maximum dictionary size %u is too large compared to the source size %u! size(source)/size(dictionary) = %f, but it should be >= 10! This may lead to a subpar dictionary! We recommend training on sources at least 10x, and preferably 100x the size of the dictionary! Breaking content into %u epochs of size %u Cover must have at least one input file dictBufferCapacity must be at least %u Failed to allocate dmer map: out of memory Constructed dictionary of size %u Failed to allocate buffers: out of memory Trying %u different sets of parameters Failed to allocate parameters Computing frequencies
%u%% %79s Cover parameters incorrect Building dictionary Failed to select dictionary Incorrect parameters d=%u Failed to initialize context k=%u �?$@Y@ #'*-02579;=@ACEGIKLNPQSTVWYZ[]^`abcefghjklmnpqrstuvwxyz{|}~����������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������������Total number of training samples is %u and is invalid Total number of testing samples is %u and is invalid. Failed to allocate scratch buffers Failed to allocate frequency table FASTCOVER parameters incorrect FASTCOVER must have at least one input file Incorrect splitPoint Incorrect accel Incorrect k dd2"
�?found %3u matches of length >= %i at pos %7u Selected dict at position %u, of length %u : saves %u (ratio: %.2f) warning : ZSTD_compressBegin_usingCDict failed warning : could not compress sample size %u warning : pathological dataset : literals are not compressible : samples are noisy or too regular FSE_normalizeCount error with offcodeCount FSE_normalizeCount error with matchLengthCount FSE_normalizeCount error with litLengthCount FSE_writeNCount error with offcodeNCount FSE_writeNCount error with matchLengthNCount FSE_writeNCount error with litlengthNCount not enough space to write RepOffsets sample set too large : reduced to %u MB ... sorting %u files of total size %u MB ... %3u:%3u bytes at pos %8u, savings %7u bytes |! warning : selected content significantly smaller than requested (%u < %u) ! consider increasing the number of samples (total size : %u MB) ! consider increasing selectivity to produce larger dictionary (-s%u) ! note : larger dictionaries are not necessarily better, test its efficiency on samples ! note : calculated dictionary significantly larger than requested (%u > %u) ! consider increasing dictionary size, or produce denser dictionary (-s%u) ! always test dictionary efficiency on real samples
%u segments found, of total size %u Not enough memory Offset Code Frequencies : %2u :%7u HUF_buildCTable error HUF_writeCTable error
%70s statistics ... finding patterns ... minimum ratio : %u