Fields

Fields are data components within a chunk. When a chunk populating, data inside are parsed and populated by the order of their defination in Chunk.Fields. Then we can access them as attributes of the chunk by their names.

Hint

Feel free to send me pull requests on GitHub if fields listed here cannot meet your needs.

Example:

class DataChunk(Chunk):
    # Fields are parsed in this order
    Fields = (
        UnsignedLongField('data_length'),
        StringField('data', length_field_name='data_length'),
        UnsignedLongField('extra_length'),
        SkipBasedOnLengthField('extra', length_field_name='extra_length'),
    )

class fields.Field(name)

Base field class.

It’s abstract and cannot be used directly.

Parameters:name – Field name.

Fixed Length Fields

Fixed length fields are used for simple data with fixed length.

class fields.BytesField(name, fmt, length)

Base class for fixed length fields.

It reads fixed count of bytes and decode them using struct.unpack().

Parameters:

• name – Field name.
• fmt – Format to be passed to struct.unpack().
• length – Field length in bytes.

class fields.UnsignedLongField(name, big_endian=False)

Read 4 bytes and convert them into an unsigned long value.

Parameters:

• name – Field name.
• big_endian – If should be decoded in big-endian bytes order. Default is False.

class fields.UnsignedShortField(name, big_endian=False)

Read 2 bytes and convert them into an unsigned short value.

Parameters:

• name – Field name.
• big_endian – If should be decoded in big-endian bytes order. Default is False.

class fields.UnsignedCharField(name, big_endian=False)

Read 1 byte and convert them into an unsigned char value.

Parameters:

• name – Field name.
• big_endian – If should be decoded in big-endian bytes order. Default is False.

Variable Length Fields

Variable length fields are used for data with variable length. They usually rely on other fields to provide their lengths.

Example:

class DataChunk(Chunk):
    Fields = (
        UnsignedLongField('length'),
        StringField('data', length_field_name='length'),
    )

class fields.VariableLengthField(name, length_field_name)

Base class for field with variable length. Its length is based on another field.

When populating, it simply read length and doesn’t move on fp or read any data.

Parameters:

• name – Field name.
• length_field_name – Field name in the same chunk indicating the length of this field.

class fields.StringField(name, length_field_name)

Read data with variable length. Its length is based on another field.

Parameters:

• name – Field name.
• length_field_name – Field name in the same chunk indicating the length of this field.

Skip Fields

Sometimes we are not interested in all fields and only want to extract part of them. In these conditions, we can use skip fields as placeholders. Skip fields simply move forward without reading any bytes so that we can save memory and parse faster.

Example:

class DataChunk(Chunk):
    Fields = (
        UnsignedLongField('data_length'),

        # Skip data whose length is extracted from 'data_length' field.
        SkipBasedOnLengthField('data', length_field_name='data_length'),

        # Skip data whose length is 3 times of the 'data_length'.
        SkipBasedOnCalcField('detailed_data', calc_func=lambda c: 3 * c.data_length),

        # Skip data left to end of the file object.
        SkipToTheEndField('extra'),
    )

class fields.SkipBasedOnLengthField(name, length_field_name)

Skip data with variable length. Its length is based on another field.

Parameters:

• name – Field name.
• length_field_name – Field name in the same chunk indicating the length of this field.

class fields.SkipBasedOnCalcField(name, calc_func)

Skip data with variable length. Its length is calculated from calc_func().

Parameters:

• name – Field name.
• calc_func – Function to calculate the length to skip. When populating, this chunk will be passed to it as a parameter.

Example:

Fields = (
    UnsignedLongField('data_length'),
    SkipBasedOnCalcField('skip', lambda c: 3 * c.data_length),
)

class fields.SkipToTheEndField(name)

Skip data to the end.

The end depends on fp.total_length.

Parameters:name – Field name.