Postprocessors

Postprocessors operate on the raw data stream and transform it. They are run after data reaches the codec and do not know or care about tremor's internal representation.

Online codecs and postprocessors can be chained to perform multiple operations in succession.

Configuration

Postprocessors can be configured by specifying them by name or by name and config. Check out the following example that exhibits both forms:

define connector my_file from file
with
    codec = "json",
    postprocessors = [
        "base64",
        {
            "name": "separate",
            "config": {
                "separator": "|"
            }
        }
    ],
    config = {
        "path": "out.json",
        "mode": "truncate"
    }
end;

Supported Postprocessors

PostprocessorName	Description
base64	Base64 Encoding
chunk	Produce chunks up to a configurable size in bytes
compress	Compression Algorithms
gelf-chunking	Graylog Extended Log Format (GELF) chunking
ingest-ns	Prepend the ingest timestamp
length-prefixed	Length prefixed data
separate	Insert a separator between single event bytes
textual-length-prefix	Textual length prefixed data