Streamfold - Functions

The Parse function extracts structured data from an event field by parsing common data formats like JSON.

Usage

Operates On: Field values
Supported Field Types: string
Label: Optional
Filter: Optional

Required Configuration

Field Name: The field to parse data from.
Format: JSON. More parsers coming soon.

Optional Configuration

Optional Destination: type: string - path selector. Destination field name to write extracted fields. If unset will default to Source Field's parent.
Overwrite Optional Destination if it exists: True | False. When enabled will overwrite the Optional Destination if defined.

You should know!

Parse will only operate on a single string value as it only provides a singular destination. Therefore any field name selector that contains an array range, like foo.bar[:] that returns more than one item, even if that selector returns N string types, will be ignored.

Example

Parse fields from an event's message field into the JSON format.

Configuration

Field Name: message
Format: JSON

Before

{
  "hostname": "localhost",
  "message": "{\"level\":\"info\",\"ts\":1692835200.8646913,\"caller\":\"server/request.go:126\",\"msg\":\"Finished request\",\"version\":\"0.0.1-a8ac28e\",\"request\":{\"method\":\"POST\",\"path\":\"/api/v1/worker/ZH4C9WNFAVQ294AKBFNEC4FQ9M/heartbeat\",\"bytes\":141},\"id\":\"01H8JDEFRSN28EE4QES7MDHPP3\",\"peer_addr\":\"172.16.131.58:53448\",\"worker\":{\"org_id\":\"01H7AH93B29EW5SGVDVRQ76JWN\",\"guid\":\"ZH4C9WNFAVQ294AKBFNEC4FQ9M\",\"org_ref\":\"streamfold\"},\"resp\":{\"status\":200,\"bytes\":33},\"dur\":{\"pretty\":\"71.049354ms\",\"nano\":71049354}}"
}

After

{
  "hostname": "localhost",
  "message": {
    "level": "info",
    "ts": 1692835200.8646913,
    "caller": "server/request.go:126",
    "msg": "Finished request",
    "version": "0.0.1-a8ac28e",
    "request": {
      "method": "POST",
      "path" : "/api/v1/worker/ZH4C9WNFAVQ294AKBFNEC4FQ9M/heartbeat"
      ...
    }
    ...
  }
}