search

tricorder-observability / Starship

Starship: next-generation Observability platform built with eBPF+WASM
GNU Affero General Public License v3.0
164 stars 24 forks source link

WASM: Create C API for generating length-prefixed fields for WASM C guest code to output data records #11

Open nascentcore-eng opened 1 year ago

nascentcore-eng commented 1 year ago

Right now, WASM C guest uses cJSON APIs to serialize C struct into JSON string, and then write the JSON string as output to WASM host environment. Tricorder then writes JSON string a JSONB column type to postgreSQL DB.

Choosing JSON as serialization format is because of it's self-explanatory. In other words, Tricorder does not need to provide human-readable enrichment to the output data.

We want to provide length-prefixed serialization, with the data schema specified by the users on the management Web UI. This would be an alternative option to JSON.

nascentcore-eng commented 1 year ago

The schema of the output from WASM should match the actual data as shown in the code. Length prefixed data allow Tricorder to create data table accordingly on the Observability data store (right now is timescale db)