search

dylibso / wasm-practitioners-guide-to-observability

Wasm Observability Workshop
https://dev.dylibso.com/docs/observe/overview
3 stars 0 forks source link

go collector cannot handle large otel output #13

Open G4Vi opened 8 months ago

G4Vi commented 8 months ago

Happens attempting to run guest auto-instr'd go modules such as: src/guest/modules/automatic/go/main.debug.instr.wasm

Error does not occur when allowed functions have been configured (likely as output is much smaller then).

Length of /tmp/go-auto-debug is 6816442 bytes 2024/03/05 23:48:36 82054e25d011bc7e41a526c576092168 2024/03/05 23:48:36 stopped collector, sent to collector 2024/03/05 23:48:37 failed to upload wasm traces to otel endpoint rpc error: code = ResourceExhausted desc = grpc: received message larger than max (13529885 vs. 4194304)

nilslice commented 8 months ago

This could be a good way to show the utility of the Observe SDK SpanFilter option, which currently only includes a time-based filter - but would reduce the size of the trace sent. It's a good thing to mention especially when doing these profile-like traces, since they include so much data, it can put resource pressure and cost burden on the sink.

see: https://github.com/dylibso/wasm-practitioners-guide-to-observability/blob/8c2c26be1c7cd614a333725667f1a2b1350abbf5/src/host/main.go#L169-L171

G4Vi commented 8 months ago

5 microseconds worked! If you want SetName to show up on the go automatic config, lower back to 0.