search

mostafa / xk6-kafka

k6 extension to load test Apache Kafka with support for various serialization formats, SASL, TLS, compression, Schema Registry client and beyond
Apache License 2.0
156 stars 70 forks source link

Add millis to consumed messages #260

Open romankristof opened 1 year ago

romankristof commented 1 year ago

Hi @mostafa, my scenario is that I am sending kafka requests with Writer to an tested app and I consume response messages with Reader. I want to measure the duration between sending the message until the app writes a response into result topic. I guess the time field on the message object coming from the Reader would be the right timestamp to collect as an time when the response was published, but since it is formated in RFC3339 format, I am missing milliseconds. Would it be possible to use different format that includes millis? Or maybe add new reponse field to keep backwards compatibility?

Thanks a lot for your time and work Roman

mostafa commented 1 year ago

Hey @romankristof,

That is to make Goja (and the user) happy when exporting datetime to JSON, so you can use it in your JS code. I'd be happy to see contributions, though. As in, make this a configurable field on the consume function using the ConsumeConfig struct, which eventually gets passed down to the consume function.

AndreasKorn commented 8 months ago

As long as there is no way to get the original (meaning with full precision) timestamp of the consumed message this is unusable to measure timings or generate metrics for the messages. Which is actually what one would like to do in load tests. Please add a way to get the message timestamp with better precision.

mostafa commented 8 months ago

Hey @AndreasKorn,

I'd be happy if you contribute this feature.

ChrisDev83 commented 7 months ago

A new consumer option for nanos precision was added. You can set nanoPrecision: true in the consumer configuration in order to have the time return in RFC3339Nano format.

Time field by default = "time": "2024-04-10T14:13:46+01:00" With nanoPrecision: true = "time": "2024-04-10T13:08:49.127Z"

To have more flexibility this could be change further to allow specification of the exact format (as per @mostafa). We had questions around the format, whether it be javascript or go formats. Or constants for each format type.