search

adevinta / zoe

The Kafka CLI for humans
https://adevinta.github.io/zoe
MIT License
286 stars 21 forks source link

Specifying clientId? Or: How to continue reading from a stream? #38

Closed haraldkubota closed 3 years ago

haraldkubota commented 3 years ago

Maybe this is just a documentation problem or a problem where the user (me) just missed it, but I seem to not be able to specify a clientId or a method to continue a stream. If I read from a stream, it starts from the start of time:

❯ zoe --silent topics consume pageviews -n 5
{"viewtime":1,"userid":"User_1","pageid":"Page_40"}
{"viewtime":11,"userid":"User_7","pageid":"Page_16"}
{"viewtime":21,"userid":"User_4","pageid":"Page_19"}
{"viewtime":31,"userid":"User_4","pageid":"Page_90"}
{"viewtime":41,"userid":"User_8","pageid":"Page_78"}

If I repeat this, I get the same result and not the next events. However I cannot find out how to stop this behavior. Setting a clientId would work as would a "from" parameter, but not via a time but instead a seek value. I guess I am missing something, e.g. what the X in --from="X" can be or how to set a clientId (if this even does make any sense).

wlezzar commented 3 years ago

Hi @haraldkubota! Zoe doesn't support consumer groups now so you cannot specify a consumer group to continue reading from a stream.

However, you can use --from to specify whhere you want to start reading from. Example: --from 'PT5m' lets you read messages received since 5 minutes ago.

The format expected by --from is a duration in the iso-8601 format. Some examples are included here:

    "PT20.345S" -- parses as "20.345 seconds"
    "PT15M"     -- parses as "15 minutes" (where a minute is 60 seconds)
    "PT10H"     -- parses as "10 hours" (where an hour is 3600 seconds)
    "P2D"       -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
    "P2DT3H4M"  -- parses as "2 days, 3 hours and 4 minutes"
    "PT-6H3M"    -- parses as "-6 hours and +3 minutes"
    "-PT6H3M"    -- parses as "-6 hours and -3 minutes"
    "-PT-6H+3M"  -- parses as "+6 hours and -3 minutes"

Does this answer your question?

haraldkubota commented 3 years ago

Thanks, especially for the link for the ISO-8601. I've never seen that before.