mhart / kinesalite

An implementation of Amazon's Kinesis built on LevelDB
MIT License
809 stars 86 forks source link

Allow getShardIterator to support timestamps in `2016-04-04T19:58:46.480-00:00` format #70

Open chodyo opened 6 years ago

chodyo commented 6 years ago

The Kinesis docs specify that the parameter of the getShardIterator call using the "AT_TIMESTAMP" ShardIteratorType should be a timestamp in milliseconds. However, the Kinesalite server multiplies the supplied parameter by 1000. Since it is expecting milliseconds, there is no need to multiply.

This should be a simple fix: remove line 84 in getShardIterator.js and touch up the corresponding unit tests. If you agree, I'd love to get a pull request in for you to take a look at.

mhart commented 6 years ago

The docs say "with precision in milliseconds" – not in milliseconds.

The example they give, 1459799926.480 – is in seconds.

chodyo commented 6 years ago

Would it be viable for Kinesalite to add support to allow any valid timestamp, then? Before I started using this module I've been using the AWS API with timestamps in milliseconds, i.e.: 1459799926480 instead of 1459799926.480.

mhart commented 6 years ago

@chody-h can you show me some code? Did you mean one of the AWS SDKs, or are you calling the API manually?

chodyo commented 6 years ago

We were using the getShardIterator method in the AWS SDK for Javascript, which also requires a timestamp with precision in milliseconds. I'm not confident anymore about how the Kinesis API handles timestamps in milliseconds because we had some time-related inconsistencies for which we never identified the root cause. Using milliseconds instead of seconds may have been our problem.

The API also notes that it accepts timestamp strings in the format 2016-04-04T19:58:46.480-00:00. Are there any plans for Kinesalite to allow that as well?

mhart commented 6 years ago

@chody-h I can look into that when I get the chance. Have changed this issue title accordingly.

chodyo commented 6 years ago

Great, thanks for your time. Let me know if you'd like any help, I'd be happy to contribute.