search

jpsingleton / Huxley2

A cross-platform JSON proxy for the GB railway Live Departure Boards SOAP API
https://huxley2.azurewebsites.net
European Union Public License 1.2
51 stars 74 forks source link

Implement ETag-based caching #4

Closed davwheat closed 2 years ago

davwheat commented 3 years ago

ETag caching is the future.

I have a project which accesses this API multiple times per minute while running, but most of the time the data returned by the API will be the same. Implementing ETag-based caching could save a lot of bandwidth, especially for length requests such as /crs

jpsingleton commented 2 years ago

Huxley is just a proxy and considered feature-complete. It doesn't hold state and doesn't know if data is stale or not. You would probably need to build something on top of the push port to implement this feature.

davwheat commented 2 years ago

There's no need to know if data is stale or not. The response just needs to be hashed before it is sent, and if the hash is the same as supplied in the ETag request header, Huxley can respond with a 304 code.

The objective isn't to prevent unnecessary API calls from the server to NRE, but to prevent unnecessary data transfer when no data has changed so the browser can use its own cached ETagged copy.

jpsingleton commented 2 years ago

Thanks for clarifying. Looks like Mads has a gist for some ETag middleware so I might see if that still works.

jpsingleton commented 2 years ago

I've added this but it's not quite that simple. NRE add a millisecond precision "generated at" timestamp to every response so the hash will always be different and so not useful as an ETag.

jpsingleton commented 2 years ago

This is available in the latest release, ignoring the generated at timestamp when hashing.

davwheat commented 2 years ago

Thank you so much! :)