search

oss-aspen / 8Knot

Dash app in development to serve open source community visualizations using GitHub data from Augur. Hosted app: https://eightknot.osci.io
MIT License
47 stars 59 forks source link

Switch from JSON-as-strings to JSON-native caching in Redis #427

Closed JamesKunstle closed 11 months ago

JamesKunstle commented 1 year ago

Redis supports JSON datatypes with the RedisJSON module. Right now, we serialize JSON data into Arrow format to store it as a read-efficient blob. We could move to a JSON-native caching solution in the future, removing de/serialization and saving time.

We're not especially sensitive to de/serialization latency but this is a minor optimization we can make in the future.

https://redis.io/docs/data-types/json/#run-with-docker

JamesKunstle commented 11 months ago

closing issue because RedisJSON isn't the right tool for our use case. we'll probably be using MongoDB for data caching and redis for rendered visualization caching.