search

juhalehtonen / towwwer

Continuous, automated website performance monitoring
MIT License
1 stars 0 forks source link

Caching saved data #20

Open juhalehtonen opened 5 years ago

juhalehtonen commented 5 years ago

Is your feature request related to a problem? Please describe. Currently the web UI gets slow fast when data accumulates, because every interaction with the service taxes the database.

Describe the solution you'd like Use Mnesia or ETS to create a caching layer that is used to serve the content to users. Automatically refresh a given site's cache when the site is updated.

Describe alternatives you've considered None. We could just not build the cache, but that sucks.

juhalehtonen commented 5 years ago

It's worth noting that in prod we can be quite starved of RAM, so using a disk-based solution might be the best way to go here.

juhalehtonen commented 5 years ago

Could also just use nginx https://www.nginx.com/resources/wiki/start/topics/examples/reverseproxycachingexample/

juhalehtonen commented 5 years ago

Added some caching via https://github.com/juhalehtonen/towwwer/commit/be813da9595016f45f94f5038e2a18815b699751 , but this still needs work.

Most relevantly, the caches are currently not cleared when data changes.

Could possibly use con_cache directly via https://github.com/sasa1977/con_cache/blob/master/lib/con_cache/operations.ex to call the delete function against our cache when data changes.