corgibytes / freshli-lib

A tool for collecting historical metrics about a project's dependencies
MIT License
17 stars 1 forks source link

Use local database for caching #26

Open mscottford opened 4 years ago

mscottford commented 4 years ago

I'd like to use a local database for caching data that we query from remote sources. This will improve performance while also reducing the chances that we are overloading those remote sources for data that hasn't changed.

I'd like to use LiteDB for this effort: https://github.com/mbdavid/litedb. And I think it would be great to store the database at ~/.freshli/cache.db by default but give users the ability to override that.

We could also change the application/library to use that datastore for querying information. There's a chance that doing so would be faster than the way we're doing it now, at least if it supports indexes.

I think that the following information would be cached here:

mscottford commented 4 years ago

And when we start grabbing vulnerability data, that would go in there, too.