srsly is a fast and lightweight JSON serialization library from Explosion.
It eliminates a lot of boilerplate for util functions that read/write compressed JSONL files (in gzip format)
Using this library each bulk indexing script is very simple, doesn't add much overhead to the pip install time, and reduces the number of lines of code quite significantly
The code base for Elasticsearch, Meilisearch and Neo4j have all been updated to use srsly to read gzipped JSONL
For future DBs, the same approach will be used to also keep things clean and readable
For Meilisearch, the settings specification is moved over to a settings.json to keep things clean and easy to find all in one place
Updates
srslyis a fast and lightweight JSON serialization library from Explosion.pip installtime, and reduces the number of lines of code quite significantlysrslyto read gzipped JSONLFor Meilisearch, the settings specification is moved over to a
settings.jsonto keep things clean and easy to find all in one place