search

cthoyt / selventa-knowledge

Updated versions of the Selventa Small and Large Corpus
MIT License
0 stars 3 forks source link

Add gzipped json artifact for each corpus #4

Closed bgyori closed 5 years ago

bgyori commented 5 years ago

Currently, each corpus comes with the original .bel file containing the raw BEL script, and a .pickle file containing the processed PyBEL Graph, but for programmatic access over the web, loading pickle files directly is not secure. One alternative is to store PyBEL Graph JSON files in version control but they tend to be very large. I propose that a Gzipped version of the JSON (.json.gz) be stored in addition to (or instead of) the .pickle files.

For reference, the large corpus is 44 MB as .pickle, 94 MB as .json, and 19MB as .json.gz.

cthoyt commented 5 years ago

I updated the tox script with df28a60879b13c0a5d1b4cfe841f5194cb83565b to automatically generate the gz version of the json. In the next release of PyBEL, I'll include export of gzipped JSON as well (that doesn't have any unnecessary whitespace) which will get propogated to the BEL-Repository library