BenEgeIzmirli / Whittler

MIT License
7 stars 2 forks source link

Use ijson if possible #16

Open BenEgeIzmirli opened 3 years ago

BenEgeIzmirli commented 3 years ago

Python's built-in JSON library loads the whole file into memory before parsing it. In many cases this is impractical. Using the stream-based ijson implementation would greatly improve memory efficiency while initializing the database.