Closed dwillis closed 6 years ago
Can you post the command line arguments this failed on? Or any dates that this bug occurs for.
Sure thing. This fails for a handful of dates so far. Among them: 2016-09-13 and 2016-09-12. The command:
python3 manage.py run_crec_parser --start_date=2016-09-11 --end_date=2016-09-13
@dwillis
I wasn't able to reproduce this on my laptop, but that has 16gb of memory so its possible that the days that trigger this error just have a larger than normal amount of text to process. So, I would first try running this on a machine with more ram if you haven't already done so. Alternatively, you can try running it with an older version of spacy ("pip install spacy<2.0") as this may be related to an issue in the newer version (nothing we're doing in the capitol words code requires any newer features).
@will-horning Ok, thanks! I'll try both of those options.
For some dates, a memory error occurs when parsing that day's Congressional Record files: