Closed brainexe closed 2 years ago
We noticed two issues after the recent reactoring.
First issue: No errors are stores anymore:
From the log:
Using temp folder /www/ltc/temp/loadtest_1159 Using remote temp folder /tmp/loadtest_1159
-> "errors" were stored in /tmp/loadtest_1159/errors/ But the "analyze_errors" method uses the temp_path aka "/www/ltc/temp/" which does not have logs
Second issue: huge csv files are failing hard to parse:
File "/www/ltc/ltc/base/models.py", line 929, in parse_csv df = df[~df['url'].str.contains('exclude_', na=False)] File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 2688, in __getitem__ return self._getitem_column(key) ... KeyError: 'url'
We noticed two issues after the recent reactoring.
First issue: No errors are stores anymore:
From the log:
-> "errors" were stored in /tmp/loadtest_1159/errors/ But the "analyze_errors" method uses the temp_path aka "/www/ltc/temp/" which does not have logs
Second issue: huge csv files are failing hard to parse: