Closed wendyrogers closed 2 years ago
How large is the file?
11mb Ben thanks and have just sent it across W
Looks like this is due to the large size of the file. I should be able to convert this specific CSV for you by running the conversion locally.
Server log:
[Thu May 22 09:05:31 2014] [error] ERROR:csviatimodeleditor:Exception on /model/268/change_csv/ [POST]
[Thu May 22 09:05:31 2014] [error] Traceback (most recent call last):
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
[Thu May 22 09:05:31 2014] [error] response = self.full_dispatch_request()
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
[Thu May 22 09:05:31 2014] [error] rv = self.handle_user_exception(e)
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
[Thu May 22 09:05:31 2014] [error] reraise(exc_type, exc_value, tb)
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
[Thu May 22 09:05:31 2014] [error] rv = self.dispatch_request()
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
[Thu May 22 09:05:31 2014] [error] return self.view_functions[rule.endpoint](**req.view_args)
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/csviatimodeleditor/__init__.py", line 336, in model_change_csv
[Thu May 22 09:05:31 2014] [error] result = chardet.detect(reopen_for_decoding.read())
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/chardet/__init__.py", line 24, in detect
[Thu May 22 09:05:31 2014] [error] u.feed(aBuf)
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/chardet/universaldetector.py", line 115, in feed
[Thu May 22 09:05:31 2014] [error] if prober.feed(aBuf) == constants.eFoundIt:
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/chardet/charsetgroupprober.py", line 59, in feed
[Thu May 22 09:05:31 2014] [error] st = prober.feed(aBuf)
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/chardet/hebrewprober.py", line 217, in feed
[Thu May 22 09:05:31 2014] [error] aBuf = self.filter_high_bit_only(aBuf)
[Thu May 22 09:05:31 2014] [error] File "/home/pythonuser/CSV-IATI-Converter.modeleditor/pyenv/lib/python2.7/site-packages/chardet/charsetprober.py", line 51, in filter_high_bit_only
[Thu May 22 09:05:31 2014] [error] aBuf = re.sub(r'([\\x00-\\x7F])+', ' ', aBuf)
[Thu May 22 09:05:31 2014] [error] File "/usr/lib/python2.7/re.py", line 151, in sub
[Thu May 22 09:05:31 2014] [error] return _compile(pattern, flags).sub(repl, string, count)
[Thu May 22 09:05:31 2014] [error] MemoryError
Current approach to large files is to run the conversion locally on a machine with more resources. Therefore I am unassigning myself from this issue.
I have bee trying to upload a large CSV file of data fro Japan but the Convertor keeps failing.