unfoldingWord-dev / d43-catalog

Lambda functions for the Door43 Catalog.
https://api.door43.org/v3/catalog
MIT License
1 stars 8 forks source link

The ts catalog converter is running out of space #60

Closed da1nerd closed 4 years ago

da1nerd commented 4 years ago

CloudWatch is giving the following error in the logs after processing a large number of resources.

[ERROR] 2019-10-14T09:51:45.856Z 586ac9b0-658c-45a3-a279-3eea53bb6a38 [Errno 28] No space left on device
Traceback (most recent call last):
File "/var/task/libraries/lambda_handlers/handler.py", line 170, in run
return self._run(**kwargs)
File "/var/task/libraries/lambda_handlers/ts_v2_catalog_handler.py", line 77, in _run
return self.__execute()
File "/var/task/libraries/lambda_handlers/ts_v2_catalog_handler.py", line 283, in __execute
api_uploads.append(prep_data_upload('
{}
/
{}
/resources.json'.format(pid, lid), res_cat, self.temp_dir))
File "/var/task/libraries/tools/ts_v2_utils.py", line 386, in prep_data_upload
write_file(temp_file, json.dumps(data, sort_keys=True))
File "/var/task/libraries/tools/file_utils.py", line 157, in write_file
out_file.write(text_to_write)
File "/usr/lib64/python2.7/codecs.py", line 708, in write
return self.writer.write(data)
File "/usr/lib64/python2.7/codecs.py", line 370, in write
self.stream.write(data)
Exception: [Errno 28] No space left on device