Closed thcrock closed 6 years ago
In the beta we've hit at least one instance of low-level (meaning, C system library) database communication being interrupted. Obviously we can't ensure that this will never happen, but we can be a little bit more resilient to it.
In this case, it was during the sync_merged_file_to_storage routine, which performs a Postgres COPY of a large file to S3 or Filesystem. This can be retried, perhaps on the outer level: https://github.com/dssg/matching-tool/blob/master/webapp/webapp/apis/upload.py#L360
sync_merged_file_to_storage
The error manifests itself as a psycopg2.DatabaseError: psycopg2.DatabaseError: error with status PGRES_COPY_OUT and no message from the libpq
psycopg2.DatabaseError: error with status PGRES_COPY_OUT and no message from the libpq
In the beta we've hit at least one instance of low-level (meaning, C system library) database communication being interrupted. Obviously we can't ensure that this will never happen, but we can be a little bit more resilient to it.
In this case, it was during the
sync_merged_file_to_storageroutine, which performs a Postgres COPY of a large file to S3 or Filesystem. This can be retried, perhaps on the outer level: https://github.com/dssg/matching-tool/blob/master/webapp/webapp/apis/upload.py#L360The error manifests itself as a psycopg2.DatabaseError:
psycopg2.DatabaseError: error with status PGRES_COPY_OUT and no message from the libpq