When testing the pipeline via bulk import the backend seems to fall over and not log an error.
Here is the code in question and the import logs.
[https://github.com/climatepolicyradar/navigator/blob/main/backend/app/api/api_v1/routers/admin.py#L248](https://github.com/climatepolicyradar/navigator/blob/main/backend/app/api/api_v1/routers/admin.py#L248)
Above calls below.
[https://github.com/climatepolicyradar/navigator/blob/44bcb67396e0788f3c5d0c4be0578d51bb167fd5/backend/app/core/validation/util.py#L99](https://github.com/climatepolicyradar/navigator/blob/44bcb67396e0788f3c5d0c4be0578d51bb167fd5/backend/app/core/validation/util.py#L99)
Log errors:
Starts writing to csv, looks like the app falls over and then begins to start up again.
| { "written_at": "2022-12-21T12:57:21.698Z", "written_ts": 1671627441698805000, "msg": "Writing CSV file into S3", "type": "log", "logger": "/app/app/core/validation/util.py", "thread": "MainThread", "level": "INFO", "module": "util", "line_no": 109, "bucket": "cpr-sandbox-data-pipeline-cache", "file": "input/2022-12-21T12.57.21.698756/bulk-import.csv", "correlation_id": "8eeba1cd-f8f6-4c59-a3c5-86d787a51d7e" } |
| OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k |
| OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k |
| { "written_at": "2022-12-21T13:04:15.191Z", "written_ts": 1671627855191208000, "msg": "Started server process \[6\]", "type": "log", "logger": "uvicorn.error", "thread": "MainThread", "level": "INFO", "module": "server", "line_no": 75, "correlation_id": "-" } |
BAK-1279 Backend Deployment in App Runner fails on bulk import
When testing the pipeline via bulk import the backend seems to fall over and not log an error. Here is the code in question and the import logs. [https://github.com/climatepolicyradar/navigator/blob/main/backend/app/api/api_v1/routers/admin.py#L248](https://github.com/climatepolicyradar/navigator/blob/main/backend/app/api/api_v1/routers/admin.py#L248) Above calls below. [https://github.com/climatepolicyradar/navigator/blob/44bcb67396e0788f3c5d0c4be0578d51bb167fd5/backend/app/core/validation/util.py#L99](https://github.com/climatepolicyradar/navigator/blob/44bcb67396e0788f3c5d0c4be0578d51bb167fd5/backend/app/core/validation/util.py#L99) Log errors: Starts writing to csv, looks like the app falls over and then begins to start up again. | { "written_at": "2022-12-21T12:57:21.698Z", "written_ts": 1671627441698805000, "msg": "Writing CSV file into S3", "type": "log", "logger": "/app/app/core/validation/util.py", "thread": "MainThread", "level": "INFO", "module": "util", "line_no": 109, "bucket": "cpr-sandbox-data-pipeline-cache", "file": "input/2022-12-21T12.57.21.698756/bulk-import.csv", "correlation_id": "8eeba1cd-f8f6-4c59-a3c5-86d787a51d7e" } | | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k | | OpenBLAS WARNING - could not determine the L2 cache size on this system, assuming 256k | | { "written_at": "2022-12-21T13:04:15.191Z", "written_ts": 1671627855191208000, "msg": "Started server process \[6\]", "type": "log", "logger": "uvicorn.error", "thread": "MainThread", "level": "INFO", "module": "server", "line_no": 75, "correlation_id": "-" } |