Unable to import sessions from CSV

[jimklo]

When trying to import sessions from a CSV file; we receive an error in the log and no sessions are actually imported:

django_1    | Django version 3.2.6, using settings 'miqa.settings'
django_1    | Starting development server at http://0.0.0.0:8000/
django_1    | Quit the server with CONTROL-C.
django_1    | [22:09:10] INFO     "GET / HTTP/1.1" 200 1152                    basehttp.py:161
django_1    | [22:09:11] INFO     "GET /api/v1/users/me HTTP/1.1" 200 83       basehttp.py:161
django_1    |            INFO     "GET /api/v1/sessions HTTP/1.1" 200 166      basehttp.py:161
django_1    |            INFO     "GET /api/v1/sessions/900a4860-cbae-44c9-a4f basehttp.py:161
django_1    |                     f-46fc326aa55a HTTP/1.1" 200 690
django_1    |            INFO     "GET /api/v1/sites HTTP/1.1" 200 230         basehttp.py:161
django_1    | field names: ['xnat_experiment_id', 'nifti_folder', 'scan_id', 'scan_type', 'experiment_note', 'decision', 'scan_note']
django_1    | common path prefix: /fs/storage/xnat-kitware/archive/sri_incoming/arc001/B-00001-F-2-20160127/RESOURCES/
django_1    | [22:09:17] ERROR    Internal Server Error: /miqa4/api/v1/sessions/900 log.py:224
django_1    |                     a4860-cbae-44c9-a4ff-46fc326aa55a/import
django_1    |                     Traceback (most recent call last):
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/core/handlers/exception.py", line 47, in
django_1    |                     inner
django_1    |                         response = get_response(request)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/core/handlers/base.py", line 181, in
django_1    |                     _get_response
django_1    |                         response = wrapped_callback(request,
django_1    |                     *callback_args, **callback_kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/dj
django_1    |                     ango/views/decorators/csrf.py", line 54, in
django_1    |                     wrapped_view
django_1    |                         return view_func(*args, **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/viewsets.py", line 125, in view
django_1    |                         return self.dispatch(request, *args,
django_1    |                     **kwargs)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 509, in dispatch
django_1    |                         response = self.handle_exception(exc)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 469, in
django_1    |                     handle_exception
django_1    |                         self.raise_uncaught_exception(exc)
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 480, in
django_1    |                     raise_uncaught_exception
django_1    |                         raise exc
django_1    |                       File "/usr/local/lib/python3.8/site-packages/re
django_1    |                     st_framework/views.py", line 506, in dispatch
django_1    |                         response = handler(request, *args, **kwargs)
django_1    |                       File
django_1    |                     "/opt/django-project/miqa/core/rest/session.py",
django_1    |                     line 148, in import_
django_1    |                         import_data(request.user, session)
django_1    |                       File "/opt/django-project/miqa/core/tasks.py",
django_1    |                     line 23, in import_data
django_1    |                         raise ValidationError({'error': f'Invalid CSV
django_1    |                     file: {str(e)}'})
django_1    |                     jsonschema.exceptions.ValidationError:
django_1    |                     <unprintable ValidationError object>
django_1    |            ERROR    "POST /api/v1/sessions/900a4860-cbae-44c9-a4 basehttp.py:161
django_1    |                     ff-46fc326aa55a/import HTTP/1.1" 500 145

Seems like there are some 'hard coded' assumptions when looking at csvContentToJsonObject, especially here

/fs/storage/XNAT is a fixed path within the SRI/Stanford XNAT install - may not be the same for other installations. It should be noted that once this path is set for XNAT, there is not mechanism in XNAT to change it's root, meaning this should be a value that is set per installation, and likely should not be hard coded.

Related, the Django container configuration specifies:

MIQA_MOUNT_DIR=/usr/var/local/miqa

which is leveraged within the docker-compose.yml as:

    volumes:
      - "${MIQA_MOUNT_DIR}:${MIQA_MOUNT_DIR}"

I'm not sure this is the right way we should mount. Our host doesn't locate scans in the same physical location as the XNAT container.

[dchiquito]

Sadly I don't think those assumptions are at fault here. The real error is being mangled by this: raise ValidationError({'error': f'Invalid CSV file: {str(e)}'}). I submitted #137 to stop obscuring the useful stack traces, once it's merged you should be able to rebuild and get a more useful error log.