Attempted rollback of ArchiveFile operations results in Exception

BUG DESCRIPTION

Problem

Any exception that causes an attempted rollback of an ArchiveFile operation results in a Exception attempting to delete a file for an existing record.

Steps to reproduce

Any exception encountered when attempting to load an mzXML file will result in this exception, however one way to reproduce it relatively simply is to attempt to load an mzXML file that does not match any existing sample.

Login to tracebase-dev
```
ssh tracebase-dev.princeton.edu
```
sudo as the tracebase user
```
sudo -iu tracebase
```

Source the tracebase conda env

source /usr/local/tracebase/bin/activate

Attempt to load an mzXML file that does not correspond to an existing sample

cd "/tracebase-staging/loaded/exp027f_long_kli_timecourse"
python /var/www/tracebase/manage.py load_msruns \
  --mzxml-files './220909_exp027f4_free plasma and tissues/scan2/skipped_samples/exp027f4_free_M01_01_plasma.mzXML' \
  --instrument "Exploris480" \
  --date "2022-09-09" \
  --lc-protocol-name "polar-HILIC-25-min" \
  --operator "Michael Neinast"

Current behavior

  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 382, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 136, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 423, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/table_loader.py", line 1951, in load_wrapper
    retval = fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 385, in load_data
    sample = self.get_sample_by_name(sample_name, from_mzxml=True)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 786, in get_sample_by_name
    self.aggregated_errors_object.buffer_error(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1415, in buffer_error
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1369, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1349, in get_buffered_traceback_string
    for step in traceback.format_stack()

The above caught exception had a partial traceback:

  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 778, in get_sample_by_name
    rec = Sample.objects.get(name=sample_name)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/usr/local/tracebase/lib/python3.9/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
EXCEPTION2(ERROR): RecordDoesNotExist: Sample record matching the mzXML file's basename [exp027f4_free_M01_01_plasma] does not exist.  Please identify the associated sample and add a row with it, the matching mzXML file name(s), and the Sequence Name to the Peak Annotation Details sheet/file.
  File "/var/www/tracebase/manage.py", line 22, in <module>
    main()
  File "/var/www/tracebase/manage.py", line 18, in main
    execute_from_command_line(sys.argv)
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 382, in handle_wrapper
    retval = fn(self, *args, **options)
  File "/var/www/tracebase/DataRepo/management/commands/load_msruns.py", line 136, in handle
    self.load_data()
  File "/var/www/tracebase/DataRepo/management/commands/load_table.py", line 423, in load_data
    return self.loader.load_data(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/table_loader.py", line 1951, in load_wrapper
    retval = fn(*args, **kwargs)
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 400, in load_data
    self.clean_up_created_mzxmls_in_archive()
  File "/var/www/tracebase/DataRepo/loaders/msruns_loader.py", line 1507, in clean_up_created_mzxmls_in_archive
    self.aggregated_errors_object.buffer_error(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1415, in buffer_error
    return self.buffer_exception(
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1369, in buffer_exception
    buffered_tb_str = self.get_buffered_traceback_string()
  File "/var/www/tracebase/DataRepo/utils/exceptions.py", line 1349, in get_buffered_traceback_string
    for step in traceback.format_stack()
EXCEPTION3(ERROR): ProgrammingError: Cannot delete a file [/tracebasedev-archive/archive/archive_files/2024-06/ms_data/exp027f4_free_M01_01_plasma_wP9ious.mzXML] from the os that is associated with existing ArchiveFile database record: {'id': 652, 'filename': 'exp027f4_free_M01_01_plasma.mzXML', 'checksum': 'f9d2074dff00cf659a0888617bc92163725a45e4', 'file_location': <FieldFile: archive_files/2024-06/ms_data/exp027f4_free_M01_01_plasma_wP9ious.mzXML>, 'data_type': 2, 'data_format': 4}.
mzXML file rollback disk archive clean up stats: 0 deleted, 0 failed to be deleted, and 0 expected files did not exist.

Expected behavior

The RecordDoesNotExist exception should be reported and the copy of mzXML file should no longer be in the archive directory.

Princeton-LSI-ResearchComputing / tracebase