search

aces / Loris-MRI

The set of scripts that preprocess and insert MRI data into the database.
10 stars 50 forks source link

NoneType error when S3 configs not set properly #1109

Closed cmadjar closed 2 months ago

cmadjar commented 2 months ago

When running run_push_imaging_files_to_s3_pipeline.py with incorrect S3 authentication settings in database_config.py, get the following error:

[ERROR   ] S3 connection failure: An error occurred (InvalidAccessKeyId) when calling the ListBuckets operation: Unknown

Traceback (most recent call last):
  File "/opt/Loris-MRI/bin/mri/python/run_push_imaging_files_to_s3_pipeline.py", line 63, in <module>
    main()
  File "/opt/Loris-MRI/bin/mri/python/run_push_imaging_files_to_s3_pipeline.py", line 59, in main
    PushImagingFilesToS3Pipeline(loris_getopt_obj, os.path.basename(__file__[:-3]))
  File "/opt/Loris-MRI/bin/mri/python/lib/dcm2bids_imaging_pipeline_lib/push_imaging_files_to_s3_pipeline.py", line 56, in __init__
    self._upload_files_to_s3()
  File "/opt/Loris-MRI/bin/mri/python/lib/dcm2bids_imaging_pipeline_lib/push_imaging_files_to_s3_pipeline.py", line 228, in _upload_files_to_s3
    self.s3_obj.upload_file(file_full_path, file["s3_link"])
  File "/opt/Loris-MRI/bin/mri/python/lib/aws_s3.py", line 104, in upload_file
    (s3_bucket_name, s3_bucket, s3_file_name) = self.get_s3_object_path_part(s3_object_name)
  File "/opt/Loris-MRI/bin/mri/python/lib/aws_s3.py", line 270, in get_s3_object_path_part
    s3_bucket = self.s3.Bucket(s3_bucket_name)
AttributeError: 'NoneType' object has no attribute 'Bucket'

And when running run_push_imaging_files_to_s3_pipeline.py with incorrect S3 bucket name in database_config.py, get the following error:

[ERROR   ] S3 <fflocalbucket> bucket not found in <https://s3.msi.umn.edu>

Traceback (most recent call last):
  File "/opt/Loris-MRI/bin/mri/python/run_push_imaging_files_to_s3_pipeline.py", line 63, in <module>
    main()
  File "/opt/Loris-MRI/bin/mri/python/run_push_imaging_files_to_s3_pipeline.py", line 59, in main
    PushImagingFilesToS3Pipeline(loris_getopt_obj, os.path.basename(__file__[:-3]))
  File "/opt/Loris-MRI/bin/mri/python/lib/dcm2bids_imaging_pipeline_lib/push_imaging_files_to_s3_pipeline.py", line 56, in __init__
    self._upload_files_to_s3()
  File "/opt/Loris-MRI/bin/mri/python/lib/dcm2bids_imaging_pipeline_lib/push_imaging_files_to_s3_pipeline.py", line 228, in _upload_files_to_s3
    self.s3_obj.upload_file(file_full_path, file["s3_link"])
  File "/opt/Loris-MRI/bin/mri/python/lib/aws_s3.py", line 104, in upload_file
    (s3_bucket_name, s3_bucket, s3_file_name) = self.get_s3_object_path_part(s3_object_name)
  File "/opt/Loris-MRI/bin/mri/python/lib/aws_s3.py", line 270, in get_s3_object_path_part
    s3_bucket = self.s3.Bucket(s3_bucket_name)
AttributeError: 'NoneType' object has no attribute 'Bucket'

Need to catch that instead of having this printed out.