Internal notifier error when uploading to "subdirectory"

[mzur]

I set up s3cmd with my AOS S3 credentials. Then I tried to upload a whole directory of images to a collection. This resulted in a 500 (InternalError): Internal notifier error (see below):

$ s3cmd put -r * s3://latest.mycollection.biigle/
upload: 'so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg' -> 's3://latest.mycollection.biigle/so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg'  [1 of 20]
 7457651 of 7457651   100% in    2s     2.72 MB/s  done
WARNING: Upload failed: /so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg (500 (InternalError): Internal notifier error)
WARNING: Waiting 3 sec...
upload: 'so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg' -> 's3://latest.mycollection.biigle/so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg'  [1 of 20]
 7457651 of 7457651   100% in    2s     2.71 MB/s  done
WARNING: Upload failed: /so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg (500 (InternalError): Internal notifier error)
WARNING: Waiting 6 sec...
upload: 'so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg' -> 's3://latest.mycollection.biigle/so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg'  [1 of 20]
 7457651 of 7457651   100% in    2s     2.68 MB/s  done
WARNING: Upload failed: /so242_2_155_subset/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg (500 (InternalError): Internal notifier error)
WARNING: Waiting 9 sec...
^CSee ya!

When I upload images flat, it works:

$ s3cmd put * s3://latest.mycollection.biigle/
upload: 'HOTKEY_2015_09_04at04_42_13IMG_0064.jpg' -> 's3://latest.mycollection.biigle/HOTKEY_2015_09_04at04_42_13IMG_0064.jpg'  [1 of 10]
 7457651 of 7457651   100% in    4s  1754.13 KB/s  done
upload: 'HOTKEY_2015_09_04at04_49_01IMG_0085.jpg' -> 's3://latest.mycollection.biigle/HOTKEY_2015_09_04at04_49_01IMG_0085.jpg'  [2 of 10]
 6101939 of 6101939   100% in    3s  1756.77 KB/s  done
upload: 'HOTKEY_2015_09_04at05_32_55IMG_0215.jpg' -> 's3://latest.mycollection.biigle/HOTKEY_2015_09_04at05_32_55IMG_0215.jpg'  [3 of 10]
 5770683 of 5770683   100% in    3s  1868.66 KB/s  done
upload: 'HOTKEY_2015_09_04at07_45_15IMG_0604.jpg' -> 's3://latest.mycollection.biigle/HOTKEY_2015_09_04at07_45_15IMG_0604.jpg'  [4 of 10]
 6196752 of 6196752   100% in    3s  1827.27 KB/s  done
upload: 'TIMER_2015_09_04at04_28_44IMG_0024.jpg' -> 's3://latest.mycollection.biigle/TIMER_2015_09_04at04_28_44IMG_0024.jpg'  [5 of 10]
 8209130 of 8209130   100% in    3s     2.08 MB/s  done
upload: 'TIMER_2015_09_04at04_29_05IMG_0025.jpg' -> 's3://latest.mycollection.biigle/TIMER_2015_09_04at04_29_05IMG_0025.jpg'  [6 of 10]
 8231412 of 8231412   100% in    4s  1840.11 KB/s  done
upload: 'TIMER_2015_09_04at04_29_47IMG_0027.jpg' -> 's3://latest.mycollection.biigle/TIMER_2015_09_04at04_29_47IMG_0027.jpg'  [7 of 10]
 7792085 of 7792085   100% in    4s  1779.27 KB/s  done
upload: 'TIMER_2015_09_04at04_30_10IMG_0028.jpg' -> 's3://latest.mycollection.biigle/TIMER_2015_09_04at04_30_10IMG_0028.jpg'  [8 of 10]
 8402363 of 8402363   100% in    4s  1960.16 KB/s  done
upload: 'TIMER_2015_09_04at04_30_30IMG_0029.jpg' -> 's3://latest.mycollection.biigle/TIMER_2015_09_04at04_30_30IMG_0029.jpg'  [9 of 10]
 8087202 of 8087202   100% in    3s  2006.14 KB/s  done
upload: 'TIMER_2015_09_04at04_30_51IMG_0030.jpg' -> 's3://latest.mycollection.biigle/TIMER_2015_09_04at04_30_51IMG_0030.jpg'  [10 of 10]
 8207316 of 8207316   100% in    4s  1787.37 KB/s  done

[St4NNi]

Hi Martin and thanks for the report. We could reproduce this issue and tracked it down to two problems:

• 1. An error message that needs to be improved, it is not clear why this error occurs.
• 2. The bug originates from a broken subdirectory name validation, it seems like we do validate subdirectories with the bucket naming rules and not the correct object key naming scheme .

You can try this out with a subdirectory that does not include underscores or capital letters and it should work.

We will prepare a fix for this in the next patch.

BR

[St4NNi]

Our Version 2 Release should have this addressed.

Will close this for now if it still persists feel free to re-open it.