Zooniverse upload-chunks: "Locations is invalid"

[lucasgautheron]

Detailed report here: https://github.com/LAAC-LSCP/ChildProject/discussions/270#discussioncomment-1346021

Investigate whether this is because of the implementation or because of panoptes.

[lucasgautheron]

Follow-up on Zooniverse: https://www.zooniverse.org/talk/18/2145052

[lucasgautheron]

It appears the problematic file is empty. Not sure why. The timestamps are okay. A subject extracted from the same recording and close to the faulty one was successfully uploaded too; so it cannot be the original audio.

We'll know more when @alecristia does ls -lR /path/to/the/chunks | grep ".mp3" and posts the results here.

In the meantime, I've done two commits:

74a7f86a9cc8eef263467a7f35d127fba50b98a8 (adding a switch to keep uploading even if one of the chunks failed to upload) a0c18d3d0552d11c026b3fa0aa6bc6762ca493b8 (make extract-chunks overwrite empty files; this means that if the first extraction fails for some reason a produce an empty file, running it again would solve the issue)

[lucasgautheron]

I cannot reproduce the empty file by applying extract-chunks on the segments here: https://gin.g-node.org/LAAC-LSCP/vanuatu-zoo/src/master/samples/loud_5s_10perchild/segments_20210916_151531.csv

So, my guess is:

1. extract-chunks was run once and interrupted, which left the mp3 file empty
2. it was run again after, but since the mp3 file existed it was skipped

Which should be fixed by a0c18d3 (which makes extract-chunks overwrite empty files). Still, I'd be curious to know the output of ls -lR /path/to/the/chunks | grep ".mp3" to confirm or refute this hypothesis.

[lucasgautheron]

According to https://github.com/LAAC-LSCP/ChildProject/discussions/270#discussioncomment-1367552:

• it is more than one mp3 that is empty, there's 30 of them
• all of them are from different recordings, which suggests that the problem happened once per thread (since a recording cannot be processed by more than one thread)
• therefore, it is not impossible that the problem comes from an interruption of the extraction process
• not sure why the modification times are different though (although, they seem quite close, all within a few minutes range)

My conclusion: I have no better explanation then the hypothesis above (extract-chunks was interrupted).

@alecristia: I suggest to update the package, run extract-chunks again and remove the newly created metadata file (keeping only the former). If this fixes it for you, I suggest to close this issue.