Problem: Enduro fails to process a directory of files uploaded to MinIO

[djjuhasz]

Describe the bug

When a transfer that is a directory of files is uploaded to the MinIO "sips" bucket for processing, Enduro creates a separate processing workflow for each file in the directory, and then each workflow fails with an fatal error in the download-activity.

To Reproduce

Steps to reproduce the behavior:

1. Upload https://github.com/artefactual/archivematica-sampledata/tree/master/SampleTransfers/Images to the MinIO sips bucket

One Enduro processing workflow is started for each file in the directory, then each workflow fails with a download-activity error like:

create file: open /tmp/enduro1526123462/Images/G31DS.TIF: no such file or directory

Expected behavior

1. The whole directory should be processed as a single transfer by a single processing workflow
2. The transfer should be successfully processed and create a single AIP

Screenshots

Additional context

• NewBlobEvent() takes an isDir parameter, but it is always set to false for MinIO watcher events
• The filesystem watcher does set the isDir parameter to true when appropriate (https://github.com/artefactual-sdps/enduro/blob/main/internal/watcher/filesystem.go#L153)

[Diogenesoftoronto]

This seems to need a bit of an update, filesystemwatcher does not have a isDir or similar attribute being set in the link provided: https://github.com/artefactual-sdps/enduro/blob/main/internal/watcher/filesystem.go#L132 . Context is updated most recently in @djjuhasz latest issue

[djjuhasz]

I updated the isDir link to https://github.com/artefactual-sdps/enduro/blob/main/internal/watcher/filesystem.go#L153