Open jeremyh opened 2 years ago
I think we need to add support for accessory files stored on s3 but I'm keen to know what @omad and @SpacemanPaul think about this.
Toktam's point makes sense to me. At a minimum, supporting accessory files stored remotely seems a reasonable use case given the community's increasing reliance on cloud storage.
@jeremyh Happy for me to add support for this?
Yep!
Reported by Belle and Toktam
This is because
DatasetAssembler
is writing a package locally, and assumes all file references are local.These derivative processing systems write local files but are trying to reference remote accessory files.
I'm not certain if this is what we want to do (accessories were originally references to non-measurement extra files included in the package), but we should decide on the preferred way for people to do this.