search

ERDDAP / erddap

ERDDAP is a scientific data server that gives users a simple, consistent way to download subsets of gridded and tabular scientific datasets in common file formats and make graphs and maps. ERDDAP is a Free and Open Source (Apache and Apache-like) Java Servlet from NOAA NMFS SWFSC Environmental Research Division (ERD).
Creative Commons Zero v1.0 Universal
76 stars 54 forks source link

S3 Support #95

Open matcor-bodc opened 1 year ago

matcor-bodc commented 1 year ago

Hello,

We've recently looked into using our ERRDAP instance with a Minio S3 object store. Unfortunately it looks like ERDDAP currently only works with AWS S3 locations. Is this something that could be added in? It certainly would be a big help for us and hopefully could be useful for others

BobSimons commented 1 year ago

Yes. Someone else mentioned this.

When I wrote the S3 support, it only existed at Amazon. I didn't anticipate that would change, but I suspect there is a straightforward way to deal with this.

Currently, if ERDDAP sees that a file's path matches String2.AWS_S3_REGEX (which is Amazon-specific), it knows to deal with the file via the AWS S3 Java API. Since files in your S3 installation don't match that regex, ERDDAP just sees them as goofy/invalid file names. Perhaps there could be a new tag (in datasets.xml?) to specify additional regexes for other S3 installations. I'll leave it to Chris to consider this.