ERDDAP / erddap

ERDDAP is a scientific data server that gives users a simple, consistent way to download subsets of gridded and tabular scientific datasets in common file formats and make graphs and maps. ERDDAP is a Free and Open Source (Apache and Apache-like) Java Servlet from NOAA NMFS SWFSC Environmental Research Division (ERD).
Creative Commons Zero v1.0 Universal
84 stars 58 forks source link

Add maven-antrun-plugin to download ERDDAP large file resources #98

Closed mwengren closed 1 year ago

mwengren commented 1 year ago

@ChrisPJohn This PR should be considered after #97.

Using maven-antrun-plugin, we can automate some processes to configure a local development environment for ERDDAP, including:

These commands can be run individually via:

mvn antrun:run@download-erddap-content
mvn antrun:run@download-etopo
mvn antrun:run@make-data-dir

They also should be executed automatically when a Maven build is run e.g. mvn package.

Note: I ran into an issue with the overwrite=false flag in the download-erddap-content task - docs seem to indicate this should prevent the unzip operation from overwriting destination files if they already exist, but it did not work when I tested. This comes into play if you modify the defaults in content/erddap/setup.xml and want to retain your changes after an ERDDAP version release - if you use this file for running ERDDAP in development mode, it will check to ensure defaults have been modified before fully starting. Not a deal breaker however.

ChrisPJohn commented 1 year ago

Thanks for sharing this. Since one of the goals of using maven to download large resources is to avoid having them in the GitHub repo, I do want to find an alternative place to host the resources before we merge this.

I also want to make sure we keep the developer set up simple. I want to think about what else is needed for developer environment set up (using maven to install dependencies and downloading the erddap.war file). Ideally all of the environment setup should be a single maven command if we're moving to using maven for that.