Use Docker for ERDDAP distributions

[BobSimons]

Investigate using Docker for ERDDAP distributions. In particular, this may make it much easier to bundle other software with ERDDAP, e.g., ncWMS, or MapServer.

[7yl4r]

I have bundled ERDDAP alongside some other docker containers in marinebon/mbon-dashboard-server. It works well and I am happy to help others use docker-compose to do the same.

[abkfenris]

If @kwilcox ok's adapting axiom-data-science/docker-erddap, then I'm willing to make a PR to add the Dockerfile, and setup Github Actions and Packages to build and host the image.

[BobSimons]

There hasn't been any progress on this, but there is new documentation for installing ERDDAP via Docker (thanks to a group at the 2022 IOOS Code Sprint): https://ioos.github.io/erddap-gold-standard/index.html

[abkfenris]

I hadn't realized that @kwilcox had given a thumbs up to adapting the code from axiom's repo.

@BobSimons how do you go from what is in this repo to the erddap.war and erddapContent.zip assets attached to a release?

[BobSimons]

I have a checklist for releases from before I used Git. I have a small script (makeErddapWar.bat) which makes erddap.war. erddapContent.zip is made by running some Java code Erddap.makeErddapContentZip(theSourceDirectory, theDestinationDirectory);

[abkfenris]

Do either of those depend on resources that are outside of this repo? I couldn't find either script in the repo, just a reference to Erddap.makeErddapContentZip().

It would be really helpful to have the release checklist to work through to make sure that I'm getting all of the nuance needed to make a correct release.

[BobSimons]

Erddap.makeErddapContentZip() is a method in WEB-INF/classes/gov/noaa/pfel/erddap/Erddap.java. Thus it is already in the Git archive.

I will add the makeErddapWar.bat file to the Git archive.

[abkfenris]

I've started a PR #79 to work towards building a Docker image as part of the repo using Github Actions.

Right now I've got the workflow trigger for every push to the repo. Github Actions does support triggering workflows on release creation, but that can be much harder to test (at least without making a ton of releases to test how the workflows run).

So far I've adapted makeErddapWar.bat into the build job, and extracted out the long list of files that are used as inputs to jar into jar-inputfilesto try to make it easier to maintain and reuse acrossmakeErddapWar.batand the workflow. Currently it will automatically builderddap.war`, though it appears to be coming out a few MB smaller. I'm not sure if there is a difference in compression or if there is something missing.

It looks like Erddap.makeErddapContentZip() is making a standard zip file with what looks to be the contents of https://github.com/BobSimons/erddapContent . Should I clone that repo, or is there a better way to get that data?

I've tried to set up enough of an job that TestAll can be run, but I'm getting a failure trying to run that. I don't know enough Java to know where to start digging into that, so it would be awesome if someone could take a look there. I did figure out the classpath formatting is different between win and linux (; vs :).

I've got the start of a docker job that uses the erddap.war created in the build job and a derived version of Axiom's Dockerfile to build an image. I've got a whole bunch of things commented out right now there (waiting on test to pass/pushing to repo/logging in/setting up Docker metadata), but it works.

[BobSimons]

@abkfenris, please email me: bob.simons@noaa.gov .

[BobSimons]

After some private discussions, I think I don't like your proposed system to use GitHub Actions, to test ERDDAP, generate erddap.war, then make the Docker version of ERDDAP. The main problems are that:

• It's very hard to set up and maintain a full ERDDAP test environment. ERDDAP works with live, changing datasets, so it is a complex testing environment.
• Not all tests pass. There are known problems (e.g., bugs in netcdf-java reported to Unidate but unsolved, and test datasets that are now gone).
• The release process involves me testing erddap.war on a public server before releasing it to others. It's better to catch bugs here than after releasing ERDDAP to others.
• I don't want to turn over control of making erddap.war. That leaves me responsible for other people's work. That's trouble. And it necessitates a lot of interaction so that all changes to the script that makes erddap.war are properly committed and tested. That is unnecessary and error prone.

I think it is far better for you to make a (semi) automated system that:

• Is triggered by someone running a script, not GitHub Actions.
• Uses my erddap.war when I make it available.
• Has my products in my GitHub repository, and your products in your repository. That way we each have responsibility for what is in our own repository. The line of demarcation is clear. And ERDDAP-related issues will be handled in my repository and Docker-related issues will be handled in your repository.

That avoids tons of trouble and unnecessary interactions. It is a clear line of separation of responsibilities. That approach is easier for you -- you can develop and test your script any time by running it against the current official erddap.war. And my release notes will point to your GitHub repository as the place to go for the Docker version.