As a system admin, I want docker-wrapped WRES to clean up after itself

[epag]

---

Author Name: Jesse (Jesse) Original Redmine Issue: 55141, https://vlab.noaa.gov/redmine/issues/55141 Original Date: 2018-09-14

---

Given a release of WRES docker images to NWCAL When the new version is running fine Then the oldest version docker stuff should be cleaned up

The recent version(s?) should be retained, because we might wish to roll back or do other testing on them.

---

Related issue(s): #198 Redmine related issue(s): 56097, 65385

---

[epag]

---

Original Redmine Comment Author Name: Jesse (Jesse) Original Date: 2018-09-25T22:18:31Z

---

This ticket is in the general data management theme.

[epag]

---

Original Redmine Comment Author Name: Jesse (Jesse) Original Date: 2019-04-30T17:31:38Z

---

During the past two releases (1.7 and 1.6), I have manually cleaned up data by keeping at least the most recent two versions of wres and their docker-compose files.

Things to clean up:

1. Tars of docker images (everything but the current, because important ones will have already been imported into docker's storage).
2. Not-running docker containers
3. Docker images in docker storage (everything but the past two versions of each WRES image and docker-compose)
4. Evaluation outputs (although that is in a different ticket #56097)

[epag]

---

Original Redmine Comment Author Name: Jesse (Jesse) Original Date: 2019-06-27T17:34:41Z

---

One of the biggest sources of stale large data is mitigated by use of docker registry in that no tar files are copied then imported, rather the docker images are gotten straight from the registry.

However, today we see one of the -ti machines has a full disk and this is unrelated to images in /tmp nor related to the quantity of images, the image totals from @docker image ls@ are around 5GiB while there are 15GiB used on the filesystem.

Perhaps these are old containers or perhaps they are the current containers with larger-than-usual amounts of log messages due to actual use of the WRES on the -ti platform (COWRES).

[epag]

---

Original Redmine Comment Author Name: Jesse (Jesse) Original Date: 2019-06-27T17:41:52Z

---

There are 7 dead containers that are exited, found from @sudo docker ps --filter status=dead --filter status=exited -aq@

When running the following, around 38MiB space was freed (went from 20KiB available to 38MiB available). @sudo docker ps --filter status=dead --filter status=exited -aq | xargs sudo docker rm -v@

To see the byte counts of the logs themselves when printed, easy enough (idk if docker compresses these or not):

$ sudo docker logs 4b4dc739b22e | wc -c
4801326
$ sudo docker logs 9047e968ae93 | wc -c
15469885
$ sudo docker logs 11bbf479359d | wc -c
2719055
$ sudo docker logs a3b68182ee1a | wc -c
10006189
$ sudo docker logs a475a6cc3e59 | wc -c
232687
$ sudo docker logs d894564b3681 | wc -c
1513

That totals 32MiB of logs if uncompressed. This does not account for the 15GiB used.

[epag]

---

Original Redmine Comment Author Name: Jesse (Jesse) Original Date: 2019-06-27T17:43:03Z

---

I can see 44KiB in /tmp, but I see two other people who have directories in /tmp

[epag]

---

Original Redmine Comment Author Name: Jesse (Jesse) Original Date: 2019-06-27T17:46:22Z

---

/usr dir is 1.4GiB /var dir excluding /mnt is 2.3GiB

[epag]

---

Original Redmine Comment Author Name: Jesse (Jesse) Original Date: 2019-06-27T17:51:42Z

---

On another -ti machine, there are 6.0GiB free (after removing our tars to compare apples to apples). Totals from @docker image ls@ are around 5.2GiB. Total used by entire system is 9.1GiB. Similar or same docker images are present, however I do not see the two additional directories in /tmp from those two other users, I see one directory from a different user.