Update minio

[laurencejackson]

We have been using a very old version of minio that is incompatible with future releases, to avoid technical debt we should move to the new versions.

This PR makes the following changes:

• updates minio to RELEASE.2022-11-08T05-27-07Z which no longer uses the minio gateway. It also includes an updated UI which will make it easier to monitor file storage. In future we should look to distributing the artifact storage across minio instances.
• uses mc container to create mlflow bucket. The credentials are still required but can be passed in any way supported by boto3. The recommended way to do this is to add a credentials file to ~/.aws/credentials which will be mounted to the docker container at runtime.
• replaces related tests

Since the new version of minio is not compatible with the old version we need to migrate the old files over to the new one. Our minio is so old that the migration process published by minio didn't work (https://blog.min.io/deprecation-of-the-minio-gateway/), I kept coming across errors when trying to import configs or export metadata. I have done a simple filesystem mirror locally though and that seems to have worked. Instructions below:

We need to start two instances of minio to mirror them. First we need to separate the existing data new data directory. mv <minio data dir> <new location>

Then we need to recreate the old directory mkdir <minio data dir>

After this we can start the old minio using a simple docker run: docker run -p 9000:9000 --name minio_old -v <new location>:/data -e 'MINIO_ROOT_USER=minio' -e 'MINIO_ROOT_PASSWORD=qwer1234!' minio/minio:RELEASE.2021-03-17T02-33-02Z server /data

Then we can start the new server docker compose up -d --build

Then we need to set aliases using mc mc alias set oldminio http://localhost:9000 USER PWD mc alias set newminio http://localhost:8002 USER PWD

Then we can start the migration: mc mirror --preserve oldminio/mlflow newminio/mlflow

[github-actions[bot]]

Coverage Report

File	Stmts	Miss	Cover	Missing
mlops
Experiment.py	144	29	80%	45–50, 65–66, 83, 85, 90, 160–174, 192, 194–195, 236–237, 241, 243–246, 250–251, 261
ProjectFile.py	22	1	95%	38
cli.py	16	16	0%	1–46
mlops/data/tools
tools.py	35	31	11%	14–67
mlops/data/transforms
LoadImageXNATd.py	38	27	29%	21–25, 47–90
TOTAL	282	104	63%

Tests	Skipped	Failures	Errors	Time
13	2 :zzz:	0 :x:	0 :fire:	2m 32s :stopwatch: