search

singularityhub / sregistry

server for storage and management of singularity images
https://singularityhub.github.io/sregistry
Mozilla Public License 2.0
103 stars 42 forks source link

Error while pushing images #404

Open imerelli opened 1 year ago

imerelli commented 1 year ago

Sorry to bother you again... I'm now trying to push images into sregistry using singularity from another server. I added the remote and provided the token:

singularity remote list

Cloud Services Endpoints \==================== NAME URI ACTIVE GLOBAL EXCLUSIVE SylabsCloud cloud.sylabs.io NO YES NO bioinfotiget sregistry.bioinfotiget.it YES NO NO

Keyservers \==================== URI GLOBAL INSECURE ORDER https://sregistry.bioinfotiget.it YES NO 1 \ Active cloud services keyserver

...but when I try to push an image I get this error:

singularity push -U centos7-python3-r4-deg.sif library://imerelli/rnaseq/centos7-python3-r4-deg:latest WARNING: Skipping container verification 0.0b / 1.0GiB [------------------------------------------------------------------------------] 0 % 0.0 b/s 0s FATAL: Unable to push image to library: Put "http://127.0.0.1:9000/sregistry/rnaseq/centos7-python3-r4-deg%3Asha256.49bfd3dcac50e3fbcfbb4b535b1924c98ced1fc49a16446bfb4af5ee7da0b08c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=newminio%2F20220909%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220909T205220Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host%3Bx-amz-content-sha256&partNumber=1&uploadId=2daaf94b-b02f-4518-88c2-269c45b6e467&X-Amz-Signature=9f4d33630bca145689d9b7f433c2fb6b6ac14cb2814c258fb1d9fcde909b60be": dial tcp 127.0.0.1:9000: connect: connection refused

Port 9000 should be open to internet on the sregistry server? Honestly, I don't think so, because it seems an internal call since it's using 127.0.0.1 Maybe something related to minio server? I did't change anything in the configuration about this service (should I?) Any other hint? logs and ports seems ok.

docker compose logs | grep 9000 sregistry-minio-1 | API: http://172.18.0.3:9000 http://127.0.0.1:9000

nmap localhost Starting Nmap 7.70 ( https://nmap.org ) at 2022-09-10 12:52 UTC Nmap scan report for localhost (127.0.0.1) Host is up (0.000019s latency). Other addresses for localhost (not scanned): ::1 Not shown: 992 closed ports PORT STATE SERVICE 22/tcp open ssh 25/tcp open smtp 80/tcp open http 111/tcp open rpcbind 443/tcp open https 631/tcp open ipp 8090/tcp open opsmessaging 9000/tcp open cslistener

vsoch commented 1 year ago

The way to debug this is to docker-compose logs for the minio /uwsgi containers and see what the error message(s) are.

imerelli commented 1 year ago

Thank you for the suggestion. But this time logs are not helping me in understanding what is happening:

docker compose logs for uwsgi sregistry-uwsgi-1 | [pid: 54|app: 0|req: 18/184] 155.253.6.100 () {34 vars in 497 bytes} [Sat Sep 10 15:18:05 2022] GET /v1/entities/imerelli => generated 305 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 1) sregistry-uwsgi-1 | GET GetNamedCollectionView sregistry-uwsgi-1 | [pid: 60|app: 0|req: 61/185] 155.253.6.100 () {34 vars in 517 bytes} [Sat Sep 10 15:18:05 2022] GET /v1/collections/imerelli/rnaseq => generated 342 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 1) sregistry-uwsgi-1 | GET GetNamedContainerView sregistry-uwsgi-1 | [pid: 60|app: 0|req: 62/186] 155.253.6.100 () {34 vars in 561 bytes} [Sat Sep 10 15:18:05 2022] GET /v1/containers/imerelli/rnaseq/centos7-python3-r4-deg => generated 522 bytes in 26 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 2) sregistry-uwsgi-1 | GET PushNamedContainerView sregistry-uwsgi-1 | [pid: 54|app: 0|req: 19/187] 155.253.6.100 () {34 vars in 718 bytes} [Sat Sep 10 15:18:05 2022] GET /v1/images/imerelli/rnaseq/centos7-python3-r4-deg:sha256.49bfd3dcac50e3fbcfbb4b535b1924c98ced1fc49a16446bfb4af5ee7da0b08c?arch=amd64 => generated 563 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 2) sregistry-uwsgi-1 | [pid: 54|app: 0|req: 20/188] 155.253.6.100 () {34 vars in 471 bytes} [Sat Sep 10 15:18:05 2022] GET /version => generated 58 bytes in 2 msecs (HTTP/1.1 200) 5 headers in 141 bytes (1 switches on core 3) sregistry-uwsgi-1 | POST RequestMultiPartPushImageFileView sregistry-uwsgi-1 | Start multipart upload 13e3176e-156d-4422-9be2-cdeb41b1e69e sregistry-uwsgi-1 | [pid: 60|app: 0|req: 63/189] 155.253.6.100 () {36 vars in 535 bytes} [Sat Sep 10 15:18:05 2022] POST /v2/imagefile/3/_multipart => generated 96 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 132 bytes (1 switches on core 3) sregistry-uwsgi-1 | PUT RequestMultiPartPushImageFileView sregistry-uwsgi-1 | [pid: 55|app: 0|req: 39/190] 155.253.6.100 () {36 vars in 536 bytes} [Sat Sep 10 15:18:07 2022] PUT /v2/imagefile/3/_multipart => generated 499 bytes in 19 msecs (HTTP/1.1 200) 5 headers in 133 bytes (1 switches on core 0) sregistry-uwsgi-1 | PUT RequestMultiPartAbortView sregistry-uwsgi-1 | Delete imagefile signal running. sregistry-uwsgi-1 | [pid: 54|app: 0|req: 21/191] 155.253.6.100 () {36 vars in 546 bytes} [Sat Sep 10 15:18:07 2022] PUT /v2/imagefile/3/_multipart_abort => generated 0 bytes in 31 msecs (HTTP/1.1 200) 4 headers in 93 bytes (1 switches on core 1)

docker compose logs for the minio sregistry-minio-1 | WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated. sregistry-minio-1 | Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD sregistry-minio-1 | MinIO Object Storage Server sregistry-minio-1 | Copyright: 2015-2022 MinIO, Inc. sregistry-minio-1 | License: GNU AGPLv3 https://www.gnu.org/licenses/agpl-3.0.html sregistry-minio-1 | Version: RELEASE.2022-09-07T22-25-02Z (go1.18.6 linux/amd64) sregistry-minio-1 | sregistry-minio-1 | Status: 1 Online, 0 Offline. sregistry-minio-1 | API: http://172.18.0.3:9000 http://127.0.0.1:9000 sregistry-minio-1 | Console: http://172.18.0.3:45563 http://127.0.0.1:45563 sregistry-minio-1 | sregistry-minio-1 | Documentation: https://docs.min.io

vsoch commented 1 year ago

But I think they are? This seems important:

sregistry-minio-1 | WARNING: MINIO_ACCESS_KEY and MINIO_SECRET_KEY are deprecated.
sregistry-minio-1 | Please use MINIO_ROOT_USER and MINIO_ROOT_PASSWORD
sregistry-minio-1 | MinIO Object Storage Server
vsoch commented 1 year ago

Minio is on port 9000 so logically the error (and that message) hints that the server is not correctly starting. Notice toward the bottom the status is Offline.

imerelli commented 1 year ago

If I put MINIO_ROOT_USER and MINIO_ROOT_PASSWORD in the .minio-env file the warning goes away. But the error is still present. Moreover, if I remove MINIO_ACCESS_KEY and MINIO_SECRET_KEY the site does not start because I think uwsgi needs these parameters. So in the end I think MINIO_ACCESS_KEY and MINIO_SECRET_KEY are ok and the warning can be ignored for now.

I also think that minio is up and running because I installed mc for logging and trying a push here what is reported:

./mc admin trace -v myminio
minio:9000 [REQUEST s3.NewMultipartUpload] [2022-09-12T07:07:18.125] [Client IP: 172.18.0.5] minio:9000 POST /sregistry/rnaseq/centos7-python3-r4-deg%3Asha256.49bfd3dcac50e3fbcfbb4b535b1924c98ced1fc49a16446bfb4af5ee7da0b08c?uploads minio:9000 Proto: HTTP/1.1 minio:9000 Host: minio:9000 minio:9000 X-Amz-Date: 20220912T070718Z minio:9000 Accept-Encoding: identity minio:9000 Amz-Sdk-Request: attempt=1 minio:9000 Authorization: AWS4-HMAC-SHA256 Credential=minio/20220912/us-east-1/s3/aws4_request, SignedHeaders=host;x-amz-content-sha256;x-amz-date, Signature=3b1bcb0d7f09fa064e7fed0fdac0d1e26b441e047f4d832b35864d91974bf7a4 minio:9000 Content-Length: 0 minio:9000 User-Agent: Boto3/1.21.46 Python/3.6.13 Linux/4.18.0-240.1.1.el8_3.x86_64 Botocore/1.24.46 minio:9000 Amz-Sdk-Invocation-Id: 8f9e4967-291e-4bcf-a3c3-36aff45dcd24 minio:9000 X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 minio:9000 minio:9000 [RESPONSE] [2022-09-12T07:07:18.127] [ Duration 1.562ms ↑ 131 B ↓ 345 B ] minio:9000 200 OK minio:9000 Content-Type: application/xml minio:9000 Vary: Origin,Accept-Encoding minio:9000 X-Content-Type-Options: nosniff minio:9000 X-Xss-Protection: 1; mode=block minio:9000 Accept-Ranges: bytes minio:9000 Content-Length: 345 minio:9000 Content-Security-Policy: block-all-mixed-content minio:9000 Server: MinIO minio:9000 Strict-Transport-Security: max-age=31536000; includeSubDomains minio:9000 X-Amz-Request-Id: 17140B2A29DF7F22 minio:9000 <?xml version="1.0" encoding="UTF-8"?> < InitiateMultipartUploadResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/">sregistry< Key>rnaseq/centos7-python3-r4-deg:sha256.49bfd3dcac50e3fbcfbb4b535b1924c98ced1fc49a16446bfb4af5ee7da0b08caa9764b3-f62a-4f5a-9d68-4b5f4a0d247c< /UploadId>< /InitiateMultipartUploadResult> minio:9000

It seems that for some reasons the upload starts, but then it gets interrupted. No idea of what is happening.

vsoch commented 1 year ago

That’s correct! You’ll need to update the two variables here: https://github.com/singularityhub/sregistry/blob/49e644bd6bf40853edadd7e1573c812a74553226/shub/apps/library/views/minio.py#L49. Also with a change in environment variables I’ve found I often need to stop, remove, and recreate the containers.

vsoch commented 1 year ago

These docs might help debug, these variables are new: https://docs.min.io/minio/baremetal/reference/minio-server/minio-server.html#envvar.MINIO_ROOT_USER

imerelli commented 1 year ago

I modified the .minio-env file, which now contains only MINIO_ROOT_USER and MINIO_ROOT_PASSWORD I also modified the shub/apps/library/views/minio.py file changing all the occurrence of MINIO_ACCESS_KEY and MINIO_SECRET_KEY, with MINIO_ROOT_USER and MINIO_ROOT_PASSWORD, respectively. I stopped and cancelled everything and restarted the whole system brand new.

Results: the warning message disappeared and the system is up, but still get exactly the same error while pushing the image. I really think the problem is somewhere else...

vsoch commented 1 year ago

Can you please show me the logs for each of minio, uwsgi, and nginx? If minio wasn't running because of those credentials it's likely that was at least part of the problem (or in other words, you wouldn't get a working push in that state).

imerelli commented 1 year ago

Below the logs related to a push attempt. For minio there are no logs apart from the startup signal. In case I can contact you also by mail to organize a quick chat.

docker compose logs for the minio sregistry-minio-1 | MinIO Object Storage Server sregistry-minio-1 | Copyright: 2015-2022 MinIO, Inc. sregistry-minio-1 | License: GNU AGPLv3 https://www.gnu.org/licenses/agpl-3.0.html sregistry-minio-1 | Version: RELEASE.2022-09-07T22-25-02Z (go1.18.6 linux/amd64) sregistry-minio-1 | sregistry-minio-1 | Status: 1 Online, 0 Offline. sregistry-minio-1 | API: http://172.18.0.4:9000 http://127.0.0.1:9000 sregistry-minio-1 | Console: http://172.18.0.4:41575 http://127.0.0.1:41575 sregistry-minio-1 | sregistry-minio-1 | Documentation: https://docs.min.io

docker compose logs for the nginx sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:13 +0000] "GET /v1/entities/imerelli HTTP/1.1" 200 305 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-" sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:13 +0000] "GET /v1/collections/imerelli/rnaseq HTTP/1.1" 200 342 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-" sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:14 +0000] "GET /v1/containers/imerelli/rnaseq/centos7-python3-r4-deg HTTP/1.1" 200 522 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-" sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:14 +0000] "GET /v1/images/imerelli/rnaseq/centos7-python3-r4-deg:sha256.49bfd3dcac50e3fbcfbb4b535b1924c98ced1fc49a16446bfb4af5ee7da0b08c?arch=amd64 HTTP/1.1" 200 563 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-" sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:14 +0000] "GET /version HTTP/1.1" 200 58 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-" sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:14 +0000] "POST /v2/imagefile/3/_multipart HTTP/1.1" 200 96 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-" sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:16 +0000] "PUT /v2/imagefile/3/_multipart HTTP/1.1" 200 496 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-" sregistry-nginx-1 | 155.253.6.100 - - [12/Sep/2022:13:23:16 +0000] "PUT /v2/imagefile/3/_multipart_abort HTTP/1.1" 200 0 "-" "Singularity/3.8.7 (Linux amd64) Go/1.16.13" "-"

docker compose logs for the uwsgi sregistry-uwsgi-1 | GET NamedEntityView sregistry-uwsgi-1 | <QueryDict: {}> sregistry-uwsgi-1 | imerelli sregistry-uwsgi-1 | [pid: 56|app: 0|req: 14/49] 155.253.6.100 () {34 vars in 497 bytes} [Mon Sep 12 08:23:13 2022] GET /v1/entities/imerelli => generated 305 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 0) sregistry-uwsgi-1 | GET GetNamedCollectionView sregistry-uwsgi-1 | [pid: 56|app: 0|req: 15/50] 155.253.6.100 () {34 vars in 517 bytes} [Mon Sep 12 08:23:13 2022] GET /v1/collections/imerelli/rnaseq => generated 342 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 1) sregistry-uwsgi-1 | GET GetNamedContainerView sregistry-uwsgi-1 | [pid: 55|app: 0|req: 10/51] 155.253.6.100 () {34 vars in 561 bytes} [Mon Sep 12 08:23:13 2022] GET /v1/containers/imerelli/rnaseq/centos7-python3-r4-deg => generated 522 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 2) sregistry-uwsgi-1 | GET PushNamedContainerView sregistry-uwsgi-1 | [pid: 55|app: 0|req: 11/52] 155.253.6.100 () {34 vars in 718 bytes} [Mon Sep 12 08:23:14 2022] GET /v1/images/imerelli/rnaseq/centos7-python3-r4-deg:sha256.49bfd3dcac50e3fbcfbb4b535b1924c98ced1fc49a16446bfb4af5ee7da0b08c?arch=amd64 => generated 563 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 142 bytes (1 switches on core 0) sregistry-uwsgi-1 | [pid: 55|app: 0|req: 12/53] 155.253.6.100 () {34 vars in 471 bytes} [Mon Sep 12 08:23:14 2022] GET /version => generated 58 bytes in 2 msecs (HTTP/1.1 200) 5 headers in 141 bytes (1 switches on core 1) sregistry-uwsgi-1 | POST RequestMultiPartPushImageFileView sregistry-uwsgi-1 | Start multipart upload 3047bf5b-cbf1-42f9-9faa-5c1d52652f1d sregistry-uwsgi-1 | [pid: 55|app: 0|req: 13/54] 155.253.6.100 () {36 vars in 535 bytes} [Mon Sep 12 08:23:14 2022] POST /v2/imagefile/3/_multipart => generated 96 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 132 bytes (1 switches on core 3) sregistry-uwsgi-1 | PUT RequestMultiPartPushImageFileView sregistry-uwsgi-1 | [pid: 56|app: 0|req: 16/55] 155.253.6.100 () {36 vars in 536 bytes} [Mon Sep 12 08:23:16 2022] PUT /v2/imagefile/3/_multipart => generated 496 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 133 bytes (1 switches on core 3) sregistry-uwsgi-1 | PUT RequestMultiPartAbortView sregistry-uwsgi-1 | Delete imagefile signal running. sregistry-uwsgi-1 | [pid: 56|app: 0|req: 17/56] 155.253.6.100 () {36 vars in 546 bytes} [Mon Sep 12 08:23:16 2022] PUT /v2/imagefile/3/_multipart_abort => generated 0 bytes in 32 msecs (HTTP/1.1 200) 4 headers in 93 bytes (1 switches on core 2)

imerelli commented 1 year ago

Hi, I changed this line in the config.py file (Before it was 127.0.0.1:9000) and the error changed

MINIO_EXTERNAL_SERVER = ( "sregistry.bioinfotiget.it:9000" # minio server for Singularity to interact with )

FATAL: Unable to push image to library: Put "http://sregistry.bioinfotiget.it:9000/sregistry/rnaseq/centos7-python3-r4-deg%3Asha256.49bfd3dcac50e3fbcfbb4b535b1924c98ced1fc49a16446bfb4af5ee7da0b08c?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=minio%2F20220917%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20220917T144406Z&X-Amz-Expires=300&X-Amz-SignedHeaders=host%3Bx-amz-content-sha256&partNumber=1&uploadId=f5e68e33-2b7b-467c-ab13-63b07cb32999&X-Amz-Signature=e1cdbf1296a1a4cd2fe9d4fa0a4b7a096f06e05bfb163eabdca942ce76ea4638": dial tcp 131.175.207.216:9000: i/o timeout

Instead of connection refused I have a timeout. I want to point out that the only minio I have is the one that starts with docker compose so, according to the documentation, it is not clear to me if I have to change this line or not.

Also, have I to change this? MINIO_SERVER = "minio:9000" I don't think so because changing this variable the system does not start correctly

What about this? MINIO_SSL = False I'm using SSL for the web site, but I not sure if I have to change this. I don't think so because changing this variable the system does not start correctly.

Final questions: connecting to 131.175.207.216:9000 should I see something? How can I connect to the minio interface? My server is on openstack, so I opened port 9000. Should I open other ports?

Please help me in solving this, I really think I'm close to the solution, but still I don't see how to solve the problem. Since there are no other options for a private storage of singularity images, this is very important for me...

vsoch commented 1 year ago

MINIO_EXTERNAL_SERVER = ( "sregistry.bioinfotiget.it:9000" # minio server for Singularity to interact with )

Okay so let's step back a second. There are two mini server URLs, the first (above) is what the external URL should look like. If there is a timeout, you can try testing this locally with ping and various network tools. I would read the mini docs to see what basic interactions with a deployed registry should look like, and verify they work before coming back to sregistry. You can tell that some aspect is working because in the payload you showed me, you got as far as to have an upload id. If I were in your shoes, I'd engage with other minio users and the developer - for example this looks similar to what you are experiencing https://github.com/minio/console/issues/1818 and could have resulted from a new Minio update.

Also, have I to change this? MINIO_SERVER = "minio:9000"

This is the server that the other containers see, essentially internal networking. Try shelling into any container and looking at /etc/hosts. You should be able to ping minio from the inside of the other containers.

MINIO_SSL = False I'm using SSL for the web site, but I not sure if I have to change this. I don't think so because changing this variable the system does not start correctly.

This means you aren't using https. When I'm working on this I traditionally start without https, and then when I've confirmed it works I will add SSL and change this or True. You might consider removing ssl temporary while you debug this further because you will eventually need this. It's driving the logic here https://github.com/singularityhub/sregistry/blob/49e644bd6bf40853edadd7e1573c812a74553226/shub/apps/library/views/minio.py#L44-L80 if you are interested.

connecting to 131.175.207.216:9000 should I see something?

Yes you should. Check out the tutorial. https://docs.min.io/docs/minio-quickstart-guide.html

Please help me in solving this, I really think I'm close to the solution, but still I don't see how to solve the problem. Since there are no other options for a private storage of singularity images, this is very important for me...

I'm doing my best, of course without being able to debug it myself.

vsoch commented 1 year ago

I'm going to start preparing a PR with some of the Minio changes we've discovered in this issue! Hopefully if I can reproduce your issue I can help too.

vsoch commented 1 year ago

okay I've tested the entire thing from a fresh build to login -> push -> pull, and updated the docs accordingly. https://github.com/singularityhub/sregistry/pull/405. I did the entire workflow without https and couldn't reproduce your issue (using the new minio) so I think my advice (if you can't get the above working) is to step back and test first without https, and add when you are sure that is working. If you test from my branch it should also be using a newer Python and have the bugs fixed that we chat about, so it might be a good idea too.

And a few more checks:

imerelli commented 1 year ago

I tested the new version of your software, recreating everything from scratch and without using https (for doing this I had to update singularity to 3.10). The problem is still the same. My feeling is that there is a problem with ports. When I try from a browser to connect to http://131.175.207.216:9000 I'm pushed to http://131.175.207.216:33187 which is clearly not available. I disabled firewalld and, although the server is on an openstack based virtual machine, all ports are open (this was one of the first things I tried).

vsoch commented 1 year ago

Ah so we actually have been debugging your deployment and not an issue with (the expected) deployment of sregistry. That's a bit out of scope, no?

You need to get Minio running on OpenStack first it seems. I think perhaps you should start with that https://github.com/minio/minio/issues?q=openstack. I've never used OpenStack so I can't offer experience, but I agree with you it sounds like you haven't properly deployed a service on your cluster.

vsoch commented 1 year ago

Another strategy is to look at similar services and do what they do, e.g., https://docs.openstack.org/install-guide/environment-sql-database-ubuntu.html. This seems important:

set the bind-address key to the management IP address of the controller node to enable access by other nodes via the management network.

imerelli commented 1 year ago

Sorry, but this time I don't see the point. Openstack is only used to manage virtual machine. There should be no difference between a Centos7 on a physical server and a Centos7 on a virtualized server. Minio is delivered with your docker compose, right? I don't think I have to install a separated one.

vsoch commented 1 year ago

Yes but I don't think it's the fault of the docker-compose here, it's something about this custom setup. That's all I'm saying.

vsoch commented 1 year ago

There also appears to be something called "nova" https://www.edureka.co/community/65178/how-can-i-configure-docker-service-inside-openstack. and controlling ingress https://ovh.github.io/cds/hosting/ready-to-run/docker-compose/full-example/