search

NASA-PDS / devops

Parent repo for PDS DevOps activities
Apache License 2.0
0 stars 0 forks source link

As a user, I want PDS EN produced Docker images to support multiple platforms (x86 and ARM) #50

Closed tloubrieu-jpl closed 1 year ago

tloubrieu-jpl commented 1 year ago

Checked for duplicates

Yes - I've already checked

🧑‍🔬 User Persona(s)

other developers or testers mostly.

💪 Motivation

...so that I can deploy PDS EN docker images on my macbook using a M1 chip.

📖 Additional Details

This need to be implemented in the CICD.

Acceptance Criteria

Given When I perform Then I expect

⚙️ Engineering Details

No response

tloubrieu-jpl commented 1 year ago

Note this is preventing @miguelp1986 from using the registry docker compose script on his laptop.

nutjob4life commented 1 year ago

Should this go onto the sprint-backlog?

nutjob4life commented 1 year ago

@miguelp1986 I hate to be that guy that says, "It works for me" … but … it does 😇

I got it to work on my Mac Studio (M1 Ultra). Here are the steps I ran ($ is my shell prompt):

$ date -u
Tue Jul 11 23:58:48 UTC 2023
$ hostname
mirasol.local
$ sw_vers
ProductName:        macOS
ProductVersion:     13.4
BuildVersion:       22F66
$ arch
arm64
$ docker version 
Client:
 Cloud integration: v1.0.33
 Version:           24.0.2
 API version:       1.43
 Go version:        go1.20.4
 Git commit:        cb74dfc
 Built:             Thu May 25 21:51:16 2023
 OS/Arch:           darwin/arm64
 Context:           desktop-linux

Server: Docker Desktop 4.20.1 (110738)
 Engine:
  Version:          24.0.2
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.4
  Git commit:       659604f
  Built:            Thu May 25 21:50:59 2023
  OS/Arch:          linux/arm64
  Experimental:     true
 containerd:
  Version:          1.6.21
  GitCommit:        3dce8eb055cbb6872793272b4f20ed16117344f8
 runc:
  Version:          1.1.7
  GitCommit:        v1.1.7-0-g860f061
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

$ cd /tmp
$ git clone --quiet https://github.com/NASA-PDS/registry.git
$ cd registry/docker/certs
$ ./generate-certs.sh
Generating RSA private key, 2048 bit long modulus
...................................................+++++
..........................+++++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
..............................................+++++
...........+++++
e is 65537 (0x10001)
Signature ok
subject=/C=CA/ST=CALIFORNIA/L=LA/O=ORG/OU=PDS/CN=elasticsearch
Getting CA Private Key
$ cd ..
$ docker compose --profile=int-registry-batch-loader up --quiet-pull --remove-orphans
[+] Running 11/8
…
docker-elasticsearch-init-1                  | [INFO] Loaded 500 document(s)
…
docker-elasticsearch-init-1                  | [INFO] Loaded 1500 document(s)
…
docker-elasticsearch-init-1                  | [INFO] Loaded 3122 document(s)
docker-elasticsearch-init-1 exited with code 0
…
docker-registry-api-1                        | 
docker-registry-api-1                        |   .   ____          _            __ _ _
docker-registry-api-1                        |  /\\ / ___'_ __ _ _(_)_ __  __ _ \ \ \ \
docker-registry-api-1                        | ( ( )\___ | '_ | '_| | '_ \/ _` | \ \ \ \
docker-registry-api-1                        |  \\/  ___)| |_)| | | | | || (_| |  ) ) ) )
docker-registry-api-1                        |   '  |____| .__|_| |_|_| |_\__, | / / / /
docker-registry-api-1                        |  =========|_|==============|___/=/_/_/_/
docker-registry-api-1                        |  :: Spring Boot ::                (v3.0.4)
…
docker-registry-loader-test-init-1           | [INFO] Wrote 44 product(s)
docker-registry-loader-test-init-1           | [SUMMARY] Summary:
docker-registry-loader-test-init-1           | [SUMMARY] Skipped files: 0
docker-registry-loader-test-init-1           | [SUMMARY] Loaded files: 44
docker-registry-loader-test-init-1           | [SUMMARY]   Product_Bundle: 6
docker-registry-loader-test-init-1           | [SUMMARY]   Product_Collection: 9
docker-registry-loader-test-init-1           | [SUMMARY]   Product_Document: 3
docker-registry-loader-test-init-1           | [SUMMARY]   Product_SPICE_Kernel: 26
docker-registry-loader-test-init-1           | [SUMMARY] Failed files: 0
docker-registry-loader-test-init-1           | [SUMMARY] Package ID: bfbdb458-1a2c-4da2-ba65-a42a56bbec10
docker-registry-loader-test-init-1           | Setting archive status ...
…

After a long, long time, I entered this from another session:

$ cd /tmp/registry/docker
$ docker compose --profile=int-registry-batch-loader ps
NAME                            IMAGE                                 COMMAND                  SERVICE                CREATED             STATUS              PORTS
docker-elasticsearch-1          opensearchproject/opensearch:1.2.4    "./opensearch-docker…"   elasticsearch          6 minutes ago       Up 6 minutes        0.0.0.0:9200->9200/tcp, 9300/tcp, 0.0.0.0:9600->9600/tcp, 9650/tcp
docker-registry-api-1           nasapds/registry-api-service:latest   "/usr/local/registry…"   registry-api           6 minutes ago       Up 6 minutes        80/tcp, 0.0.0.0:8080->8080/tcp
docker-registry-web-archive-1   nginx                                 "/docker-entrypoint.…"   registry-web-archive   6 minutes ago       Up 6 minutes        0.0.0.0:81->80/tcp
docker-tls-termination-1        nasapds/proxycrypt                    "/entrypoint.sh ngin…"   tls-termination        6 minutes ago       Up 6 minutes        80/tcp, 0.0.0.0:8443->443/tcp

which looks okay.

I can visit http://localhost:8080/ with a browser and I get the Registry's "Swagger" API. If I try some of the endpoints like /products, I get data.

Let's take a look at each image:

$ for image in `docker compose --profile=int-registry-batch-loader images | sed 1d | awk '{print $2}'`; do echo $image; docker image inspect $image | egrep 'Architecture'; done
opensearchproject/opensearch
Error response from daemon: No such image: opensearchproject/opensearch:latest
nasapds/registry-loader
        "Architecture": "arm64",
postman/newman
        "Architecture": "amd64",
nasapds/registry-api-service
        "Architecture": "arm64",
nasapds/registry-loader
        "Architecture": "arm64",
nasapds/registry-sweepers
        "Architecture": "arm64",
nginx
        "Architecture": "arm64",
nasapds/proxycrypt
        "Architecture": "amd64",

and ignore the opensearch one for now. We can see that we're already publishing using mostly arm64 images, but nasapds/proxycrypt is an amd64. However, that's okay, because Rosetta2 translates the amd64 instructions to arm64 instructions (and on this snazzy Mac studio, runs them even faster!).

Is it possible you don't have Rosetta2 installed? You can install it from the command line with:

softwareupdate --install-rosetta

or simply try to launch any amd64 app from the Finder and you'll be prompted to install it.

NOW THAT BEING SAID, this doesn't satisfy the ticket. I think we should have nasapds/proxycrypt be published as a multi-platform image too. In fact, all of our images ought to. But at least we are already publishing the Registry images as multi-platform!

Let me know if this helps you get going @miguelp1986.

miguelp1986 commented 1 year ago

@nutjob4life yes that did help, thank you Sean. Once I cleared my machine of all my Docker containers my issues went away. I did notice a stack trace during the build process. int-registry-batch-loader Stack Trace.txt

nutjob4life commented 1 year ago

@miguelp1986 good to hear! Sorry about the stack trace. Could you submit a ticket for that? Include not just the stack trace, but also the commands up to and including the one that caused it.