search

open-component-model / ocm-project

OCM Project Backlog
Apache License 2.0
0 stars 0 forks source link

Slowness and strange behaviour of OCM handling OCI Images #293

Open Skarlso opened 2 days ago

Skarlso commented 2 days ago

There are some problem dealing with newly created TAGs and images and there is also a significant slowness of the new ORAS code to deal with OCI images. The following terminal actions tell the story:

➜  podinfo git:(main) ✗ time ocm15 transfer commontransportarchive --copy-resources ./transport-archive ghcr.io/skarlso/ocm-transfer-test-0.15.0-1
transferring component "ocm.software/podinfo"...
  transferring version "ocm.software/podinfo:1.0.5"...
    version "ocm.software/podinfo:1.0.5" already present -> skip transport
transferring component "ocm.software/podinfo/backend"...
  transferring version "ocm.software/podinfo/backend:1.0.0"...
    version "ocm.software/podinfo/backend:1.0.0" already present -> skip transport
transferring component "ocm.software/podinfo/frontend"...
  transferring version "ocm.software/podinfo/frontend:1.0.0"...
    version "ocm.software/podinfo/frontend:1.0.0" already present -> skip transport
transferring component "ocm.software/redis"...
  transferring version "ocm.software/redis:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](library/redis:6.0.1)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
ocm15 transfer commontransportarchive --copy-resources ./transport-archive   1,50s user 2,00s system 2% cpu 2:01,97 total
➜  podinfo git:(main) ✗ time ocm16 transfer commontransportarchive --copy-resources ./transport-archive ghcr.io/skarlso/ocm-transfer-test-0.16.0-1
transferring component "ocm.software/podinfo"...
  transferring version "ocm.software/podinfo:1.0.5"...
  ...adding component version...
transferring component "ocm.software/podinfo/backend"...
  transferring version "ocm.software/podinfo/backend:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](stefanprodan/podinfo:6.2.0)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
transferring component "ocm.software/podinfo/frontend"...
  transferring version "ocm.software/podinfo/frontend:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](stefanprodan/podinfo:6.2.0)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
transferring component "ocm.software/redis"...
  transferring version "ocm.software/redis:1.0.0"...
  ...resource 0 config[configdata.ocm.software]...
  ...resource 1 image[ociImage](library/redis:6.0.1)...
  ...resource 2 manifests[kustomize.ocm.fluxcd.io]...
  ...adding component version...
ocm16 transfer commontransportarchive --copy-resources ./transport-archive   2,35s user 2,81s system 2% cpu 3:48,74 total
➜  podinfo git:(main) ✗ ocm oci artifacts download ghcr.io/skarlso/ocm-transfer-test-22/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O downloaded30
➜  podinfo git:(main) ✗ ocm-transfer-test-0.15.0-1/stefanprodan/podinfo / sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a

➜  podinfo git:(main) ✗ ocm15 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0

donwload-v0.15.0: downloaded
➜  podinfo git:(main) ✗ ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0-with-16
donwload-v0.15.0-with-16: downloaded
➜  podinfo git:(main) ✗ time ocm15 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0-time

donwload-v0.15.0-time: downloaded
ocm15 oci artifacts download  -O donwload-v0.15.0-time  0,14s user 0,16s system 10% cpu 2,876 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.15.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O donwload-v0.15.0-time-16
donwload-v0.15.0-time-16: downloaded
ocm16 oci artifacts download  -O donwload-v0.15.0-time-16  0,21s user 0,31s system 3% cpu 17,191 total
➜  podinfo git:(main) ✗ time ocm15 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O download-v0.16.0-time

Error: getting indexed artifact sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc: oci artifact "ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc" not found in skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo
ocm15 oci artifacts download  -O download-v0.16.0-time  0,10s user 0,07s system 10% cpu 1,659 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O download-v0.16.0-time

Error: mkdir download-v0.16.0-time: file exists
ocm16 oci artifacts download  -O download-v0.16.0-time  0,12s user 0,06s system 14% cpu 1,216 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:fbc7e3038e8f8235d4d0c04484ee5d3019eb941a9643b733e60c18443d228f3a -O download-v0.16.0-with-16-time
Error: getting indexed artifact sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc: oci artifact "ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc" not found in skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo
ocm16 oci artifacts download  -O download-v0.16.0-with-16-time  0,12s user 0,06s system 12% cpu 1,497 total

The SHA is taken each time directly from the repository by checking what the created tag corresponds to. In case of 0.16.0 using the hash isn't working.

Using the tag also doesn't work:

time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo:6.2.0 -O download-v0.16.0-with-16-time-tag
Error: getting indexed artifact sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc: oci artifact "ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo@sha256:f60e14b08375a64528113dd8808b16030c771f626e66961dfaf511b74d6f68dc" not found in skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo
ocm16 oci artifacts download  -O download-v0.16.0-with-16-time-tag  0,10s user 0,07s system 9% cpu 1,681 total
➜  podinfo git:(main) ✗ time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/stefanprodan/podinfo:6.2.0 -O download-v0.16.0-with-16-time-tag

Using with component-descriptor:

time ocm16 oci artifacts download ghcr.io/skarlso/ocm-transfer-test-0.16.0-1/component-descriptors/ocm.software/podinfo@sha256:6ab290e0e34815a3176955ecd0c10d836bc0369da8a1a89326e861af650b7587 -O download-v0.16.0-with-16-time-tag-3-component
download-v0.16.0-with-16-time-tag-3-component: downloaded
ocm16 oci artifacts download  -O download-v0.16.0-with-16-time-tag-3-componen  0,13s user 0,06s system 8% cpu 2,328 total

Downloading the component directly works. Whatever the reason for the layering is we should NOT break functionality working previously.

Looking at the time we also see that the oras one (16) works significantly slower during transfer actions.

Skarlso commented 1 day ago

More information from others testing this and seeing this problem: https://github.com/open-component-model/ocm/issues/994