search

confidential-containers / cloud-api-adaptor

Ability to create Kata pods using cloud provider APIs aka the peer-pods approach
Apache License 2.0
44 stars 71 forks source link

podvm: allow creation of using local path #1870

Open snir911 opened 2 weeks ago

snir911 commented 2 weeks ago

and RHEL podvm image adaptions

snir911 commented 2 weeks ago

cc @Saripalli-lavanya can you pls verify it works fine for s390?

bpradipt commented 2 weeks ago

nit: a short and complete description in the commit message for each one will be good reference when looking at the changes in future

snir911 commented 1 week ago

@stevenhorsman yes, basically it will use the Makefile to do this extra step of mounting the image (packer supports both local and network url for the same variable) This can be used for all distributions that you would like to use local image file instead network located file

Saripalli-lavanya commented 1 week ago

cc @Saripalli-lavanya can you pls verify it works fine for s390?

I tried with command PODVM_DISTRO=rhel make podvm-builder podvm-binaries podvm-image ARCH=s390x CLOUD_PROVIDER=libvirt ORG_ID=$ORG_ID ACTIVATION_KEY=$ACTIVATION_KEY PROTOC_VERSION=26.1 also placed base image inside podvm folder and calculated checksum.

[root@a3elp61 cloud-api-adaptor]#echo $IMAGE_CHECKSUM
952f1977f157913611b303638ad18b461*******
 docker buildx build -t quay.io/confidential-containers/podvm-libvirt-rhel-s390x:00017c08d06335428ae519a6d9efc66113c013ce6af20082360e2568df1e9b9f -f cloud-api-adaptor/podvm/Dockerfile.podvm.rhel \
--build-arg BUILDER_IMG=quay.io/confidential-containers/podvm-builder-rhel:00017c08d06335428ae519a6d9efc66113c013ce6af20082360e2568df1e9b9f \
--build-arg BINARIES_IMG=quay.io/confidential-containers/podvm-binaries-rhel-s390x:00017c08d06335428ae519a6d9efc66113c013ce6af20082360e2568df1e9b9f \
--build-arg PODVM_DISTRO=rhel \
--build-arg ARCH=s390x \
--build-arg CLOUD_PROVIDER=libvirt \
--build-arg IMAGE_CHECKSUM=952f1977f157913611b303638ad18b461******* --build-arg IMAGE_URL=/tmp/image.img -v /root/sl-CAATest/cloud-api-adaptor/src/cloud-api-adaptor/podvm/rhel-guest-image-9.2-20230414.17.s390x.qcow2:/tmp/image.img:Z \
--load  .

i have encountered error-1 at subscription registration. When i reverted back UBI image to 9.2 in Dockerfile.podvm_builder.rhel I didn't see this error and could proceed further. Also reverted PROTOC_ARCH related changes as I saw error-2

error-1

STEP 15/28: RUN if [[ -n "${ACTIVATION_KEY}" && -n "${ORG_ID}" ]]; then     subscription-manager register --org=${ORG_ID} --activationkey=${ACTIVATION_KEY};     fi
subscription-manager is operating in container mode. Use your host system to manage subscriptions.

Error: building at STEP "RUN if [[ -n "${ACTIVATION_KEY}" && -n "${ORG_ID}" ]]; then     subscription-manager register --org=${ORG_ID} --activationkey=${ACTIVATION_KEY};     fi": while running runtime: exit status 78
make: *** [Makefile:168: podvm-builder] Error 78

error-2

error: failed to run custom build command for `image-rs v0.1.0 (https://github.com/confidential-containers/guest-components?rev=ca6b438#ca6b4385)`

Caused by:
  process didn't exit successfully: `/src/kata-containers/src/agent/target/release/build/image-rs-fb659dde3b9259c9/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-changed=./protos/getresource.proto
  cargo:rerun-if-changed=./protos

  --- stderr
  Error: tonic build

  Caused by:
      failed to invoke protoc (hint: https://docs.rs/prost-build/#sourcing-protoc): (path: "/usr/local/bin/protoc"): Exec format error (os error 8)
warning: build failed, waiting for other jobs to finish...
make[1]: *** [Makefile:137: target/s390x-unknown-linux-gnu/release/kata-agent] Error 101
make: *** [Makefile.inc:135: /src/cloud-api-adaptor/podvm/files/usr/local/bin/kata-agent] Error 2
make[1]: Leaving directory '/src/kata-containers/src/agent'
Error: building at STEP "RUN LIBC=gnu make binaries": while running runtime: exit status 2
make: *** [Makefile:193: podvm-binaries] Error 2

Final images :

[root@a3elp61 cloud-api-adaptor]# docker images
Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
REPOSITORY                                                                              TAG                                                               IMAGE ID      CREATED             SIZE
quay.io/confidential-containers/podvm-libvirt-rhel-s390x                                00017c08d06335428ae519a6d9efc66113c013ce6af20082360e2568df1e9b9f  a771940d1d05  About a minute ago  1.11 GB
<none>                                                                                  <none>                                                            ef714c8736cd  2 minutes ago       11.6 GB
quay.io/confidential-containers/podvm-binaries-rhel-s390x                               00017c08d06335428ae519a6d9efc66113c013ce6af20082360e2568df1e9b9f  140a94f327ba  12 minutes ago      318 MB
<none>                                                                                  <none>                                                            5b9c974cde01  13 minutes ago      15.4 GB
quay.io/confidential-containers/podvm-builder-rhel                                      00017c08d06335428ae519a6d9efc66113c013ce6af20082360e2568df1e9b9f  68ccf9a775ce  40 minutes ago      7.21 GB
snir911 commented 1 week ago

@Saripalli-lavanya, Thanks! i think something like this should work (with recent changes):


PODVM_DISTRO=rhel IMAGE_LOCAL_PATH=/root/sl-CAATest/cloud-api-adaptor/src/cloud-api-adaptor/podvm/rhel-guest-image-9.2-20230414.17.s390x.qcow2 IMAGE_CHECKSUM=952f1977f157913611b303638ad18b461*******  ARCH=s390x CLOUD_PROVIDER=libvirt ORG_ID=$ORG_ID ACTIVATION_KEY=$ACTIVATION_KEY PROTOC_VERSION=26.1 YQ_ARCH=s390x PROTOC_ARCH=s390x_64 YQ_CHECKSUM="sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45" make podvm-builder podvm-binaries podvm-image
Saripalli-lavanya commented 1 week ago

@Saripalli-lavanya, Thanks! i think something like this should work (with recent changes):

PODVM_DISTRO=rhel IMAGE_LOCAL_PATH=/root/sl-CAATest/cloud-api-adaptor/src/cloud-api-adaptor/podvm/rhel-guest-image-9.2-20230414.17.s390x.qcow2 IMAGE_CHECKSUM=952f1977f157913611b303638ad18b461*******  ARCH=s390x CLOUD_PROVIDER=libvirt ORG_ID=$ORG_ID ACTIVATION_KEY=$ACTIVATION_KEY PROTOC_VERSION=26.1 YQ_ARCH=s390x PROTOC_ARCH=s390x_64 YQ_CHECKSUM="sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45" make podvm-builder podvm-binaries podvm-image

Yes, IMAGE_LOCAL_PATH worked well, same can be seen from the provided docker command from logs in the previous comment.

ill try this again with the latest changes and update shortly. Also do you have any suggestion for error-1?

snir911 commented 1 week ago

@Saripalli-lavanya it should work, it works for me and i didn't touch this part, i'm not sure what it could be

Saripalli-lavanya commented 1 week ago

@Saripalli-lavanya, Thanks! i think something like this should work (with recent changes):

PODVM_DISTRO=rhel IMAGE_LOCAL_PATH=/root/sl-CAATest/cloud-api-adaptor/src/cloud-api-adaptor/podvm/rhel-guest-image-9.2-20230414.17.s390x.qcow2 IMAGE_CHECKSUM=952f1977f157913611b303638ad18b461*******  ARCH=s390x CLOUD_PROVIDER=libvirt ORG_ID=$ORG_ID ACTIVATION_KEY=$ACTIVATION_KEY PROTOC_VERSION=26.1 YQ_ARCH=s390x PROTOC_ARCH=s390x_64 YQ_CHECKSUM="sha256:4e6324d08630e7df733894a11830412a43703682d65a76f1fc925aac08268a45" make podvm-builder podvm-binaries podvm-image

I successfully built my images using the specified command and the latest code. However, I encountered this error specifically with the UBI image 9.4 when building the builder image. Upon updating the UBI image to version 9.2, the error no longer occurred.

STEP 15/28: RUN if [[ -n "${ACTIVATION_KEY}" && -n "${ORG_ID}" ]]; then     subscription-manager register --org=${ORG_ID} --activationkey=${ACTIVATION_KEY};     fi
subscription-manager is operating in container mode. Use your host system to manage subscriptions.

Error: building at STEP "RUN if [[ -n "${ACTIVATION_KEY}" && -n "${ORG_ID}" ]]; then     subscription-manager register --org=${ORG_ID} --activationkey=${ACTIVATION_KEY};     fi": while running runtime: exit status 78
make: *** [Makefile:168: podvm-builder] Error 78
snir911 commented 1 week ago

@Saripalli-lavanya which podman/docker are you using?

Saripalli-lavanya commented 1 week ago

@Saripalli-lavanya which podman/docker are you using?

Hi @snir911, please find details below

podman version
Client:       Podman Engine
Version:      4.9.4-rhel
API Version:  4.9.4-rhel
Go Version:   go1.21.7 (Red Hat 1.21.7-1.module+el8.10.0+21318+5ea197f8)
Built:        Mon Apr  1 17:57:15 2024
OS/Arch:      linux/s390x
[root@a3elp61 ~]#
snir911 commented 1 week ago

@Saripalli-lavanya if your host is subscribed system you aren't required to set the key & org_id (although it should still work)