search

weaveworks / ignite

Ignite a Firecracker microVM
https://ignite.readthedocs.org
Apache License 2.0
3.49k stars 226 forks source link

a1.metal Arm64 run failure: timeout waiting for ignite-spawn startup #574

Open MarcoPolo opened 4 years ago

MarcoPolo commented 4 years ago

Hi! 👋

I was interested in getting this setup in an a1.metal instance since I was able to get firecracker working in one. But I couldn't seem to get it working.

I tried various versions of ignite as well (as far back as v0.5.0). I also tried doing the exact same setup on an x86 bare metal packet server to make sure I wasn't doing something wrong.

I'm not exactly sure where the error is either? Is there somewhere else I can look?

Thanks for the cool project!

Full Log:

# ignite run ubuntu:18.04 --log-level trace
TRAC[0000] Populating providers...                      
TRAC[0000] Initializing the containerd runtime provider... 
TRAC[0000] Initializing the CNI provider...             
DEBU[0000] Ensuring image ubuntu:18.04 exists, or importing it... 
TRAC[0000] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0000] index: counted 0 Image object(s)             
TRAC[0000] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0000] cache: Get Image with UID "ac23985dcabeaf12" 
TRAC[0000] index: cache miss for Image with UID "ac23985dcabeaf12" 
TRAC[0000] index: storing Image object with UID "ubuntu:18.04", meta: false 
DEBU[0000] Found image with UID ac23985dcabeaf12        
DEBU[0000] Ensuring kernel weaveworks/ignite-kernel:4.19.47 exists, or importing it... 
TRAC[0000] Client.Find; GVK: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0000] index: counted 0 Kernel object(s)            
TRAC[0000] cache: miss when listing: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0000] cache: Get Kernel with UID "58470a2ec656f4f1" 
TRAC[0000] index: cache miss for Kernel with UID "58470a2ec656f4f1" 
TRAC[0000] index: storing Kernel object with UID "weaveworks/ignite-kernel:4.19.47", meta: false 
DEBU[0000] Found kernel with UID 58470a2ec656f4f1       
TRAC[0000] index: counted 0 VM object(s)                
TRAC[0000] cache: miss when listing: ignite.weave.works/__internal, Kind=VM 
TRAC[0000] index: counted 0 VM object(s)                
TRAC[0000] cache: miss when listing: ignite.weave.works/__internal, Kind=VM 
TRAC[0000] Client.Set; UID: "445ddf01df54879e", GVK: ignite.weave.works/__internal, Kind=VM 
TRAC[0000] cache: Set VM with UID "445ddf01df54879e"    
TRAC[0000] index: storing VM object with UID "lively-night", meta: false 
TRAC[0000] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0000] index: counted 1 Image object(s)             
TRAC[0000] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0000] cache: Get Image with UID "ac23985dcabeaf12" 
TRAC[0000] index: cache hit for Image with UID "ac23985dcabeaf12" 
TRAC[0000] cacheObject: "ubuntu:18.04" checksum: "1586476860560124086" 
TRAC[0000] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0000] index: counted 1 Image object(s)             
TRAC[0000] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0000] cache: Get Image with UID "ac23985dcabeaf12" 
TRAC[0000] index: cache hit for Image with UID "ac23985dcabeaf12" 
TRAC[0000] cacheObject: "ubuntu:18.04" checksum: "1586476860560124086" 
TRAC[0000] Client.Find; GVK: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0000] index: counted 1 Kernel object(s)            
TRAC[0000] cache: hit when listing: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0000] index: listing ignite.weave.works/__internal, Kind=Kernel objects, meta: true 
TRAC[0000] cacheObject: "weaveworks/ignite-kernel:4.19.47" checksum: "1586469338005036313" 
TRAC[0000] cache: Get Kernel with UID "58470a2ec656f4f1" 
TRAC[0000] index: cache hit for Kernel with UID "58470a2ec656f4f1" 
TRAC[0000] cacheObject: "weaveworks/ignite-kernel:4.19.47" checksum: "1586469338005036313" 
INFO[0001] Created VM with ID "445ddf01df54879e" and name "lively-night" 
TRAC[0001] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0001] index: counted 1 Image object(s)             
TRAC[0001] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0001] cache: Get Image with UID "ac23985dcabeaf12" 
TRAC[0001] index: cache hit for Image with UID "ac23985dcabeaf12" 
TRAC[0001] cacheObject: "ubuntu:18.04" checksum: "1586476860560124086" 
TRAC[0001] Client.Find; GVK: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0001] index: counted 1 Kernel object(s)            
TRAC[0001] cache: hit when listing: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0001] index: listing ignite.weave.works/__internal, Kind=Kernel objects, meta: true 
TRAC[0001] cacheObject: "weaveworks/ignite-kernel:4.19.47" checksum: "1586469338005036313" 
TRAC[0001] cache: Get Kernel with UID "58470a2ec656f4f1" 
TRAC[0001] index: cache hit for Kernel with UID "58470a2ec656f4f1" 
TRAC[0001] cacheObject: "weaveworks/ignite-kernel:4.19.47" checksum: "1586469338005036313" 
DEBU[0001] containerd: Inspecting image "weaveworks/ignite:v0.6.3" 
DEBU[0001] Writing "/var/lib/firecracker/vm/445ddf01df54879e/runtime.containerd.resolv.conf" with new hash: "a1c4b64088188444578db3dee49070836807d9cf18bf0dd4a0a21f5d53436364", old hash: "" 
INFO[0001] Networking is handled by "cni"               
INFO[0001] Started Firecracker VM "445ddf01df54879e" in a container with ID "ignite-445ddf01df54879e" 
FATA[0011] timeout waiting for ignite-spawn startup     
root@ip-172-31-7-55:/home/ubuntu#
mehstg commented 4 years ago

Having exactly the same issue here on arm64. Running ignite on Ubuntu 20.04 on a Raspberry Pi 4


root@ubuntu:~# ignite run ubuntu:18.04 --log-level trace
TRAC[0000] Populating providers...                      
TRAC[0000] Initializing the containerd runtime provider... 
TRAC[0000] Initializing the CNI provider...             
DEBU[0000] Ensuring image ubuntu:18.04 exists, or importing it... 
TRAC[0000] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0000] index: counted 0 Image object(s)             
TRAC[0000] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
DEBU[0000] Importing image with ociRef "ubuntu:18.04"   
DEBU[0000] containerd: Inspecting image "ubuntu:18.04"  
INFO[0000] containerd image "ubuntu:18.04" not found locally, pulling... 
DEBU[0000] containerd: Pulling image "ubuntu:18.04"     
DEBU[0000] resolving                                     host=registry-1.docker.io
DEBU[0000] do request                                    host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=HEAD url="https://registry-1.docker.io/v2/library/ubuntu/manifests/18.04"
DEBU[0000] fetch response received                       host=registry-1.docker.io response.header.content-length=157 response.header.content-type=application/json response.header.date="Thu, 28 May 2020 15:26:37 GMT" response.header.docker-distribution-api-version=registry/2.0 response.header.strict-transport-security="max-age=31536000" response.header.www-authenticate="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/ubuntu:pull\"" response.status="401 Unauthorized" url="https://registry-1.docker.io/v2/library/ubuntu/manifests/18.04"
DEBU[0000] Unauthorized                                  header="Bearer realm=\"https://auth.docker.io/token\",service=\"registry.docker.io\",scope=\"repository:library/ubuntu:pull\"" host=registry-1.docker.io
DEBU[0000] do request                                    host=registry-1.docker.io request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=HEAD url="https://registry-1.docker.io/v2/library/ubuntu/manifests/18.04"
DEBU[0001] fetch response received                       host=registry-1.docker.io response.header.content-length=1418 response.header.content-type=application/vnd.docker.distribution.manifest.list.v2+json response.header.date="Thu, 28 May 2020 15:26:38 GMT" response.header.docker-content-digest="sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" url="https://registry-1.docker.io/v2/library/ubuntu/manifests/18.04"
DEBU[0001] resolved                                      desc.digest="sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f" host=registry-1.docker.io
DEBU[0001] fetch                                         digest="sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f" mediatype=application/vnd.docker.distribution.manifest.list.v2+json size=1418
DEBU[0001] do request                                    digest="sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f" mediatype=application/vnd.docker.distribution.manifest.list.v2+json request.header.accept="application/vnd.docker.distribution.manifest.list.v2+json, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=GET size=1418 url="https://registry-1.docker.io/v2/library/ubuntu/manifests/sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f"
DEBU[0001] fetch response received                       digest="sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f" mediatype=application/vnd.docker.distribution.manifest.list.v2+json response.header.content-length=1418 response.header.content-type=application/vnd.docker.distribution.manifest.list.v2+json response.header.date="Thu, 28 May 2020 15:26:38 GMT" response.header.docker-content-digest="sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" size=1418 url="https://registry-1.docker.io/v2/library/ubuntu/manifests/sha256:3235326357dfb65f1781dbc4df3b834546d8bf914e82cce58e6e6b676e23ce8f"
DEBU[0001] fetch                                         digest="sha256:03e4a3b262fd97281d7290c366cae028e194ae90931bc907991444d026d6392a" mediatype=application/vnd.docker.distribution.manifest.v2+json size=1152
DEBU[0001] do request                                    digest="sha256:03e4a3b262fd97281d7290c366cae028e194ae90931bc907991444d026d6392a" mediatype=application/vnd.docker.distribution.manifest.v2+json request.header.accept="application/vnd.docker.distribution.manifest.v2+json, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=GET size=1152 url="https://registry-1.docker.io/v2/library/ubuntu/manifests/sha256:03e4a3b262fd97281d7290c366cae028e194ae90931bc907991444d026d6392a"
DEBU[0001] fetch response received                       digest="sha256:03e4a3b262fd97281d7290c366cae028e194ae90931bc907991444d026d6392a" mediatype=application/vnd.docker.distribution.manifest.v2+json response.header.content-length=1152 response.header.content-type=application/vnd.docker.distribution.manifest.v2+json response.header.date="Thu, 28 May 2020 15:26:38 GMT" response.header.docker-content-digest="sha256:03e4a3b262fd97281d7290c366cae028e194ae90931bc907991444d026d6392a" response.header.docker-distribution-api-version=registry/2.0 response.header.etag="\"sha256:03e4a3b262fd97281d7290c366cae028e194ae90931bc907991444d026d6392a\"" response.header.strict-transport-security="max-age=31536000" response.status="200 OK" size=1152 url="https://registry-1.docker.io/v2/library/ubuntu/manifests/sha256:03e4a3b262fd97281d7290c366cae028e194ae90931bc907991444d026d6392a"
DEBU[0001] fetch                                         digest="sha256:12a3cf2229154ace97979da9f77ffffa3c93344de6455dd132a6c8e8902392e7" mediatype=application/vnd.docker.container.image.v1+json size=3408
DEBU[0001] fetch                                         digest="sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip size=35197
DEBU[0001] fetch                                         digest="sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip size=189
DEBU[0001] fetch                                         digest="sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip size=849
DEBU[0001] fetch                                         digest="sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip size=23720401
DEBU[0001] do request                                    digest="sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip request.header.accept="application/vnd.docker.image.rootfs.diff.tar.gzip, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=GET size=35197 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f"
DEBU[0001] do request                                    digest="sha256:12a3cf2229154ace97979da9f77ffffa3c93344de6455dd132a6c8e8902392e7" mediatype=application/vnd.docker.container.image.v1+json request.header.accept="application/vnd.docker.container.image.v1+json, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=GET size=3408 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:12a3cf2229154ace97979da9f77ffffa3c93344de6455dd132a6c8e8902392e7"
DEBU[0001] do request                                    digest="sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip request.header.accept="application/vnd.docker.image.rootfs.diff.tar.gzip, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=GET size=23720401 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434"
DEBU[0001] do request                                    digest="sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip request.header.accept="application/vnd.docker.image.rootfs.diff.tar.gzip, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=GET size=189 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce"
DEBU[0001] do request                                    digest="sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip request.header.accept="application/vnd.docker.image.rootfs.diff.tar.gzip, */*" request.header.user-agent=containerd/1.3.0+unknown request.method=GET size=849 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a"
DEBU[0001] fetch response received                       digest="sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip response.header.accept-ranges=bytes response.header.age=558337 response.header.cache-control="public, max-age=14400" response.header.cf-cache-status=HIT response.header.cf-ray=59a8fec57cba354c-LHR response.header.cf-request-id=02fd7d8f6d0000354cd21de200000001 response.header.content-length=35197 response.header.content-type=application/octet-stream response.header.date="Thu, 28 May 2020 15:26:38 GMT" response.header.etag="\"69e062ea4cb8c60e61640a496d602e74\"" response.header.expect-ct="max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"" response.header.expires="Thu, 28 May 2020 19:26:38 GMT" response.header.last-modified="Fri, 24 Apr 2020 00:21:16 GMT" response.header.server=cloudflare response.header.set-cookie="__cfduid=d7f38bf373c1420869fa4199be2fec9191590679598; expires=Sat, 27-Jun-20 15:26:38 GMT; path=/; domain=.production.cloudflare.docker.com; HttpOnly; SameSite=Lax; Secure" response.header.vary=Accept-Encoding response.header.x-amz-id-2="nqVTox7kt/gg7JTU3eggasdLkKxmI93eYNo39aFGokR3ZyeSOUz2Mt27YewcvsnTTcRp9U+quzs=" response.header.x-amz-request-id=6BCA3639F259B919 response.header.x-amz-version-id=u0OYBJ8gFQuaOzn2T0LujnzF8b8LER.W response.status="200 OK" size=35197 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f"
DEBU[0001] fetch response received                       digest="sha256:12a3cf2229154ace97979da9f77ffffa3c93344de6455dd132a6c8e8902392e7" mediatype=application/vnd.docker.container.image.v1+json response.header.accept-ranges=bytes response.header.age=552427 response.header.cache-control="public, max-age=14400" response.header.cf-cache-status=HIT response.header.cf-ray=59a8fec5ccc6354c-LHR response.header.cf-request-id=02fd7d8f9a0000354cd21df200000001 response.header.content-length=3408 response.header.content-type=application/octet-stream response.header.date="Thu, 28 May 2020 15:26:39 GMT" response.header.etag="\"0f338610ed9fba1e170d17acfe2871e8\"" response.header.expect-ct="max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"" response.header.expires="Thu, 28 May 2020 19:26:39 GMT" response.header.last-modified="Fri, 24 Apr 2020 00:21:27 GMT" response.header.server=cloudflare response.header.set-cookie="__cfduid=dc87365649fea6daac2b979bf92d6a8961590679599; expires=Sat, 27-Jun-20 15:26:39 GMT; path=/; domain=.production.cloudflare.docker.com; HttpOnly; SameSite=Lax; Secure" response.header.vary=Accept-Encoding response.header.x-amz-id-2="GoDTX+6cBZkBVF/Xxde0lz1XRSni5vuLfuvsn1z+UUDZNDB+9QDOBFwORKouhxxDOK6ZOJwWNF0=" response.header.x-amz-request-id=F661B637375397F6 response.header.x-amz-version-id=0Y3MDX4Aoc8y_63MPrRiRb0EwHUNkwgk response.status="200 OK" size=3408 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:12a3cf2229154ace97979da9f77ffffa3c93344de6455dd132a6c8e8902392e7"
DEBU[0001] layer downloaded                              desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f 35197 [] map[] <nil>}"
DEBU[0001] fetch response received                       digest="sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip response.header.accept-ranges=bytes response.header.age=558338 response.header.cache-control="public, max-age=14400" response.header.cf-cache-status=HIT response.header.cf-ray=59a8fec67d08354c-LHR response.header.cf-request-id=02fd7d900b0000354cd21e2200000001 response.header.content-length=23720401 response.header.content-type=application/octet-stream response.header.date="Thu, 28 May 2020 15:26:39 GMT" response.header.etag="\"e2d42a66b8e3a0e69d0d8080268778bf\"" response.header.expect-ct="max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"" response.header.expires="Thu, 28 May 2020 19:26:39 GMT" response.header.last-modified="Sun, 05 Apr 2020 20:25:25 GMT" response.header.server=cloudflare response.header.set-cookie="__cfduid=db96d1aede71aca7e274a6b561cbc73671590679599; expires=Sat, 27-Jun-20 15:26:39 GMT; path=/; domain=.production.cloudflare.docker.com; HttpOnly; SameSite=Lax; Secure" response.header.vary=Accept-Encoding response.header.x-amz-id-2="xD9p+gEuWRINA7rar457AJr0ei6PUtmo3E6cETPHUULqjYxcDIjOjp77F3cGTTiqvqQa6gJ9MEE=" response.header.x-amz-request-id=69BBC59008CD0A9F response.header.x-amz-version-id=gJ8rmtNhtZpexLe7nl9OslBPogW8Psjh response.status="200 OK" size=23720401 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434"
DEBU[0002] fetch response received                       digest="sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip response.header.accept-ranges=bytes response.header.age=558337 response.header.cache-control="public, max-age=14400" response.header.cf-cache-status=HIT response.header.cf-ray=59a8fec75d4d354c-LHR response.header.cf-request-id=02fd7d90930000354cd21ea200000001 response.header.content-length=189 response.header.content-type=application/octet-stream response.header.date="Thu, 28 May 2020 15:26:39 GMT" response.header.etag="\"df5e223691f4ed1ebde40081f1a4338d\"" response.header.expect-ct="max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"" response.header.expires="Thu, 28 May 2020 19:26:39 GMT" response.header.last-modified="Fri, 24 Apr 2020 00:21:15 GMT" response.header.server=cloudflare response.header.set-cookie="__cfduid=d2be8e260e47b04a236c0fd6e863f6f731590679599; expires=Sat, 27-Jun-20 15:26:39 GMT; path=/; domain=.production.cloudflare.docker.com; HttpOnly; SameSite=Lax; Secure" response.header.vary=Accept-Encoding response.header.x-amz-id-2="x6NHlL2/CShKT9YWUqMf0DrkOL0kXAXK7XeGPMJ2a/JS3HYLlksKCc9n3+9gZT/mILssSG7GTg8=" response.header.x-amz-request-id=084023F4CA654FAA response.header.x-amz-version-id=Lw2vRLbRLgPQq_OUj9QL9HvaaLnfBsha response.status="200 OK" size=189 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce"
DEBU[0002] fetch response received                       digest="sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a" mediatype=application/vnd.docker.image.rootfs.diff.tar.gzip response.header.accept-ranges=bytes response.header.age=558337 response.header.cache-control="public, max-age=14400" response.header.cf-cache-status=HIT response.header.cf-ray=59a8fec75d50354c-LHR response.header.cf-request-id=02fd7d90990000354cd21eb200000001 response.header.content-length=849 response.header.content-type=application/octet-stream response.header.date="Thu, 28 May 2020 15:26:39 GMT" response.header.etag="\"3dde2be7ab3c78df7ecd39b9ccad5442\"" response.header.expect-ct="max-age=604800, report-uri=\"https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\"" response.header.expires="Thu, 28 May 2020 19:26:39 GMT" response.header.last-modified="Fri, 24 Apr 2020 00:21:15 GMT" response.header.server=cloudflare response.header.set-cookie="__cfduid=d5aa211dec90ce536a9718fd5d93f86aa1590679599; expires=Sat, 27-Jun-20 15:26:39 GMT; path=/; domain=.production.cloudflare.docker.com; HttpOnly; SameSite=Lax; Secure" response.header.vary=Accept-Encoding response.header.x-amz-id-2="yfstDc6IhcSYx216v8Z7eaP7D5I/FiJCTQbuwjR7Os9BS0Aq9c+exo30WuCXNdER8bwb1O7QwgM=" response.header.x-amz-request-id=00C7B751349871AF response.header.x-amz-version-id=XUlrkfAjLnkU_YRnV1Yv5e4JAs2wN.PL response.status="200 OK" size=849 url="https://registry-1.docker.io/v2/library/ubuntu/blobs/sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a"
DEBU[0002] layer downloaded                              desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce 189 [] map[] <nil>}"
DEBU[0002] layer downloaded                              desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a 849 [] map[] <nil>}"
DEBU[0004] layer downloaded                              desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434 23720401 [] map[] <nil>}"
DEBU[0004] unpack layer                                  desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434 23720401 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:2e95f1db4570b6afb86ab045946ae08258e6fa441dff3c1f203c16e144d8d16d 0 [] map[] <nil>}"
DEBU[0007] layer unpacked                                desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:c2cd007b69f7e5f37c851aa689e0e617fbaa3fe3e470f714337a03e569b7f434 23720401 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:2e95f1db4570b6afb86ab045946ae08258e6fa441dff3c1f203c16e144d8d16d 0 [] map[] <nil>}"
DEBU[0007] unpack layer                                  desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f 35197 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:ccf35b27430d91d07564e70d2ea340563697e4a91dfe03c40675d906bd7059a2 0 [] map[] <nil>}"
DEBU[0008] layer unpacked                                desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:3c087f8bf83d28baa9c53fc5cfb0dfc79d5062cfad77563076553b04259e822f 35197 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:ccf35b27430d91d07564e70d2ea340563697e4a91dfe03c40675d906bd7059a2 0 [] map[] <nil>}"
DEBU[0008] unpack layer                                  desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a 849 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:f0a302c91d12daa5f4d7a8ca4e3b66464190f53539cc36f6b1c15d6c09894d65 0 [] map[] <nil>}"
DEBU[0008] layer unpacked                                desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:474b5c43e9a24e1958763941a17a7cac60b13460a47f2ae9067ec03ceadab33a 849 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:f0a302c91d12daa5f4d7a8ca4e3b66464190f53539cc36f6b1c15d6c09894d65 0 [] map[] <nil>}"
DEBU[0008] unpack layer                                  desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce 189 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:307fb5586e3d8e2da3ad3e01f1fe9637af6efb8e21522dedd92cfd9909695941 0 [] map[] <nil>}"
DEBU[0008] layer unpacked                                desc="{application/vnd.docker.image.rootfs.diff.tar.gzip sha256:7083e6591d76b9a6c13497a6a89bfbae3e7eb2eb31bec366bec3884c8e4517ce 189 [] map[] <nil>}" diff="{application/vnd.oci.image.layer.v1.tar sha256:307fb5586e3d8e2da3ad3e01f1fe9637af6efb8e21522dedd92cfd9909695941 0 [] map[] <nil>}"
DEBU[0008] image unpacked                                chainID="sha256:4f584c5554ce70e1d50b3019b4d2ea8b3bb71341f2f0109dad09045d0e7a9f98" config="sha256:12a3cf2229154ace97979da9f77ffffa3c93344de6455dd132a6c8e8902392e7"
DEBU[0008] containerd: Inspecting image "ubuntu:18.04"  
TRAC[0008] Client.New; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0008] index: counted 0 Image object(s)             
TRAC[0008] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0008] index: counted 0 Image object(s)             
TRAC[0008] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
INFO[0008] Starting image import...                     
DEBU[0008] Allocating image file and formatting it with ext4... 
DEBU[0008] Copying in files to the image file from a source... 
DEBU[0011] Retrieving minimum size for "/dev/loop5"     
DEBU[0011] Minimum size: 23164 blocks                   
DEBU[0014] Truncating "/var/lib/firecracker/image/d48dcacec72cae29/image.ext4" to 94879744 bytes 
TRAC[0014] Client.Set; UID: "d48dcacec72cae29", GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0014] cache: Set Image with UID "d48dcacec72cae29" 
TRAC[0014] index: storing Image object with UID "ubuntu:18.04", meta: false 
INFO[0014] Imported OCI image "ubuntu:18.04" (57.4 MB) to base image with UID "d48dcacec72cae29" 
DEBU[0014] Ensuring kernel weaveworks/ignite-kernel:4.19.47 exists, or importing it... 
TRAC[0014] Client.Find; GVK: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0014] index: counted 0 Kernel object(s)            
TRAC[0014] cache: miss when listing: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0014] cache: Get Kernel with UID "b1aeb0d89009fb9e" 
TRAC[0014] index: cache miss for Kernel with UID "b1aeb0d89009fb9e" 
TRAC[0014] index: storing Kernel object with UID "weaveworks/ignite-kernel:4.19.47", meta: false 
DEBU[0014] Found kernel with UID b1aeb0d89009fb9e       
TRAC[0014] index: counted 0 VM object(s)                
TRAC[0014] cache: hit when listing: ignite.weave.works/__internal, Kind=VM 
TRAC[0014] index: listing ignite.weave.works/__internal, Kind=VM objects, meta: true 
TRAC[0014] index: counted 0 VM object(s)                
TRAC[0014] cache: miss when listing: ignite.weave.works/__internal, Kind=VM 
TRAC[0014] Client.Set; UID: "8b7d7ffa701f5d29", GVK: ignite.weave.works/__internal, Kind=VM 
TRAC[0014] cache: Set VM with UID "8b7d7ffa701f5d29"    
TRAC[0014] index: storing VM object with UID "hidden-dew", meta: false 
TRAC[0014] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0014] index: counted 1 Image object(s)             
TRAC[0014] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0014] cache: Get Image with UID "d48dcacec72cae29" 
TRAC[0014] index: cache hit for Image with UID "d48dcacec72cae29" 
TRAC[0014] cacheObject: "ubuntu:18.04" invalidated, checksum mismatch: "" -> "1590679611186672498" 
TRAC[0014] cacheObject: full load triggered for "ubuntu:18.04" 
TRAC[0014] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0014] index: counted 1 Image object(s)             
TRAC[0014] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0014] cache: Get Image with UID "d48dcacec72cae29" 
TRAC[0014] index: cache hit for Image with UID "d48dcacec72cae29" 
TRAC[0014] cacheObject: "ubuntu:18.04" checksum: "1590679611186672498" 
TRAC[0014] Client.Find; GVK: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0014] index: counted 1 Kernel object(s)            
TRAC[0014] cache: miss when listing: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0014] cache: Get Kernel with UID "b1aeb0d89009fb9e" 
TRAC[0014] index: cache hit for Kernel with UID "b1aeb0d89009fb9e" 
TRAC[0014] cacheObject: "weaveworks/ignite-kernel:4.19.47" checksum: "1590678960396609169" 
INFO[0019] Created VM with ID "8b7d7ffa701f5d29" and name "hidden-dew" 
TRAC[0019] Client.Find; GVK: ignite.weave.works/__internal, Kind=Image 
TRAC[0019] index: counted 1 Image object(s)             
TRAC[0019] cache: miss when listing: ignite.weave.works/__internal, Kind=Image 
TRAC[0019] cache: Get Image with UID "d48dcacec72cae29" 
TRAC[0019] index: cache hit for Image with UID "d48dcacec72cae29" 
TRAC[0019] cacheObject: "ubuntu:18.04" checksum: "1590679611186672498" 
TRAC[0019] Client.Find; GVK: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0019] index: counted 1 Kernel object(s)            
TRAC[0019] cache: miss when listing: ignite.weave.works/__internal, Kind=Kernel 
TRAC[0019] cache: Get Kernel with UID "b1aeb0d89009fb9e" 
TRAC[0019] index: cache hit for Kernel with UID "b1aeb0d89009fb9e" 
TRAC[0019] cacheObject: "weaveworks/ignite-kernel:4.19.47" checksum: "1590678960396609169" 
DEBU[0019] containerd: Inspecting image "weaveworks/ignite:v0.6.3" 
DEBU[0019] Writing "/var/lib/firecracker/vm/8b7d7ffa701f5d29/runtime.containerd.resolv.conf" with new hash: "c75f87fd85fd9326e72b6b2c87edaeae44e56231d2af917193ed59a624c904da", old hash: "" 
INFO[0023] Networking is handled by "cni"               
INFO[0023] Started Firecracker VM "8b7d7ffa701f5d29" in a container with ID "ignite-8b7d7ffa701f5d29" 
FATA[0033] timeout waiting for ignite-spawn startup   ```
darkowlzz commented 4 years ago

Hi, thanks for trying ignite. ARM support was added recently in https://github.com/weaveworks/ignite/pull/511 and is not part of release v0.6.3 or before. We have a new document about Using Ignite on ARM but it doesn't goes into details about how to build all the container images for arm64. You can run the make targets in the images/ directory with appropriate variables to build it yourself. Or you can use the images I built for testing it.

If you build ignite binary yourself for arm64, you can also use the arm64 sandbox image listed above. The latest version of ignite (master branch), has an option to set the sandbox image using the flag --sandbox-image with the ignite run command. That can be used to pass the sandbox image.

We'll be releasing a new version of ignite in the coming weeks with all the binaries and conatiner images with arm64 support.

The error in the logs is possibly due to the failure in the sandbox image. Using an arm64 specific sandbox image, kernel image and OS image should fix that issue.

Hope this helps.

mehstg commented 4 years ago

Excellent. Thanks for the help

stealthybox commented 4 years ago

@mehstg We are looking to release ARM support this week.

mehstg commented 4 years ago

That's great. I will be looking out for it!

stealthybox commented 4 years ago

@mehstg all of the images and binaries are pushed. Let us know if you end up trying again :) https://github.com/weaveworks/ignite/releases/tag/v0.7.0