lf-edge / eden

Eden is where EVE and Adam get tried and tested:
https://projecteve.dev
Apache License 2.0
50 stars 48 forks source link

Fix fetch ovmf files #946

Closed europaul closed 9 months ago

europaul commented 9 months ago

Between docker engine versions 24 and 25 the format of how docker stores images in tar files on disk was changed from

0b57d9f51ad5bac74c477a3266fff9642c591727b83e7d6add0151a3c7389482/layer.tar"
326204e656f636fd616113cf1e6be6d856111a5ff930d2ab0b3190819e5a0940/layer.tar"
3f4d09dc1da74b14077735090daef253abf8ede68eac9c2a63deadd2e195d617/layer.tar"
73f03a1aaf81f0f67d60e5e89b3a8cb24a7d111fb50fdeca939471c3e1ac7399/layer.tar"
7a5f3ff7cfa848ac27c946f4730563231f299a67d03cb81b314b29777a304813/layer.tar"
908dff9063c73a25293eeb76e324c2854a592f1c946bf01a607c0372a8f8c6f7/layer.tar"
a63fcacd55224b42d8ec83c6d29bbce9101361a52a992d6eec67731a6a1b6b97/layer.tar"
ad6965eba0c89534556d3ad8fb549bc85e47f882c408c9a72205d89a2ac0a2a1/layer.tar"
b0a7b7417fce3a4725aae67b8628514625a7bbbf7d25b892ab5c05d95107cc6c/layer.tar"

to

blobs/sha256/06c823d744406518ea8ced756506a4a164f16d9e18783def87c786088fc34a43
blobs/sha256/14e332bbc80735cef4f91e9f99f83ed795fa6e1e0acc24a069a77f7e502d8d27
blobs/sha256/2a110bb756a43e416f4fbc723e4e8e997c5f58ec47c562e81926bdbc3303e7d0
blobs/sha256/4176c3e0f4a50f066062f232860b22f5759ccfa74aabb3f67cfa96dd84e39458
blobs/sha256/453a0eb4b492282f5a2b19bfe59029df29e5be90c0522d0723b84cff6a4bac82
blobs/sha256/4f49855f91bee5d3fa4aafbf7b85d0428bfacb6bd2743ff099874619c0b7849f
blobs/sha256/5b31dc02100d77113630b0f8579bb56d404d39dd3aa2de2d0bb9a108a3c15dc2
blobs/sha256/5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef
blobs/sha256/65c0461e49665472b468323f4f17f870af97573377fa177563277bd6e220d6ea
blobs/sha256/6fbaf1656cb034d194fd548b952f7750798e8d7948d28a0fac7ee0ff68743825
blobs/sha256/7e87142ff81d92f861a30dc93d746032b335ec5ceee924cb86bd9f18a5ae8626
blobs/sha256/860426fe07ac525e03e249e0af2dd402165540eaa6292f6f7b8b2ec65619041f
blobs/sha256/88931268ad454ac44e85c6f69164546cb6b7da17f76f04a7765b31d2e7b52339
blobs/sha256/88aa4fd8f520a3bc6dd32a035aba2740f5731c8ff574b03868036eb593919db8
blobs/sha256/8fa83a0a6d0db8a34a4a0a04e45ba92ba0cfed01281d1ec8921807f7051b389a
blobs/sha256/afe632d8938cc7bfce6e5677212d004d846d2597cee0947c5c306b6c33f6e2c2
blobs/sha256/b43580de0917f84cd0234575c02cfb43ca0c7c3064243b65fccfdd8454e75518
blobs/sha256/ddb35c5eca0391dcaec28b75771391103cde1514ca88e65b577ed00100a1b756
blobs/sha256/e35135e8d4833195853e90c5c47989ef6ce49bed913f70623a11de3c7645f989
blobs/sha256/faf7d1c3ed31304554e0b4d8de04ac4a07d6cfa0e6c5317dd219d6ef7ea6149a

This breaks our current approach to save images as tar files and parse them for the necessary files.

Since we can no longer support multiple formats of those tar files, this commit changes the way we fetch files from docker images to use the docker API instead of relying on docker internals.

The new way to fetch files can be expressed through the following docker CLI commands:

id=$(docker create image-name)
docker cp $id:path local-tar-file
docker rm -v $id
tar -xvf local-tar-file
deitch commented 9 months ago

I think this is the mainstreaming of the switch to containerd under the covers. It opens up plenty of vistas for better ways of doing things, and has been in the works for almost two years. FWIW, this is why linuxkit has its own cache, so it can do these things. Once containerd is fully enabled in docker engine, we can deprecate the linuxkit cache.

deitch commented 9 months ago

FYI yetus is complaining

europaul commented 9 months ago

FYI yetus is complaining

fixed most yetus complains - the ones that are remaining are tolerable imo.

eriknordmark commented 9 months ago

@europaul testing looks good. Would it make sense to add the comments/config to make yetus shut up about cyclo and add the missing return value check?

europaul commented 9 months ago

@europaul testing looks good. Would it make sense to add the comments/config to make yetus shut up about cyclo and add the missing return value check?

Yes, I disabled the cyclop linter globally - I think it nags more than it helps us to write simple well-structured functions.

Added the return value check.