Closed ChengyuZhu6 closed 2 months ago
Needs to add related unit tests.
I’ve added code and unit tests to enable the reuse of the meta store and layers in the image client.
cc @arronwy @Xynnn007 @bpradipt
The template of meta_store.json
is as follows:
{
"image_db": {
"sha256:fce289e99eb9bca977dae136fbe2a82b6b7d4c372474c9235adc1741675f587e": {
"id": "sha256:fce289e99eb9bca977dae136fbe2a82b6b7d4c372474c9235adc1741675f587e",
"digest": "sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a",
"reference": "mcr.microsoft.com/hello-world",
"image_config": {
"created": "2019-01-01T01:29:27.650294696Z",
"architecture": "amd64",
"os": "linux",
"config": {
"User": "",
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/hello"
],
"WorkingDir": ""
},
"rootfs": {
"type": "layers",
"diff_ids": [
"sha256:af0b15c8625bb1938f1d7b17081031f649fd14e6b233688eea3c5483994a66a3"
]
},
"history": [
{
"created": "2019-01-01T01:29:27.416803627Z",
"created_by": "/bin/sh -c #(nop) COPY file:f77490f70ce51da25bd21bfc30cb5e1a24b2b65eb37d4af0c327ddc24f0986a6 in / "
},
{
"created": "2019-01-01T01:29:27.650294696Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/hello\"]",
"empty_layer": true
}
]
},
"signed": false,
"layer_metas": [
{
"decoder": "Gzip",
"encrypted": false,
"compressed_digest": "sha256:1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced",
"uncompressed_digest": "sha256:af0b15c8625bb1938f1d7b17081031f649fd14e6b233688eea3c5483994a66a3",
"store_path": "/tmp/.tmpYioDI2/layers/sha256_1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced"
}
]
}
},
"layer_db": {
"sha256:1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced": {
"decoder": "Gzip",
"encrypted": false,
"compressed_digest": "sha256:1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced",
"uncompressed_digest": "sha256:af0b15c8625bb1938f1d7b17081031f649fd14e6b233688eea3c5483994a66a3",
"store_path": "/tmp/.tmpYioDI2/layers/sha256_1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced"
}
},
"snapshot_db": {}
}
The template of
meta_store.json
is as follows:{ "image_db": { "sha256:fce289e99eb9bca977dae136fbe2a82b6b7d4c372474c9235adc1741675f587e": { "id": "sha256:fce289e99eb9bca977dae136fbe2a82b6b7d4c372474c9235adc1741675f587e", "digest": "sha256:92c7f9c92844bbbb5d0a101b22f7c2a7949e40f8ea90c8b3bc396879d95e899a", "reference": "mcr.microsoft.com/hello-world", "image_config": { "created": "2019-01-01T01:29:27.650294696Z", "architecture": "amd64", "os": "linux", "config": { "User": "", "Env": [ "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" ], "Cmd": [ "/hello" ], "WorkingDir": "" }, "rootfs": { "type": "layers", "diff_ids": [ "sha256:af0b15c8625bb1938f1d7b17081031f649fd14e6b233688eea3c5483994a66a3" ] }, "history": [ { "created": "2019-01-01T01:29:27.416803627Z", "created_by": "/bin/sh -c #(nop) COPY file:f77490f70ce51da25bd21bfc30cb5e1a24b2b65eb37d4af0c327ddc24f0986a6 in / " }, { "created": "2019-01-01T01:29:27.650294696Z", "created_by": "/bin/sh -c #(nop) CMD [\"/hello\"]", "empty_layer": true } ] }, "signed": false, "layer_metas": [ { "decoder": "Gzip", "encrypted": false, "compressed_digest": "sha256:1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced", "uncompressed_digest": "sha256:af0b15c8625bb1938f1d7b17081031f649fd14e6b233688eea3c5483994a66a3", "store_path": "/tmp/.tmpYioDI2/layers/sha256_1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced" } ] } }, "layer_db": { "sha256:1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced": { "decoder": "Gzip", "encrypted": false, "compressed_digest": "sha256:1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced", "uncompressed_digest": "sha256:af0b15c8625bb1938f1d7b17081031f649fd14e6b233688eea3c5483994a66a3", "store_path": "/tmp/.tmpYioDI2/layers/sha256_1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced" } }, "snapshot_db": {} }
Thanks @ChengyuZhu6 for fixing this. Few questions As you know one of the aspects I'm experimenting with is to figure out a way to embed image layers in rootfs to avoid downloading the image layers again during pod creation. So for this scenario as I understand from this PR I need to do the following
Is my understanding correct? Would you know of a way (tool/script etc) to create meta_store.json ?
As you know one of the aspects I'm experimenting with is to figure out a way to embed image layers in rootfs to avoid downloading the image layers again during pod creation.
image-rs will download the image only once, even if multiple containers in a pod use same image.
Populate image layers in a specific path in rootfs Create a meta_store.json file to indicate the layer details and the path inside the rootfs Ensure this meta_store.json is reference by kata-agent/image-rs to reuse the embedded layers
Just a reminder: we don't need to populate the image layers to rootfs. Instead, we should store the layers in the image-rs workdir/layers
directory. For example, in kata, the layers (such as sha256:1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced
) should be stored in /run/kata-containers/image/layers/sha256_1b930d010525941c1d56ec53b97bd057a67ae1865eebf042686d2a2d18271ced
. Image-rs will then overlay the layers onto rootfs.
Would you know of a way (tool/script etc) to create meta_store.json ?
I'm not sure if there is an existing tool or script for this, but it sounds like an interesting idea.
Support to write
meta_store
tometa_store.json
. Set the absolute path for meta store withmeta_store.json