search

SUSE / rookcheck

Apache License 2.0
4 stars 11 forks source link

Image files are always downloaded multiple times. #206

Open kshtsk opened 3 years ago

kshtsk commented 3 years ago

Rookcheck does not cache big image files while trying to download them, it is waste of traffic and time. For example, given we have this log:

Downloading image from http://download.suse.de/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2

Before actually download it would be great to check digest of the file if possible:

curl -I http://download.suse.de/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2
HTTP/1.1 302 Found
date: Tue, 20 Oct 2020 22:50:05 GMT
server: Apache/2.4.43 (Linux/SUSE)
x-mirrorbrain-mirror: dist.suse.de
x-mirrorbrain-realm: prefix
link: <http://download.suse.de/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2.meta4>; rel=describedby; type="application/metalink4+xml"
link: <http://dist.suse.de/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2>; rel=duplicate; pri=1; geo=de
link: <http://dist.nue.suse.com/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2>; rel=duplicate; pri=2; geo=de
link: <http://mirror.suse.cz/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2>; rel=duplicate; pri=3; geo=cz
link: <http://ibs-mirror.prv.suse.net/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2>; rel=duplicate; pri=4; geo=us
link: <http://mirror.suse.asia/dist/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2>; rel=duplicate; pri=5; geo=cn
digest: MD5=53HuoyARHkQAPsvn/pYCPg==
digest: SHA=sf/H3cUKiQsuN/n9xR5swIsu3Q4=
digest: SHA-256=vx5V125wORLGjB81i+qhDXSuXhHKJmLC4Ye89NNbgW8=
location: http://dist.suse.de/install/SLE-15-SP2-JeOS-GM/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2
content-type: text/html; charset=iso-8859-1

Any from md5, sha, and sha-256 can be used.

Just for illustration:

> sha256sum /tmp/rookcheck/rookcheck-kyr-596a/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2
bf1e55d76e703912c68c1f358beaa10d74ae5e11ca2662c2e187bcf4d35b816f  /tmp/rookcheck/rookcheck-kyr-596a/SLES15-SP2-JeOS.x86_64-15.2-OpenStack-Cloud-GM.qcow2

and

echo vx5V125wORLGjB81i+qhDXSuXhHKJmLC4Ye89NNbgW8= | base64 -d - | xxd -c 32 -ps
bf1e55d76e703912c68c1f358beaa10d74ae5e11ca2662c2e187bcf4d35b816f
jhesketh commented 3 years ago

Yep, this would be a welcome improvement. However, if you have the qcow2 image already on your filesystem, you can point ROOKCHECK_LIBVIRT__IMAGE to it and it should use that rather then fetching from http.

kshtsk commented 3 years ago

Yep, this would be a welcome improvement. However, if you have the qcow2 image already on your filesystem, you can point ROOKCHECK_LIBVIRT__IMAGE to it and it should use that rather then fetching from http.

Thanks, but I knew that.