search

travisghansen / gentoo-cloud-image-builder

Create gentoo openstack-compatible images
15 stars 5 forks source link

curl: (3) URL rejected: Malformed input to a URL function #3

Closed NucleaPeon closed 1 year ago

NucleaPeon commented 1 year ago

Did a git clone and immediately ran ./launch.sh -t build. Output is this:

gentoo-cloud-image-builder $ ./launch.sh -t build
downloading current iso -----BEGIN
Hash:
install-amd64-minimal-20231022T164658Z.iso
-----BEGIN
iQEzBAEBCAAdFiEEU05CCatJ7uHBnZYWLERpXbn2BD0FAmU6VQEACgkQLERpXbn2
cti
zw3sLblc
DzYn+laAsTxIVx13HWn5K7N5cTxnCKGG4BglQJO92xVLd2qDSN+EQe6fhPzfIUtb
fr3VBlysZ1FjPGrbxkUONMKU+XPlxJSURrvxNgkfW
a6OCsTQ1qtVLTxLpFAMEwe8riQkvqXhSD2PK5lpkPqJ4YXMvnTsVtbvHq42Ttatz
yLIe815+vOpU7krdSNASKpDjtQ6cVw==
=PQZy
-----END
curl: (3) URL rejected: Malformed input to a URL function
downloading current stage <!DOCTYPE
<html><head>
<title>404
head><body>
<h1>Not
<p>The
<hr>
<address>Apache
body><
curl: (3) URL rejected: Malformed input to a URL function
downloading current portage portage-20231025.tar.bz2
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   317  100   317    0     0    613      0 --:--:-- --:--:-- --:--:--   613
creating gentoo.img image file
Formatting 'gentoo.img', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=10737418240 lazy_refcounts=off refcount_bits=16
creating builder iso
creating config iso
char device redirected to /dev/pts/13 (label charserial1)
qemu-system-x86_64: -cdrom iso/-----BEGIN
Hash:
install-amd64-minimal-20231022T164658Z.iso
-----BEGIN
iQEzBAEBCAAdFiEEU05CCatJ7uHBnZYWLERpXbn2BD0FAmU6VQEACgkQLERpXbn2
cti
zw3sLblc
DzYn+laAsTxIVx13HWn5K7N5cTxnCKGG4BglQJO92xVLd2qDSN+EQe6fhPzfIUtb
fr3VBlysZ1FjPGrbxkUONMKU+XPlxJSURrvxNgkfW
a6OCsTQ1qtVLTxLpFAMEwe8riQkvqXhSD2PK5lpkPqJ4YXMvnTsVtbvHq42Ttatz
yLIe815+vOpU7krdSNASKpDjtQ6cVw==
=PQZy
-----END: Could not open 'iso/-----BEGIN
Hash:
install-amd64-minimal-20231022T164658Z.iso
-----BEGIN
iQEzBAEBCAAdFiEEU05CCatJ7uHBnZYWLERpXbn2BD0FAmU6VQEACgkQLERpXbn2
cti
zw3sLblc
DzYn+laAsTxIVx13HWn5K7N5cTxnCKGG4BglQJO92xVLd2qDSN+EQe6fhPzfIUtb
fr3VBlysZ1FjPGrbxkUONMKU+XPlxJSURrvxNgkfW
a6OCsTQ1qtVLTxLpFAMEwe8riQkvqXhSD2PK5lpkPqJ4YXMvnTsVtbvHq42Ttatz
yLIe815+vOpU7krdSNASKpDjtQ6cVw==
=PQZy
-----END': File name too long

This doesn't seem to have worked. I can look into why when I have time. Host: gentoo amd64 with dependencies installed.

travisghansen commented 1 year ago

Obviously hasn’t been touched in ages, but I’ll take a look and see if I can modernize it :)

NucleaPeon commented 1 year ago

This project comes up on the first page of search results for "Gentoo cloud-image", so there's that. Appreciate it! If this isn't a project you're interested in anymore, I can look into doing it myself.

travisghansen commented 1 year ago

I almost have updates done that make it 0 effort again.

travisghansen commented 1 year ago

OK, just committed some updates. Enjoy!

NucleaPeon commented 1 year ago

Thanks! Sorry for the delay in my response; I did get it working, but there was one error I had to overcome and one improvement I want to make, so I'm forking your repo, applying the patches and creating a PR.

While investigating the fix, I used my local gentoo rsync repo so I wouldn't spam remote providers and this is the code I used to do so, in case it helps anyone.

builder/full_install.sh

 echo "copying repos.conf"
 mkdir --parents ${R}/etc/portage/repos.conf
-cp /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf
+# cp /mnt/gentoo/usr/share/portage/config/repos.conf /mnt/gentoo/etc/portage/repos.conf/gentoo.conf
+cat > ${R}/etc/portage/repos.conf/gentoo.conf << EOF
+[DEFAULT]
+main-repo = gentoo
+
+[gentoo]
+location = /var/db/repos/gentoo
+sync-type = rsync
+sync-uri = rsync://192.168.254.20/gentoo-portage
+auto-sync = yes
+sync-rsync-verify-jobs = 1
+sync-rsync-verify-metamanifest = yes
+sync-rsync-verify-max-age = 24
+#sync-openpgp-key-path = /usr/share/openpgp-keys/gentoo-release.asc
+#sync-openpgp-keyserver = hkps://keys.gentoo.org
+#sync-openpgp-key-refresh-retry-count = 40
+#sync-openpgp-key-refresh-retry-overall-timeout = 1200
+#sync-openpgp-key-refresh-retry-delay-exp-base = 2
+#sync-openpgp-key-refresh-retry-delay-max = 60
+#sync-openpgp-key-refresh-retry-delay-mult = 4
+#sync-webrsync-verify-signature = yes
+EOF

I also experienced one verify error on a sync before this point, but it wasn't reproduceable.

Anyhow, I'd like to have my PR merged before closing this issue since people will get an error while running full_install.sh.

But seriously, thank you for working on this!

travisghansen commented 1 year ago

I ran into the verification error as well. Frankly I would prefer it not run a sync during the script but the portage tarballs apparently now lack some bits required to use portage (I haven’t used gentoo in a while so I could be off here) due to a missing db or something. If you know how to rebuild the db without doing a sync I would update to that.

NucleaPeon commented 1 year ago

I'm not that familiar with the internals of portage unfortunately. There is a emerge-websync command that is more stable, I think it grabs an older update through https and it has never had verification issues in my experience. If I find time I can look into that command and see if/what it does to the portage db, it might/should be possible to replicate it in a script.