gen.sh from non-debian distro?

[mdeguzis]

There is no notice that gen.sh should only work on Debian-based systems, but apt-utils is a dep:

#Check some basic dependencies this script needs to run
deps ( ) {
    #Check dependencies
    deps="apt-utils xorriso syslinux rsync wget p7zip-full realpath"

If, say on Arch Linux, there is apt on Arch.

[directhex]

The only command used from apt-utils is /usr/bin/apt-ftparchive. Is that in Arch?

[mdeguzis]

Yes. Could this be written in as detection for Arch users? Either that, or I could submit some code to account for it, like distro_check=$(lsb_release -i | cut -c 17-25). Or, could be checking the path to the binary.

[mdeguzis]

This is also true for packages like '/usr/bin/realpath' on Arch, from package "core utils". It's messy, but I currently just do a dirty sed replace on the deps line to what I need. I can list you the packages that provide you deps, with the /usr/bin line if you want.

[directhex]

It'd probably be smarter as an immediate solution to validate the specific commands we use ie "if [ -x /usr/bin/apt-ftparchive ] then" or "which apt-ftparchive &&" (and suggest apt-gets to match), that way it'll still make suggestions assuming a deb distro without actually enforcing it

[mdeguzis]

Makes sense, and as a fallback, offer to install said packages. Below are what packages are needed by Arch, and I can happily supply the commands if you would like to write in support for them. If not, and you make this change, that would allow me and others to install them ahead of time, have the binary checked and recognized, and avoid failure in the script.

• apt-utils, provided by apt (AUR), /usr/bin/apt-ftparchive,
• xorriso, provided by libisoburn
• syslinux, provided by package syslinux
• rsync, provided by package rsync
• wget, provided by package wget
• p7zip-full, provided by package p7zip
• realpath, provided by package coreutils

[mdeguzis]

For whatever reason of insanity, I am making a "steamos-mega-downloader" to pull in and burn/image steamos to a USB drive. Whatever you decide is fine, i'll work around anything you are ok with. I forked your repo for now. I built in pkg lists checks that will install the needed packages as a workaround. Agreed it should validate the specific commands, but handling automatic package install is another matter. Maybe just put in the readme the needed packages for Debian/SteamOS or Arch Linux would be easiest.

[mdeguzis]

I wonder if it's not possible to build this on Arch Linux: http://slexy.org/view/s2rAIriWkS

[sharkwouter]

This would be pretty darn cool, more distros should be able to run this.

[mdeguzis]

On another note, syslinux does not provide isohdpfx.bin on SteamOS. If you do add any type of support for other distros,which I am willing to help with, isolinux contains this file, which is not in the dependency list.

[sharkwouter]

The package isolinux provides that file on debian jessie.

[mdeguzis]

I don't remove packages Valve provide. It is then my guess that this is already included with SteamOS, and is a modified package. I can't check the pool from where I am (blocked), so I'd imagine it pulled from there first before I added sources for xorriso.

[sharkwouter]

Reprepro might be able to replace create the dist directories for the package pool of the disk. Documentation can be found here: http://www.red-bean.com/doc/reprepro/manual.html

[mdeguzis]

@sharkwouter , it can. Using reprepro is a matter of preference. It does a lot of things well, and automatically, but does have it's drawbacks. For me, as fairly fresh to packaging frequently now, it's great, but i'm sure @directhex has his reasons for using aptarchive.

[sharkwouter]

I actually managed to get a functioning iso using reprepro. I made the following changes:

--- a/gen.sh
+++ b/gen.sh
@@ -302,11 +302,18 @@ createiso ( ) {
        fi

        #Generate our new repos
-       echo "Generating Packages.."
-       apt-ftparchive generate ${APTCONF}
-       apt-ftparchive generate ${APTUDEBCONF}
-       echo "Generating Release for ${DISTNAME}"
-       apt-ftparchive -c ${APTCONF} release ${BUILD}/dists/${DISTNAME} > ${BUILD}/dists/${DISTNAME}/Release
+       #echo "Generating Packages.."
+       #apt-ftparchive generate ${APTCONF}
+       #apt-ftparchive generate ${APTUDEBCONF}
+       #echo "Generating Release for ${DISTNAME}"
+       #apt-ftparchive -c ${APTCONF} release ${BUILD}/dists/${DISTNAME} > ${BUILD}/dists/${DISTNAME}/Release
+       mv ${BUILD}/pool ${BUILD}/poolbase
+       rm -rf ${BUILD}/dists
+       mkdir ${BUILD}/conf
+       /bin/echo -e "Origin: Valve Software LLC\nSuite: testing\nCodename: ${DISTNAME}\nComponents: main contrib non-free\nUDebComponents: main\nArchitectures: i386 amd64\nDescription: SteamOS distribution based on Debian 8.0 Jessie\nContents: udebs . .gz\nUDebIndices: Packages . .gz" > ${BUILD}/conf/distributions
+    reprepro -Vb ${BUILD} includedeb ${DISTNAME} ${BUILD}/poolbase/*/*/*/*.deb
+    reprepro -Vb ${BUILD} includeudeb ${DISTNAME} ${BUILD}/poolbase/*/*/*/*.udeb
+    rm -rf ${BUILD}/poolbase ${BUILD}/db ${BUILD}/conf

        #gpg --default-key "0E1FAD0C" --output $BUILD/dists/$DISTNAME/Release.gpg -ba $BUILD/dists/$DISTNAME/Release
        cd ${BUILD}

Still early, but still pretty cool. I should try to get this running on Arch or something.

[mdeguzis]

That's just simple operation of the includedeb piece of reprepro. Normally, what I do (after I learned more) is point include to the folder that has the .changes files. If you have multiple .changes files, to avoid confusing reprepro, you setup a incoming hook. This will generate and update all the archives, signing, DISTS, db/, etc. I designate two sections in my processing incoming, one named default, and one testing. Because dpkg doensn't like an underscore in the DIST (e.g. brewmaster_testing), I have a script that parses a folder called incoming_testing, and changes brewmaster to brewmaster_testing, then calls reprepro processincoming testing. Once I properly setup processincoming's config file, I can mass export and handle packages, rather than build and sync one package at a time.

I don't generally use apt-ftparchive, so I may be approaching your use of this wrong.

[sharkwouter]

I forked this repo to do some more testing with this: https://github.com/sharkwouter/stephensons-rocket

I managed to build a working ISO in Fedora with it and these instructions: https://github.com/sharkwouter/stephensons-rocket/blob/brewmaster/fedora-instructions.txt

Unfortunately I had to manually compile reprepro and xorriso. Reprepro is not in the Fedora repo and the compile of xorriso in it hasn't been compiled with the required flags.

Changes I've made are adding the isohdpfx.bin file from Debian in commit https://github.com/sharkwouter/stephensons-rocket/commit/6fdef92872903458b9302194a29825fd89c1efa9 and making the following changes the gen.sh script: https://github.com/sharkwouter/stephensons-rocket/commit/1a6ebc52c707dff27e58ce792b1b12defdafc252

The code is still pretty rough, but it does work. You should be able to easily get this working on Arch or any other distro as well.

[mdeguzis]

Nice, i'll do some testing with this over the weekend. I'll checkover the option to build the latest iso from my downloader script.

[sharkwouter]

Stepheson's Rocket now builds on multiple distros. Check REQUIREMENTS.md for details on how to set up your distro for building. If your distro is missing, please consider committing a pull request with instructions for it.