search

monome / norns-image

79 stars 16 forks source link

Build image automatically (CI) #9

Closed simonvanderveldt closed 6 years ago

simonvanderveldt commented 6 years ago

We'll need to figure out how to automatically build the image, especially useful during development so everyone who has a device is actually using the same image, but obviously also nice once real releases start happening.

Current notes for how to build the image live here https://github.com/tehn/norns-image/blob/master/image.txt

ranch-verdin commented 6 years ago

moved the instructions for wifi hotspot setup to image.txt in https://github.com/tehn/norns-image/pull/8

simonvanderveldt commented 6 years ago

I'm taking a look at https://github.com/RPi-Distro/pi-gen which is the official "tool" (i.e. bunch of scripts) used by the raspberry foundation to build raspbian. Luckily there is a script to build inside a container so it's nicely isolated and should be relatively easy to run on something like Travis. Unfortunately that script isn't really perfect, it will need some tweaking to work in a usable manner.

Apart from that I'm not getting many warm and fuzzy feelings from the results I've seen so far, I'm starting with building the stage2, which builds the "lite" image variant. Stage1 just before that one already installs and changes quiet some things that I don't see any reason for to install. Same with the debootstrap used as the initial starting point. Will have to take a look if it's possible to get rid of these things before they are even getting built/installed.

@tehn How married are we to raspbian? :roll_eyes:

tehn commented 6 years ago

this certainly came up earlier--- possibly switching to arch.

it's feasible. i think there will be some unexpected hurdles. of course what's of primary importance is that the kernel is actively being worked on (https://github.com/raspberrypi/linux) which is not tied to raspbian.

part of using raspbian is about timesaving given that we got the audio system working well with the RT kernel. switching would mean getting a lot of things working quickly. i'd say it's worth a shot but it's outside my experience and there are a ton of end-user-specific issues i need to focus on.

on the other hand, starting with a stable raspbian lite and stripping it back (with a script) doesn't seem like a bad starting place either. unless this is a more broad debian vs. arch issue?

here's a list of the custom things that would need to work beyond a basic install on a consumer rpi3:

instructions (sometimes vague) contained in image.txt

also-- way long ago when we were doing the sound codec driver, murray attempted a travis CI for the kernel and it ran up against travis' time limit for the process-- perhaps there's a way around this with a cross-compiling config.

all thoughts welcome.

simonvanderveldt commented 6 years ago

@tehn I'll continue exploring raspbian for now, so I can properly evaluate the pro's and con's.

One important question though: How much of the Linux part of norns do you want to make (easily) accessible to the users? Should they be able to simply install some packages or would there only be a complete fixed norns image to install and that's it?

tehn commented 6 years ago

a complete image would be sufficient. if people want to hack it, the formulas you're creating will facilitate that. i think it'd be a lot of extra work to turn everything into packages-- modding the image will be somewhat less common of a use case.

simonvanderveldt commented 6 years ago

a complete image would be sufficient. if people want to hack it, the formulas you're creating will facilitate that. i think it'd be a lot of extra work to turn everything into packages-- modding the image will be somewhat less common of a use case.

@tehn If the "unit of delivery" will always be an image we might be better served with something like buildroot, which allows one to define which kernel and "packages" the install, which settings to change and will automatically build an image based on that. There's already support for the Raspberry Pi in there as well. Run-time there's no package manager, so the images are "fixed" in that regard. Might make support a bit easier as well because there are less things the user can mess up this way :)

tehn commented 6 years ago

agreed. for example, if you apt-get upgrade now it will kill our custom kernel-- @catfact learned this recently as i did not post a big warning.

i checked out buildroot but haven't had time to dig in, so it'd be amazing if you wanted to take a look

(edited, upgrade, not update)

ranch-verdin commented 6 years ago

I stumbled upon this earlier, which could be a starting point: https://github.com/gamaral/rpi-buildroot the author posted a video of an exceptionally fast boot into a game engine: https://www.youtube.com/watch?v=4Fjfqz6FxC8

@tehn oops, I also ran apt-get update - could that explain why my system runs the OLED smoothly with dtoverlay-loaded driver?

tehn commented 6 years ago

yes, i've seen this video! it's a wonderfully fast boot.

@ranch-verdin does your sound card still work? hard to know why your OLED would work smoothly, which is why we all need to get on the same image. ---- EDIT: did you upgrade or update? upgrade is bad, update is ok.

ranch-verdin commented 6 years ago

@tehn just tested & the sound card is still working. also:

pi@norns:~$ uname -r
4.9.47-rt37-v7+
simonvanderveldt commented 6 years ago

i checked out buildroot but haven't had time to dig in, so it'd be amazing if you wanted to take a look

Sure, it will probably make my life easier as well because it's meant for our usecase :) Have used it in the past, though that's already 2 years ago, so will need to do some reading to get up to speed.

I stumbled upon this earlier, which could be a starting point: https://github.com/gamaral/rpi-buildroot the author posted a video of an exceptionally fast boot into a game engine: https://www.youtube.com/watch?v=4Fjfqz6FxC8

Thanks for the pointers! I'll have a look how he configured everything to figure out what makes it so quick.

catfact commented 6 years ago

yeah update won't hurt anything

upgrade was incredibly stupid of me, was setting up build chains for this and that, totally missed the new kernel listing.

i might need help setting up cross-compilation for some of these things (e.g. supercollider plugins) if we end up with a buildroot system... that's how it would work right?

tehn commented 6 years ago

is CI still an issue? or do we just build the image locally and post a release? shall i close this issue?

simonvanderveldt commented 6 years ago

I'd very much like to be able to just build the image easily, both for ourselves as well as with CI But we'll have to look into speed. I don't think we'll be able to use Travis because of the 50mins limit, but we can look at other options.

simonvanderveldt commented 6 years ago

@tehn I think we can close this one for now since we're not going to be building it automatically at the moment anyway? Otherwise I can have another look at https://github.com/RPi-Distro/pi-gen to see if we can maybe just generate the raspbian based image in one go.