Norns OS requirements

[simonvanderveldt]

Not really an issue, but I'd like to make sure we have a solid understanding of our requirements for norns's OS before we decide how to build it.

From the discussions so far I've distilled the following end-user targeted requirements:

• Should boot quickly. More specifically, should show signs of life to the user max 6s after the device is started by holding down button 1 for 3s
• The user should be able to apply updates without opening the device
• Applying updates should be safe/not brick the device
• Updates should not wipe the user's work
• Minimum of 2GB storage available to the user
• The configuration of the kernel should enable all norns hardware to work out of the box
• Some additional drivers should be enabled as kernel modules to allow them to be used with norns out of the box. For example MIDI USB devices. Which ones t.b.d.
• Kernel/console output should be available on mini-usb on the back
• Mobile/battery powered usage should be possible, currently we have no minimum amount of hours battery life defined

And the following development related requirements:

• Be able to have a quick develop, build, test feedback cycle.
• Be able to develop using the developers desired development tools

[simonvanderveldt]

The above is just an initial draft, please provide feedback/changes/additions where you can :)

Questions regarding the end-user requirements:

• Do we want user-made changes to the system to be persisted or should it always start with the system as built by us?
• What do we want the update experience to be for the user?
  • Should updates be automatic? I.e. do we always want everyone to be on the latest version?
• How much storage do we want to have available for the user as a minimum?
• How resilient to instant poweroff (i.e. pulling the power) does it need to be?
• Which kernel modules do we want to include? For example bluetooth, usb wlan, usb audio, etc
• Do we have any requirements regarding power consumption? Is the battery meant for mobile usage or just as a form of UPS?

Questions regarding the development requirements:

• How quick do we want the develop, build, test cycle to be? Would doing rebuilds (with caching so only the changes would be compiled) of the image and then dd-ing be quick enough?
• Would it be feasible to do development of the norns app on regular computer instead of on the device?

[tehn]

in order, skipping the ones that don't need clarification:

• ease of updates. most important is that the unit should not need to be opened up, ie using rpiboot to flash whole images. we reserve this action for recovery from catastrophe, for normal users. so "easy" means any way that doesn't require opening the unit.
• console out the rear mini-USB, this is essential for debugging, I need it as a backup for doing support in case something is really busted. it's enabled by default, let's make sure it stays

end-user:

• user-made changes. do you mean user changes to the base set of software? i feel that may be difficult to preserve-- any conflicts should be reverted to a working state. i'd almost opt for overwriting the "system" portion entirely. settings, user scripts, user sound, etc all needs to be absolutely preserved. but i'm trying to imagine what sort of changes a power-user would make exactly. like adding pd just for fun?
• i don't think updates should be automatic. they should be notified of availability, but it should require an action to go forward. nothing worse than an untimely update (particularly for a performance-centric object)
• user space minimum: 2GB would be nice, more would be great, but we should err on the conservative side of the system needs in the future. user space is not meant to be an archive: it's a working set. people can insert a usb thumbdrive and do some housekeeping when needed, or transfer file via maiden (web)
• pulling power. that's why we have a battery! (that was a fundamental reason, actually.) i'm not sure how much we can do? we can monitor (via sysfs) if the power plug is in before running an update. my intention was to auto-shutdown gracefully if the battery gets below 5% (or something)
• the battery is meant for some mobile usage, but with the understanding that battery life varies wildly according to peripheral connection and cpu use. so very much a UPS, and some mobility. not having it turn off when switching USB cables is pretty great.

development

• i'm ok with needing to dd a full image if there are substantial changes, it's not so much trouble. thinking we'll get past this point quickly enough ;)
• it is already possible to develop the norns app on a desktop, though you don't get the screen/knobs/keys. we could of course add an emulation layer for encoders/keys, and a framebuffer redirect to a virtual window was mentioned

[simonvanderveldt]

@tehn Thanks for the answers. A couple of quick follow ups before I go to bed, will update the issue description based on these tomorrow.

user-made changes. do you mean user changes to the base set of software? i feel that may be difficult to preserve-- any conflicts should be reverted to a working state. i'd almost opt for overwriting the "system" portion entirely. settings, user scripts, user sound, etc all needs to be absolutely preserved. but i'm trying to imagine what sort of changes a power-user would make exactly. like adding pd just for fun?

Yeah, I meant changes a user does to anything on the filesystem that doesn't live in /home. Mainly asking because it would probably increase differences between systems which would mean more difficulty troubleshooting when issues arise, but on the other hand not allowing user changes to the root filesystem might limit the user somewhat. I'm all for overwriting the system partition to ensure a consistent experience.

i don't think updates should be automatic. they should be notified of availability, but it should require an action to go forward. nothing worse than an untimely update (particularly for a performance-centric object)

Good to know. My initial description was relatively broad, but I would like to add that automatic doesn't mean it happens at a random moment :) I was mainly asking to get to know if you'd prefer to try to get everyone to use the same/latest version or not. The update could also trigger on shutdown or startup (with a prompt) when an update is available. So the feedback and process to update is automatic but the user is still in control. This as an alternative for the user themselves needing to check for updates.

pulling power. that's why we have a battery! (that was a fundamental reason, actually.) i'm not sure how much we can do? we can monitor (via sysfs) if the power plug is in before running an update. my intention was to auto-shutdown gracefully if the battery gets below 5% (or something)

I expected the battery would be there for this case, but wanted to check the reason for it to be sure. We can definitely check battery % before doing an update and ensure a minimum, shouldn't be that hard.

Auto-shutdown below a certain % might be useful as well, though if we choose the have the system not be edittable by the user we might be able to run it read-only which means it wouldn't really matter if the power just cut out.

[ranch-verdin]

i'm trying to imagine what sort of changes a power-user would make exactly. like adding pd just for fun?

I'm thinking specifically that users may want/need to hack on matron C code & ugens. However, cross-compiling toolchain with a make deploy command might be better in some ways than developing directly on the device.

So I've warmed up to the idea of no gcc on the norns, as long as all parts of the norns software can be modified by users with a fast build/deploy/test cycle suitable for serious development work.

[tehn]

Auto-shutdown below a certain % might be useful as well, though if we choose the have the system not be editable by the user we might be able to run it read-only which means it wouldn't really matter if the power just cut out.

whoa i didn't think of read-only for the non-user part, that's slick

agreed re: assessment of "automatic" updates

@rv honestly the development cycle for matron is great on-device. i can cycle through testing so rapidly with this method. i can't imagine a deploy method being quite as graceful, but if it could be imagined, i'm up for it. basically we need a sane path to support development that is not a hindrance. on-device is as natural as using just another computer, that's already set up to do basically everything you want it to do.

that said i'm still pretty ok with updates plowing any modifications. provided people can keep their own forks, they can just re check-out after an update. ie, git perms could even live on the user part

[simonvanderveldt]

I'm thinking specifically that users may want/need to hack on matron C code & ugens. However, cross-compiling toolchain with a make deploy command might be better in some ways than developing directly on the device.

Good point! This is quiet an interesting aspect.

@tehn Would developing/creating extensions to SuperCollider (I don't know SC, but I believe there are ugens and quarks from a quick search) be something that you want the users to be able to do easily?

[ranch-verdin]

There are some obvious potential timepits here:

1. Learning Yocto and using that
2. Tacking a working on-device dev environment onto buildroot
3. Getting bootsplash image into kernel space (we have proof of concept now that buildroot can achieve fast splash in userspace, whereas debian requires we dig back into initramfs/plymouth)
4. automating debian image generation or labour-intensive manual procedure

I only suggest dropping on-device development as a way to sidestep these potential quagmires! But maybe the deploy method is always going to suck compared to on-device. I had a lot of joy with a cross-compile/make deploy approach on the aleph. This approach means users could choose, e.g, atom on their laptop if they're not down with the vim/emacs hacker orthodoxy.

On balance, I'm going to stick my neck out & say we should risk the 'learning Yocto' timepit (but be prepared to back out in a hurry and rethink). I could be very easily swayed on this though - very tough question!

[tehn]

Would developing/creating extensions to SuperCollider (I don't know SC, but I believe there are ugens and quarks from a quick search) be something that you want the users to be able to do easily?

given that ugens are the path for custom DSP it'd be good to support this if possible, though if i recall @catfact suggested there's a bit of toolchain setup that would is pretty specific and possibly heavy (cmake etc). so perhaps this part makes more sense for the make deploy idea.

custom sclang is of course trivial-- that will live in the user part, and is a basic sort of "custom dsp" (a huge benefit to using sc)-- this doesn't require make deploy at all but is fundamentally part of the web editor/etc, so maybe that's enough to satisfy immediate custom sound.

so yes, i'm warming to @ranch-verdin 's proposal to drop on-device development. particularly if we have a well-documented method for on-linux-laptop development (a good emulation layer would be a bonus).

but i'm feeling very tenuous about diving into yocto given the needed time investment

[simonvanderveldt]

given that ugens are the path for custom DSP it'd be good to support this if possible, though if i recall @catfact suggested there's a bit of toolchain setup that would is pretty specific and possibly heavy (cmake etc). so perhaps this part makes more sense for the make deploy idea.

custom sclang is of course trivial-- that will live in the user part, and is a basic sort of "custom dsp" (a huge benefit to using sc)-- this doesn't require make deploy at all but is fundamentally part of the web editor/etc, so maybe that's enough to satisfy immediate custom sound.

It sounds like for starters it would make sense to enable easy use of sclang in the web editor and at a later point look at custom ugens?

so yes, i'm warming to @ranch-verdin 's proposal to drop on-device development. particularly if we have a well-documented method for on-linux-laptop development (a good emulation layer would be a bonus).

but i'm feeling very tenuous about diving into yocto given the needed time investment

Agreed with both these points. I'll look into the possibility of developing on another machine including using networking over USB this week.

The buildroot stuff is working out pretty wel so far, the basics are all there and extending it it pretty easy. I'd like to get it to a minimum working version that includes running norns. Then we can all see what the config looks like and try the build a couple of times.

In the meantime development of the norns app can go the same way everyone has been working so far I think, nothing really limits us in that regard, right?

@artfwo I believe you mentioned some development workflow related things on slack as well, do you think these are covered well enough by the discussion so far?

[catfact]

to be honest, i think custom ugens are really niche. i'd predict that almost no one will want to make their own, and if they do it shouldn't require a lot of dev/test cycle on the device.

connecting the web app to sclang and the norns SC extensions directory would be super awesome - that is a very accessible way for users to add their own processing.

as far as the linux config goes, one way to expand more the DSP palette by quite a lot would be to include a build of sc3-plugins https://github.com/supercollider/sc3-plugins. that brings in the LADSPA host ugen, among other things.

[simonvanderveldt]

as far as the linux config goes, one way to expand more the DSP palette by quite a lot would be to include a build of sc3-plugins https://github.com/supercollider/sc3-plugins. that brings in the LADSPA host ugen, among other things.

The version number of sc3-plugins is the same as supercollider. Do you know if they are released/tested in tandem?

[artfwo]

The version number of sc3-plugins is the same as supercollider. Do you know if they are released/tested in tandem?

@simonvanderveldt not necessarily, but the latest plugins package generally works with latest supercollider.

@artfwo I believe you mentioned some development workflow related things on slack as well, do you think these are covered well enough by the discussion so far?

yeah, i think we've got most issues covered.

[tehn]

we have done it!