search

hypriot / os-rootfs

The root filesystem of HypriotOS
http://blog.hypriot.com/post/how-to-get-docker-working-on-your-favourite-arm-board-with-hypriotos/
MIT License
69 stars 40 forks source link

Transactional AB Updates of the core OS #41

Open joeblew99 opened 8 years ago

joeblew99 commented 8 years ago

I am trying to get Transactional AB updates working with hypriot.

Transactional AB Updates is what CoreOS and Ubuntu Core also offers. ChromeOS pioneered this as far as i know. The use case for this is so that you can update Hypriot OS (the host) itself in a transactional way, such that if the update fails you can rollback everything.

For me this is a huge feature that would make this very useful for my project, my clients project, and also the general public to be able to run Owncloud and other things. It advances everyone.

One aspect of this is the disk partitioning. Hence why i am raising this in this repo.

Another aspect is modifying the boot to tell it to boot up which partition, based on the state of the machine. This might be a bit tough based on the many boards out there.

I spent a few weeks lookig into how Ubuntu COre does it and its not that complex. Ubuntu have written a ton of golang code to wrap lots of this, and so its kind of easy to follow whats going on.

So, i am raising this idea here to see if anyone else wants to work on this with me, if the core developers are thinkng about this, or dont see any value in this... Would be nice if my patches are accepted...

Govinda-Fichtner commented 8 years ago

@joeblew99 This sounds awesome. We basically did not this see as one of our near term goals - guess we also thought this might be quite complex. But if you say after some investigation that this is not to complex it might be worth to work on this together.

I do have some free time next week which I could use here. If you had an initial PR and some pointers where to look to get started with this I would have a first look.

What do you think?

MathiasRenner commented 8 years ago

Updating the core of HypriotOS is an frequently requested feature by the community. This feature beeing stable puts HypriotOS on a new level!

joeblew99 commented 8 years ago

I am also having some free time.

I am really happy that this is something we want. It really helps to hear that.

I don't have any PR because I was working on this 6 months ago for an iot startup and its their code. I was focussing on slip streaming some changes to Ubuntu code for their needs. Long story but I learnt a bit about it then.

The place to look is the Ubuntu core code. I would look at the raspberry pi code there. From that it's easier to form an understanding of it. It's basically all about initrd. Its also written in golang which is awesome. https://github.com/snapcore/snapd/blob/master/boot/kernel_os.go

The hardest part is understanding it. Also different boards need slightly different code.

The other trick of this is the pxe boot. I am bringing this up because I feel it's 100% out of scope for Hypriot as this stage and much more work / tricky. But here is how it works. Coreos uses pixie, a nice golang wrapper that acts as a server to allow the raspberry pi to boot / load the OS over the network. The raspberry pi hits the router which then tells the raspberry pi to use a different dhcp server. The dhcp server is the pixie server. Then a few things happen etcet. The point however is that Hypriot does not need booting over the network. It's out of scope and there be dragons and boiling the ocean there.

I am in Berlin / Potsdam in case anyone else is around to hack on this together :)

Joe

On Thu, 16 Jun 2016, 14:36 Govinda Fichtner, notifications@github.com wrote:

@joeblew99 https://github.com/joeblew99 This sounds awesome. We basically did not this see as one of our near term goals - guess we also thought this might be quite complex. But if you say after some investigation that this is not to complex it might be worth to work on this together.

I do have some free time next week which I could use here. If you had an initial PR and some pointers where to look to get started with this I would have a first look.

What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hypriot/os-rootfs/issues/41#issuecomment-226472272, or mute the thread https://github.com/notifications/unsubscribe/ALcac47TOJmUq3BhHHTlwIFGfGCmypa0ks5qMUNFgaJpZM4I3SlT .

joeblew99 commented 8 years ago

Btw this is news to me.

http://arstechnica.com/information-technology/2016/06/goodbye-apt-and-yum-ubuntus-snap-apps-are-coming-to-distros-everywhere/

So add Hypriot to the list. This is great news.

On Thu, 16 Jun 2016, 15:36 Joe Blue, joeblew99@gmail.com wrote:

I am also having some free time.

I am really happy that this is something we want. It really helps to hear that.

I don't have any PR because I was working on this 6 months ago for an iot startup and its their code. I was focussing on slip streaming some changes to Ubuntu code for their needs. Long story but I learnt a bit about it then.

The place to look is the Ubuntu core code. I would look at the raspberry pi code there. From that it's easier to form an understanding of it. It's basically all about initrd. Its also written in golang which is awesome. https://github.com/snapcore/snapd/blob/master/boot/kernel_os.go

The hardest part is understanding it. Also different boards need slightly different code.

The other trick of this is the pxe boot. I am bringing this up because I feel it's 100% out of scope for Hypriot as this stage and much more work / tricky. But here is how it works. Coreos uses pixie, a nice golang wrapper that acts as a server to allow the raspberry pi to boot / load the OS over the network. The raspberry pi hits the router which then tells the raspberry pi to use a different dhcp server. The dhcp server is the pixie server. Then a few things happen etcet. The point however is that Hypriot does not need booting over the network. It's out of scope and there be dragons and boiling the ocean there.

I am in Berlin / Potsdam in case anyone else is around to hack on this together :)

Joe

On Thu, 16 Jun 2016, 14:36 Govinda Fichtner, notifications@github.com wrote:

@joeblew99 https://github.com/joeblew99 This sounds awesome. We basically did not this see as one of our near term goals - guess we also thought this might be quite complex. But if you say after some investigation that this is not to complex it might be worth to work on this together.

I do have some free time next week which I could use here. If you had an initial PR and some pointers where to look to get started with this I would have a first look.

What do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hypriot/os-rootfs/issues/41#issuecomment-226472272, or mute the thread https://github.com/notifications/unsubscribe/ALcac47TOJmUq3BhHHTlwIFGfGCmypa0ks5qMUNFgaJpZM4I3SlT .

DieterReuter commented 8 years ago

@joeblew99 Great to hear this requirement, we'd love to support it!

AB booting would be a great feature to have and you're right, it heavily depends on the boards you're using.

PXE boot with CoreOS is damn easy to use, and it also is able to boot the complete systems via the internet, I've already run this as an experiment in the past with CoreOS! Just loading CoreOS over HTTP while booting your PC, an OS always up-to-date.

BTW, the Hypriot team is located in Bamberg, just a 4 hours drive from Berlin.

joeblew99 commented 8 years ago

I traveling down near there in a few weeks for some energy tech project review. Nothing to do with IT Can you shoot me your contacts via email. Would be awesome to meetup.

Cheers

Joe

On Thu, 16 Jun 2016, 15:47 Dieter Reuter, notifications@github.com wrote:

@joeblew99 https://github.com/joeblew99 Great to hear this requirement, we'd love to support it!

AB booting would be a great feature to have and you're right, it heavily depends on the boards you're using.

PXE boot with CoreOS is damn easy to use, and it also is able to boot the complete systems via the internet, I've already run this as an experiment in the past with CoreOS! Just loading CoreOS over HTTP while booting your PC, an OS always up-to-date.

BTW, the Hypriot team is located in Bamberg, just a 4 hours drive from Berlin.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/hypriot/os-rootfs/issues/41#issuecomment-226489694, or mute the thread https://github.com/notifications/unsubscribe/ALcacxkG-dSe8VRFoIlh7lgAmbvcL6leks5qMVPcgaJpZM4I3SlT .

DieterReuter commented 8 years ago

@joeblew99 you can find our email addresses in our GH profiles, I can't find yours.

joeblew99 commented 8 years ago

I dont have it public. Got sick of spam. I just sent you an email and my mobile :)

joeblew99 commented 8 years ago

I love the way coreos does network boot too. Its amazing. The reason i suggested it to be out of scope for now is this:

  1. Hypriot is for ARM boards and allot of these are not in Data Centers (yet), and so its normal that you can get the base OS onto the machine yourself. This of course is likely to change, for example i was deploying to Armv8 boards in a French Data Center.
  2. Its a matter of what to do first without boiling the ocean
  3. what most useful for now ? For me, its that once the server is setup by me, i can set and forget and when new versions of Hypriot are released it just updates.

If others have thoughts on this love to be wrong. Just thinking of whats useful and not to boil the ocean too much

ghost commented 6 years ago

Any updates about this ?

If gokrazy an option ? https://github.com/gokrazy/gokrazy

It seems to have the ability to do updates. https://github.com/gokrazy/autoupdate