Update on the project?

[WaaromZoMoeilijk]

I'd like to know if there are any updates on the project. Or can I opt in on a discussion somewhere? As of today I managed to get a fast OC server running on the rpi with ubuntu core (not snappy, so apt-get manager). This all with a setup script just like the VM's from enoch85 at techandme.se. (write img to sd card, boot and automatically runs the script at login)

I'd like to share my project with you, though not really any activity here...

Regards,

Ezra

[jospoortvliet]

Hey @ezraholm50 let me ping @enoch85 @miska @kyrofa asking where they are at.

For the base image, see https://github.com/owncloud/pi-image/issues/17 - I think we should make a decision what platform to move forward with. Which is furthest ahead, fulfilling most requirements and comes with some ppl working on it.

It seems Snappy was most active but I can't judge that 100%.

The question then is if we can all converge on that: get the stuff @miska developed integrated in it, together with the scripts and tools @enoch85 has built, and create a single base image.

[WaaromZoMoeilijk]

@jospoortvliet Sounds like a plan. I'll write a summary on my version soon. The readme also states quite a lot. Just need some ideas on updating the build for users that have an older version. Also mine is based on the VM's enoch85 has made. Though he reviewed a small part it would need a full review by you guys since I'm in my beginner stage.

I look forward to hear from you guys!

Regards,

Ezra

Op do 25 feb. 2016 16:48 schreef Jos Poortvliet notifications@github.com:

Hey @ezraholm50 https://github.com/ezraholm50 let me ping @enoch85 https://github.com/enoch85 @miska https://github.com/miska @kyrofa https://github.com/kyrofa asking where they are at.

For the base image, see #17 https://github.com/owncloud/pi-image/issues/17 - I think we should make a decision what platform to move forward with. Which is furthest ahead, fulfilling most requirements and comes with some ppl working on it.

It seems Snappy was most active but I can't judge that 100%.

The question then is if we can all converge on that: get the stuff @miska https://github.com/miska developed integrated in it, together with the scripts and tools @enoch85 https://github.com/enoch85 has built, and create a single base image.

— Reply to this email directly or view it on GitHub https://github.com/owncloud/pi-image/issues/19#issuecomment-188848625.

Met vriendelijke groet,

E. Holm

[kyrofa]

@jospoortvliet ownCloud packaged for Snappy is working pretty well and checks a lot of the marks you've laid out, but the 15.04 image isn't a good long-term solution for ownCloud. 15.04 has the limitation of not being able to host data out of the writeable partition, which means each device would need a custom partition layout to use the external HD (workable but not ideal). Canonical is aware of that limitation and is working on a solution for 16.04, but the timing obviously isn't perfect for this device.

The upshot is that it does have people working on it-- it's one of the demos at MWC, and will soon be an official Canonical snap.

[kyrofa]

Quick update: It's now owncloud.canonical on rolling 16.04 (still owncloud-server.kyrofa on 15.04).

[jospoortvliet]

@kyrofa yeah, I read about that limitation. We could consider delaying the production of the first 500 until that date, or going live with this for the first batch - both could work. What do others think, like @oparoz and @enoch85 ?

[oparoz]

TL;DR

If I had to pick today, I would go with what we have, the VM, simply because that has existed for a while and has official support and people dedicated to the project, but the lack of automatic updates bothers me and I would probably switch to Snappy as soon as it's deemed stable. The other projects seem to have been worked on with a silo approach and from what's I've seen, don't meet the requirements, but there are components which could be extracted.

Long version

From my point of view, if we can define and agree on the requirements for a base OS (#17), we don't even need to delay the project. The OS is irrelevant and can be interchanged. What matters are the tools and GUI built on top of it and which deliver the experience to the user. Every image has to include those. Think about them as essential components which lead to the image being certified to run well on the platform and which prevents lock-ins.

That certification process would also be a good trial run for each project to help pick a "winner", if we really have to, because I don't think we're in a position to do that today. Also, all these projects are "fresh", with the exception of the ownCloud VM and the images adding the drive as part of a larger cluster (from memory), so there is little history related to most teams or the products behind them, thus the importance of being able to easily switch platform without losing data or time.

So, maybe in a few weeks, we'll end up with a couple of options to choose from and hopefully some OS agnostic tools which can be implemented on all images.

Regarding Snappy, I wouldn't automatically discard it if we can get some assurances on the state of 16.04. It depends on when WD needs to take delivery of the image, but shipping a RC wouldn't be a show-stopper. It's not uncommon to have to update equipment right after having bought it.

[jospoortvliet]

We had a chat with WD today about the state. There's some cool stuff coming: the Raspberry 3 which is 64bit (awesome!), Snappy with 16.04 and of course ownCloud 9.0 (and full PHP 7 support!).

WD proposed: why don't we do a small batch of about 50 devices, kind of like a test run, something we're good shipping a RC on. I like the idea of just shipping a RC, because these 50 would be for the adventurous types anyway - so I said, yeah, let's do that.

So, we agreed that I'd ask you lot to do that: create something which basically works but is, perhaps, experimental: a RC. PHP 7, 16.04 beta - that's fine.

The deadline would be soon - something like next week, if possible. Just to have something to ship, not perfect - something for people who want to try and help us make this happen.

What do you think, can we do this?

[kyrofa]

@jospoortvliet would the devices ship already flashed? The snap itself is ready for that, but we'll have to do a rather non-standard flash to get files on the HD instead of the SD (unless you want to include that limitation in the RC :smile: ).

[oparoz]

I think it mainly depends on whether the owncloud.canonical snap is deemed stable enough for every day use, apart from the odd bug, because if the Pi reboots randomly or if data gets lost, then users won't be happy about it.

@kyrofa - I thought 16.04 made it possible to boot from the shipped SD card while having other partitions on the HD.

[kyrofa]

@oparoz 15.04 had that capability as well-- it still takes work to actually set that partition up. 16.04 will have the ability to do that without any setup required, but that isn't complete yet.

[oparoz]

Thanks! :+1:

[jospoortvliet]

I checked with WD and we can preload both the SD and the HDD with an image we supply, so I'm guessing that means we can go with 16.04? I think there's benefit in going with the most 'advanced' tech, and the stuff we want to build the final product on - so ppl can help shake out the bugs rather than help rebase it on something else.

In other words, I would suggest to go for the full 16.04 Snappy, PHP 7, ownCloud 9.0 final (coming Tuesday anyway) and whatever else is new and fancy, and warn the First Fifty that what they get is a testing system.

That means they can help test and finalize our setup for when we go For Real - some time in April I'd say, based on the plans for 16.04 and so on ;-)

[enoch85]

@kyrofa Feels like we have some catching up to do. I'm idle in #snappy on IRC. Just ping me.

[kyrofa]

@jospoortvliet alright I'll await the ownCloud 9 release, update the snap, and post here when done.

@oparoz regarding stability (and the supported feature set), that's your call. It'd be great to get some people using it to work out any kinks. There's certainly still work to be done.

So to be explicit, if you want to ship an RC Ubuntu Core 16.04-based device, the limitations will be the following:

1. No un-versioned data support. Ubuntu Core has the ability to rollback updates, and in order to do that it keeps versioned data directories. The first step to updating to a new version of a snap is to automatically copy the old data into the new data directory, so if one needs to roll back the old data is still there. This works for a lot of use-cases but not ownCloud's, where for example my ownCloud's data is hundreds of GBs. This limitation will waste hard drive space and cause updates to be slow directly proportional to the space taken by ownCloud's data. There's no working around this.
2. No built-in non-OS hard drive support (external or otherwise). This means by default ownCloud will want to save all its data on the SD card where the OS is flashed, which is obviously not ideal for the same reasons (1) is a problem. Fortunately we can work around this by placing the writeable partition on the HD and removing it from the SD, but that still needs to happen (I do think we can come up with a golden OS image and a golden HD image, though).

I have been given assurances that both of these issues will be resolved in the 16.04 release of Snappy.

@enoch85 I'll ping you tomorrow :smiley: .

[oparoz]

It'd be great to get some people using it to work out any kinks. There's certainly still work to be done.

I agree. People with devices should try the oc9 snap as soon as it's available. That way, while working on scripts, etc. they'll hit real life issues and can report back.

I don't see No un-versioned data support as too much of an issue, if we can help users back up their data by providing at least a script per example. Ideally, there should be an app giving admins a few options as to where the backup should be shipped per example.

If I understand the problem with the hard drive correctly, you mean that 2 images need to be shipped together: one for the SD card and one for the hard drive so that at boot time, the OS can find the proper partition to use for data. If that's the case, then only the WD people can say if it's possible to do.

[kyrofa]

I don't see No un-versioned data support as too much of an issue, if we can help users back up their data by providing at least a script per example. Ideally, there should be an app giving admins a few options as to where the backup should be shipped per example.

Well the issue is that a backup is created every time the snap is updated, which isn't good long-term, that's all.

If I understand the problem with the hard drive correctly, you mean that 2 images need to be shipped together: one for the SD card and one for the hard drive so that at boot time, the OS can find the proper partition to use for data.

You do understand correctly. I'll try to investigate this further and distil the process as much as I can.

[oparoz]

Well the issue is that a backup is created every time the snap is updated, which isn't good long-term, that's all.

Yes, but the rollback feature is optional, no? It could be that during the RC phase the only way to recover from a bad install is to create a new SD card with the newest OS.

So for now, snaps can't access any mounted devices? Like it would be impossible for someone to mount another drive for backup?

[kyrofa]

Yes, but the rollback feature is optional, no?

No, the transactional update process is integral to the design of Snappy. And it's a good thing-- something the snap tries to take advantage of with the database etc. It's just that the data is currently copied along with it which sucks.

So for now, snaps can't access any mounted devices? Like it would be impossible for someone to mount another drive for backup?

Well... yes and no. Snaps can technically do anything with the right security profiles, but by default they're pretty locked down. The right answer is that no snap that is accepted to the store right now will be able to access a mounted HD (since such security settings probably wouldn't be approved). But nothing prevents that snap from being made and sideloaded.

Also, the discussion of snaps is completely external to the ability to SSH in and do whatever you like. If one is SSHd in, one can of course mount another HD and back data up manually.

[oparoz]

Thanks!

[jospoortvliet]

So we can ship both a SD card image and a hard drive image, so that would work even if it's slightly nasty.

For the first 20-piece-batch, I think we should go with that. Some issues are fine - then people who get a device have an incentive to help us out ;-)

oC 9 is out, btw, awesomeness, right? It's about time to start assembling a first set of images... I'll be in the office tomorrow to send the Pi I have there to @oparoz and I'm hoping @miska can work with @enoch85 and @kyrofa to get his code running on their images...

[enoch85]

cc @ezraholm50 --^

[kyrofa]

Bad news on the timing: my rpi2 just went belly-up. I'm working on a replacement, but I'll be delayed on arm snaps until then.

[jospoortvliet]

@kyrofa where are you located? I could perhaps send you a test kit ;-)

[kyrofa]

@jospoortvliet thanks for the offer! I did manage to get another rpi2, so while I wouldn't turn down a test kit to make sure we were all using the same stuff, I'm no longer blocked by this.

However, I seem to have hit an issue with upgrading to ownCloud 9: occ upgrade disables my calendar, and when I login and re-enable it my events are gone. Has anyone else experienced this? Seems to work fine for everything else, files, users, contacts, etc., but it apparently wipes the calendar. I'm still investigating. Most recent work to upgrade to OC9 is here, with a snap uploaded here if anyone wants to play around with that. I won't upload to the store until I figure this out.

[oparoz]

@kyrofa - If your calendar data is still in the DB, can you try to switch your ownCloud branch to "daily" and then do another upgrade to see if your issues have been solved on stable9? I think the initial image should be based off stable9 anyway as there were too many issues with the new apps and .htaccess on 9.0.

[jospoortvliet]

http://people.canonical.com/~kyrofa/owncloud-pi/ first test image is out! see also this email for info. Scripts are now in the repo. Time to test!

[WaaromZoMoeilijk]

Sorry guys, was away for work last week. So snappy it is? Nice! @jospoortvliet is there any chance to get a test kit? Don't have any spare HD's around and the wife's website runs on the only PI left here.

I'll let enoch85 fill me in, or if someone else could tell me what i missed, please. Already read up on some of the posts but not all.