search

nextcloud / nextcloudpi

📦 Build code for NextcloudPi: Raspberry Pi, Odroid, Rock64, curl installer...
https://nextcloudpi.com
2.49k stars 295 forks source link

Support ODROID-HC1 #160

Closed renne closed 6 years ago

renne commented 7 years ago

The ODROID-HC1 (Home Cloud One) is optimized for NAS-functionality and is much more powerful than a RaspberryPi.

Please support an Nextcloud image for the HC1.

miodzicho commented 7 years ago

I'm also interested in Nextcloud image on HC-1

nachoparker commented 7 years ago

There has been some discussion on wether it would be more interesting to do a full port of NextCloudPi or just use the NextCloudPi ARM container

If anybody is interesting in helping with this, we could start by trying the docker image and see how it goes.

Since NextCloudPi moved recently to Raspbian 9, the docker image still has not been updated, so it is a bit older and there is no web access to the NCP extras. Still NC12 though.

edit: I have ordered an HC1 myself, but I don't expect to have much time to support two boards, specially if we go for the 'port everything' instead of 'docker image' solution.

miodzicho commented 7 years ago

Sounds good to me. I'm going to try docker image next week, will report how it works.

nachoparker commented 7 years ago

great

testbird commented 6 years ago

Maybe your installer script also works, or can be made to work, in other non-raspi centered debian based embedded distributions like armbian, or the debian based distros from the vendors.

testbird commented 6 years ago

Make it a debian meta package?

nachoparker commented 6 years ago

Maybe your installer script also works, or can be made to work, in other non-raspi centered debian based embedded distributions like armbian, or the debian based distros from the vendors.

Yes, we will probably take that path.

Conversations started here about using armbian and porting to other boards.

My odroid HC1 just arrived, and I am updating the docker container to test it on the odroid. Depending on the resources I have I will probably end up porting to Armbian, but supporting and maintaining NextCloudPi already takes a lot of my time.

Also there is the issue that

testbird commented 6 years ago

Adjusting the ncp scripts to install on openmediavault might also become a strong option now with its new hardware support, at the same time joining with a larger community. (Even making it installable in plain debian and its derivates [1])

See https://forum.openmediavault.org/index.php/Thread/19618-OMV-3-for-ODROID-XU4-HC1-HC2-MC1/ https://forum.armbian.com/index.php?/topic/3953-preview-generate-omv-images-for-sbc-with-armbian/

[1] https://wiki.openmediavault.org/index.php?title=Install_OMV_on_Debian_via_apt

nachoparker commented 6 years ago

More brainstorming here! awesome

Mmmm I have no idea of how OMV works. Is it based on debian? I would need to take a close look.

It would maybe be interesting to cooperate with them but, how would that go? would they be interested?

I would also need to have a look at the .deb for OMV. Does it install the whole LAMP stack also? maybe as .deb dependencies, or contained inside?

I was thinking to take the docker container approach to magically have it working on all ARM boards. Thoughts?

Pant commented 6 years ago

+1 for the docker container approach. It even opens up the way for docker swarm experiments!

testbird commented 6 years ago

I think many people are looking for a build-system that is able to also produce iso images (amd,intel,...) as well as for multiple embedded devices and docker.

Openmediavault is a debian based distro, you'd first need to try it out (install, configure) to see if you like what and how they made it. There was once an owncloud plugin, but it was discontinued and an installation manual was posted. When you try it out and have to ask a question, tell them you created nextcloudpi and may be interested in a base system that is more universal than raspbian.

You'd need to know if they would welcome your contributions, say in apache https configurations and you get along well.

OMV has a gui plugin to manage docker images, maybe try to setup your nextcloudpi docker image ontop of the OMV image to get your HC1 running.

Docker is a good option for admins these days. There are also OMV docker images out ("mostly for plugin development". Your strength is knowing the configurations for a ready-to-use, well preconfigured nextcloud image. If you'd like to share the workload of maintaining such a system, your scripts may more easily evolve into an omv script or plugin (collaborating on the underlying server configuration) then a debian package. (Nevertheless, the OMV community seems to work towards making their server configuration installable in plain debian.).

For updates I prefer distribution package managers updating individual files (possibly within a container, used only to run the system on arbitrary, otherwise unsupported platforms). This avoids redundancy and the burden on container maintainers to provide security updates with large container updates (possibly missing from some not so well maintaining container provider).

nachoparker commented 6 years ago

Thanks for the info, I'll definitely have a look

For updates, I think you can provide a light update if you optimize docker layers. I am still thinking the whole thing, but I would like to reach the point where you can update even across NC versions, like the snap supposedly does. That would be the final goal, but I am still undecided on how to do it exactly.

testbird commented 6 years ago

Because I had overlooked it for a long time: The important things on https://www.openmediavault.org/ can only be found in the footer...! (very unfortunate)

miodzicho commented 6 years ago

Hi All. I have received HC1, I tried to install it without container. But unofrtunatelly facing weird HDD errors, trying to narrow if this is nextcloud related, or hardware. If interested, here is story : https://forum.odroid.com/viewtopic.php?f=97&t=28372&p=203017#p203017

I think will try container now, to see if this issue will happen again.

nachoparker commented 6 years ago

Ok, thanks for the info. Hopefully I will start playing with the HC1 soon. The container is a bit old ( Nextcloud 12.0.0 ), but it should work fine. Follow the instructions in ownyourbits to move your docker storage to the HDD and keep in mind that files over 2GB will be truncated because of PHP stupid 32bit limitation.

Let us know how it works!

testbird commented 6 years ago

Update on the base system comparison: Recent projects seem much more geared towards public self-hosting than openmediavault:

I found https://freedombox.org has become production ready, and had not seen https://yunohost.org before.

Freedombox looks as if it has become the debian way for self-hosting services now.

Yunohost

nachoparker commented 6 years ago

Hi,

didn't know about those projects, thanks.

freedombox looks really interesting. I will probably try to have a look at it. The approach seems to be to install deb packages.

We have also been looking into armbian to provide 64 bit images for RPi3 and other boards.

While the deb approach is great for regular software, I do not think is ideal for web services. I think that the docker/container approach is more appropriate because you can also manage and load balace the resources, and pack interacting services together in a nicer way ( dnsmasq + apache + mysql combo for example ).

You can as well stop and start the containers easily. You have installation + load balancing + management covered, it's pretty neat.

Maybe there is a nice way to start from a freedombox image and put the docker daemon on top.

edit: the management interface is also something very cool that could be reused ;)

nachoparker commented 6 years ago

In that regard, we could also add

linuxserver.io

picluster,

even make use of

portainer

nachoparker commented 6 years ago

also this, but seems a bit incomplete

https://resin.io/blog/announcing-balena-a-moby-based-container-engine-for-iot/

testbird commented 6 years ago

While the deb approach is great for regular software, I do not think is ideal for web services. I think that the docker/container approach is more appropriate because you can also manage and load balace the resources, and pack interacting services together in a nicer way ( dnsmasq + apache + mysql combo for example ).

I am not sure if the complexity is really necessary, and more importantly, if the docker image updating is really better. Also don't underestimate the amount of (time critical) workload to keep a docker image current and secure (build, test, signing infrastructure). You would need to compete with the debian security team or always lag behind.

The update strategy you do now with docker image actually looks like another of those intelligent things I've seen you do in ncp. The apt system for libs and things within the docker container where possible, plus ncp scripts where needet to check for and trigger the ncp and official nextcloud updates (php website code).

If you where to build a new image for every update this automated system installs you may be unnecessarily creating a centralised system (like those ubun* things that are becoming increasingly unpopular). Thus, there may be no need to introduce the complexity any of those elaborated docker management systems you mentioned.

Running the public nextcloud in a container has also security benefits, and splitting the ncp docker image into a system and a data container may allow straight forward snapshoting and backup (above tar?). So a simplistic container or app jail based deployment makes sense (thus you created one). Nevertheless, with small conditional adaptions (not overwriting entire config files) the ncp scripts will probably be able work on the freedombox, too.

testbird commented 6 years ago

A downside of docker image updating: They tend to overwrite user customizations, making local fixes or tests unreliable (e.g. my pwauth customization would be reverted as long it was not yet mergered into the image updates).

testbird commented 6 years ago

Q: Unfortunately I don't have a HC1 yet, anybody experimented / forked ncp to install on armbian yet?

I think I would try starting with the armbian image and dist-upgrade to stretch. Then I would

testbird commented 6 years ago

A yunohost downside: The web service package for nextcloud does not trigger official nextcloud updates as they become available, but needs to be updated for every tiny version step. https://github.com/YunoHost-Apps/nextcloud_ynh/commit/8f98d512b17c12b3f6225414c8dd5546ea482307

testbird commented 6 years ago

Stepping back for a second. We want nextcloud (or web apps in general) to reasily run on own hardware boxes.

Ncp delivers that, but we need more hardware platforms than raspbian. => Conclusion: Make it work in Debian and thus in armbian images.

Ncp is based on plain shell scripts that set up and manage what is needed to run the web app. This allows that users familiar with manual administration can contribute. Using some hype container orchestration would ruin or deviate that.

Ncp consist of cli-tools and web-frontends to configure distro/system (IP, DNS, updates,...), packages (db, webserver,...) and the web app itself (backups, updates,...). It is similar to cpanel and others plus consistent shell tools, presenting things to do for running the web app. The OS, packages, and web app tools might be able to work on all debian systems. A docker build allows for other systems and/or containment, but not more in terms of running nextcloud, or any other web app.

There is no web application manager in debian, ncp currently is a nextcloud application manager for the pi (raspbian at this time), but apparently we are now reaching out for hardware beyond the pi.

Iolaum commented 6 years ago

As a user of Nextcloudpi on a rpi2b nextcloudbox I 'll jump in for some feedback on this - hope you don't mind.

I appreciate staying as close to the base system as possible. This means that most of the problems a user will be encountering could be solved through google $error > copy > paste which will hopefully also mean less support burden for developers here.

I 'm not qualified on comparing the debian meta-package approach versus the docker approach. I 'd like to highlight some security factors into that. Those are the two most important I can think of:

P.S. I can see why the web administration feature gains so much traction (familiar and streamlined user experience for non technical people) but hopefully you will not neglect the the nextcloudpi-config option over an ssh connection (feature parity wise, looks don't matter in this case imo).

nachoparker commented 6 years ago

@testbird

Hi,

Sorry, I have been away for a few weeks.

There are good points there. The docker image is an interesting experiment, with some very interesting use cases.

keep a docker image current and secure (build, test, signing infrastructure). You would need to compete with the debian security team or always lag behind.

This would be automated, it is not hard.

If you where to build a new image for every update this automated system installs you may be unnecessarily creating a centralised system

Docker does not need registration. You can save and sideload images

elaborated docker management systems

That's something for a different project. The docker image for NCP doesn't need that, but I was thinking about creating a SD card image with one of those and docker already running with the NCP image.

A downside of docker image updating: They tend to overwrite user customizations, making local fixes or tests unreliable (e.g. my pwauth customization would be reverted as long it was not yet mergered into the image updates).

For sure, if you want to do customizations this solution would not work for you. There are people though that want to install NCP along other things, like pihole, and with containers this can be achieved more easily.

Also it is easy to 'try NC out' because you only have to docker pull which is really easy ;)

anybody experimented / forked ncp to install on armbian yet?

We are planning to move everything to armbian in order to support ARM64, and other boards. Regarding freedombox management interface... looks really cool. I would have to try it out.

Still we have the problem of different web based apps reconfiguring the database and web server. Containers solve that.

Ncp is based on plain shell scripts that set up and manage what is needed to run the web app. This allows that users familiar with manual administration can contribute. Using some hype container orchestration would ruin or deviate that.

The SD card version will always be there. It is more powerful, as it can, for instance, configure wifi, DHCP or other things that don't belong to a containerized service.

Ncp consist of cli-tools and web-frontends to configure distro/system (IP, DNS, updates,...), packages (db, webserver,...) and the web app itself (backups, updates,...). It is similar to cpanel and others plus consistent shell tools, presenting things to do for running the web app. The OS, packages, and web app tools might be able to work on all debian systems. A docker build allows for other systems and/or containment, but not more in terms of running nextcloud, or any other web app.

I am trying to achieve the best of both worlds... easy sideload of NCP with docker, but it is still based on debian, so the same code builds also the SD card version, people can contribute to the SDcard, and the container would benefit.

Shouldn't be hard to make it work on non Raspbian Debian systems... basically the big difference is the existence of a sudoers allpowerful pi user... I think packages should be almost identical.

The problem we have is that this is really slow because of lack of manpower. I have the boards at home, and won't have time to try them out for weeks because I am working on the wizard right now.

I don't think I can afford to support more boards by myself. We would need an armbian maintainer for this to happen. Probably not too much work, but my hands are full. Help is welcome.

nachoparker commented 6 years ago

@Iolaum

Thanks for your opinions. We need many of those :)

NextCloudPi already automatically gets the latest packages for Apache and the rest. NC docker would have to do something similar, of course. Security and easy upgrading is the most important thing here. If everything was based on docker, the migration to Stretch would have been automatic and painless for the user (that had to do dist-upgrade, and there are still some that remain in Jessie)

Hardening measures. They are important on anything connected to the internet. Nextcloudpi already has some measures like fail2ban. Hopefully it will be easy to port them and use them on a meta package or container implementation.

I aggree. I don't see the meta-package approach feasible for that same reason. The strong point of NCP in my opinion is that it comes with a collection of preconfigured utilities that work together. That can be achieved with docker, but not (I think) with .deb's.

(Nobody mentions modsecurity... is the single most valuable security layer that NCP has).

I can see why the web administration feature gains so much traction (familiar and streamlined user experience for non technical people) but hopefully you will not neglect the the nextcloudpi-config option over an ssh connection (feature parity wise, looks don't matter in this case imo).

No worries for that. The backend scripts that run all the extras are the same

nachoparker commented 6 years ago

@testbird

Looks like somebody already implemented a similar idea, using docker for autoupdates. When I have time I will investigate their solution

https://home-assistant.io/hassio/

testbird commented 6 years ago

Hey, quick update: it was a piece of cake to get the OS on the HC1 up and running:

Further on, it looks like the nextcloudpi script to setup nextcloud may be made conditional (to skip the install and setup of already configured debian packages) with which plinth (i.e. looking for the freedombox web interface).

testbird commented 6 years ago

measures like fail2ban. Hopefully it will be easy to port them and use them on a meta package or container

I see freedombox already installed fail2ban. Also syncthing, gnudip, and other nice things like matrix/xmpp server are already available.

nachoparker commented 6 years ago

@testbird thanks for the info! I definitely want to try freedombox, even though I still feel like the deb approach is less powerful to manage complex web systems comprised of several services in coordination.

I am working on building images for the odroid HC1 and the pine64 boards that I own with NCP in them.

In any case, the catch all solution to support other systems right now is the script

# curl -sSL https://raw.githubusercontent.com/nextcloud/nextcloudpi/master/install.sh | bash

but it needs to be on Debian Stretch. For the rest of systems I still think that the docker approach is the most universal. The new docker containers will also come out this week, but it will take some time until they do everything that the SD version does.

Opinions welcome

nachoparker commented 6 years ago

Well, aside from the curl installer and the docker option, I just created the first version of the SD card for the odroid HC1 / XU4. I consider it testing at this point, but it can already be used.

https://ownyourbits.com/downloads/testing/

Feedback is welcome.

edit:

Let's encrypt didn't build ok this first attempt. We can fix it with

sudo rm /usr/local/etc/nextcloudpi-config.d/letsencrypt.sh
sudo  ncp-update
testbird commented 6 years ago

Great news! (Too bad I can't test at the moment.)

As debian policy requires not to overwrite the admin's config files, this means it should also be possible install freedombox on the image and then even upgrade to debian/testing.

testbird commented 6 years ago

Oh, for those that can test: The armbian-config tool is said to allow installing to sata (HDD/SSD)

testbird commented 6 years ago

Ok, ncp-web came up fine.

Iolaum commented 6 years ago

In the raspbian nextcloudpi image the ssh is not enabled through an option but through adding an ssh file at the boot partition. Maybe armbian requires somthing similar?

testbird commented 6 years ago

I was not successful with that method either. Default Armbian has ssh enabled and requires changing the 1234 password on first login.

nachoparker commented 6 years ago

hi,

thanks for your feedback!

Thoughts?

Isn't it fast? <3<3<3 and still haven't optimized specifically

testbird commented 6 years ago

Strange that I have no ssh then. Maybe it's iptables related: Read from socked failed: Connection reset by peer. Accordingly, I could only try the ncp-web ip setting form, it said "could not find dhcpd.service" or simmilar.

I like your careful attitude towards changing existing systems. Yes, the .local (mDNS) should be configured by Armbian/FBX. May be worth a note in the install guide.

But I can't see if the ".lan" search domain is in /etc/resolv.conf, yet (no ssh) I thought the resolv.conf would be evaluated for NC's secure domains' configuration in newer NCP versions.

Can't see the speed from just the NC error page, yet. :-)

nachoparker commented 6 years ago

It's strange, I burn the SD card with the image, turn it on, and boom, I have SSH. I wonder why you don't

Can't see the speed from just the NC error page, yet. :-)

What error page are you seeing? you can't use Nextcloud either? if you can access ncp-web you should be able to access Nextcloud

testbird commented 6 years ago

Ok, re-wrote the image, and noticed the ethernet hub was hanging :-(, after restarting both I could access ssh. (Ok, that issue seemed not to be related to NCP.)

What's still strage with NCP is that I was not asked for a new root password to replace the 1234 default, as with stock armbian. The login went right into creating a new user account.

With nextcloud I got the "no trusted domain" error page. But retrying with the IP, hey I am very happy with the speed (ok, no data there yet) but finally it gives the user the impression that that thing is working right!

nachoparker commented 6 years ago

great!

Regarding the password issue: you are right, I want to restore it to armbian defaults, but I had to do that for the thing to build. See here. That is on the list.

With nextcloud I got the "no trusted domain" error page

What domain did you try?

There's a lot of polish to do for sure, it will be slow unless I find some help

thanks for the info

testbird commented 6 years ago

Hi, thanks for the link, and really also your other articles! Just a thought, aren't you using the word "we" in your writings maybe a small litle bit questionable or better improvable?

What I tried was "nextcloudpi.lan" (router/dhcp managed DNS), which works with port 4443 and ssh (now ;-)

Was anybody successful in installing docker on the HC1 on the stretch or buster release? I myself could not find a docker.io package in the repos.

testbird commented 6 years ago

Now I checked, /etc/resolv.conf contains my search domain "lan" in a line saying search lan. Is this already supposed to be parsed with the commit you made and adding <hostname>.<search domain> to the list of NC's trusted domains?

nachoparker commented 6 years ago

welcome! ;)

Just a thought, aren't you using the word "we" in your writings maybe a small litle bit questionable or better improvable?

What do you mean? don't follow :P

Was anybody successful in installing docker on the HC1 on the stretch or buster release? I myself could not find a docker.io package in the repos.

I am using docker on the HC1 right now. You have to install docker first. See here and here

Now I checked, /etc/resolv.conf contains my search domain "lan" in a line saying search lan. Is this already supposed to be parsed with the commit you made and adding . to the list of NC's trusted domains?

Mmm no, I don't think I add anything other than nextcloudpi.local (and the IPs and domains from ncp-web)

testbird commented 6 years ago

Ok, i think the sudo apt-get install docker.io docker-composer as in your second link can only work on oldstable (jessie) and unstable. There is currently no docker.io package in stable and testing.

For me on the HC1 the Armbian/stretch even the get.docker script had first failed, the docker-ce package stayed in an unconfigured state and service restarting failed. Strangely, after a reboot, now a service restart worked and I could successfully reinstall the package (configuraiton now completed).

testbird commented 6 years ago

With regards to the trusted domains, I had thought it would be good to have at least these included by default:

$IP $HOSTNAME (local and shorthand of the following) $HOSTNAME.local (mDNS) $HOSTNAME.$RESOLV_SEARCH_DOMAIN (local, manual or dhcp configured regular DNS) $NCP_CONFIGURED_DOMAINS

testbird commented 6 years ago

That could be hostname -I, hostname -A + .local and ncp configured domains.

nachoparker commented 6 years ago

domains -> aggree

docker -> I have installed it on odroid HC1 armbian stretch with

# curl -sSL get.docker.com | sh
testbird commented 6 years ago

Well, with that script I needed to do a reboot on the armbian dev (stretch) before it worked. I did not try the ncp docker on the ncp image. ;-)