Porting ownCloud

[jancborchardt]

I’m trying to port ownCloud to Sandstorm at the moment but have some problems/questions regarding the porting guide:

• The demo command spk init -p 10000 -- node main.js isn’t really applicable since the default port is just 80 and the app runs by default (simple LAMP). Do we have to define a specific port and a launch command – if so, how?
• dependencies (and submodules) aren’t mentioned anywhere – how are they handled?
• permissions neither
• setting up a user account and choosing/configuring the database

To answer these questions I had a look at the Wordpress port: https://github.com/dwrensha/wordpress-sandstorm/commits/master – but it’s very specific and a lot of customization, so is there any general documentation on this?

I do have a script to handle dependencies, permissions and pulling of apps: https://github.com/jancborchardt/owncloud-scripts/blob/master/owncloud-setup.sh could that be used?

(Also cc @DeepDiver1975 @PVince81 from ownCloud for info.)

[ocdtrekkie]

This is one I took a stab at once, when I had no idea what I was doing, and then ran off to go play with apps not made in PHP for a while. (I love PHP to death, but Sandstorm ports for it are a pain.) ownCloud's one of the first things I wanted to do, but I don't think my Linuxfu is good enough to do it myself.

So, the particular issues to handle here:

For setup of the scripts, I'd check out @jparyani 's app ports, he's ported a few PHP apps like MediaWiki. https://github.com/jparyani/mediawiki-sandstorm/blob/master/launch.sh You'll see he copies the SQLite file into /var, which is writeable, if it isn't already there on launch.

/var is the only writeable directory in an app, so if the app needs to be able to write to it's own directory, you might need to run the entire app out of /var. (If that's the case, your setup script has to copy the app there, because /var isn't included in the app package.) If it's possible to port the app in a way that doesn't write to it's own directory, that's ideal, if you like move the /data directory to /var, for example.

Regarding the port, you use the port that your app is going to use. If it's coming out on port 80, you use port 80. A bunch of node apps I ported used 8000 or 8080 or what-have-you. One of them made me specify the port in the launch command, so I had to state the port number twice in the spk init. Once to tell Sandstorm to look at that port, and once to tell the app to use that port. :)

The spk dev tool should collect all the dependencies when you're testing the app in dev mode. It'll populate a sandstorm-files file in your app's directory. This is the somewhat magical portion of the porting tool, it works very well.

[kentonv]

We definitely need to improve the docs here.

The demo command spk init -p 10000 -- node main.js isn’t really applicable since the default port is just 80 and the app runs by default (simple LAMP). Do we have to define a specific port and a launch command – if so, how?

You won't be able to open port 80 inside the sandbox since you aren't root. Instead, configure your web server to listen on a different port. Note that the command you pass to spk init should be the command that starts your web server, e.g. apache, not just the CGI command. Ideally you'll want to run this command in such a way that it writes its log to stdout and does not fork off into the background; most servers have an option for this.

(Actually, you'll probably end up wrapping your server command in a run script that also starts the database, etc.)

As Jake says, it may help to look at some of @jparyani's PHP ports, including Mediawiki, RoundCube, etc.

dependencies (and submodules) aren’t mentioned anywhere – how are they handled?

Your package ends up containing the entire userspace needed by your app (all binaries, libraries, code, assets, etc.). If you use spk dev as directed in the porting guide, it will automatically detect all your dependencies and pull them from your local system.

permissions neither

Ugh, it appear we haven't gotten this added to the wiki yet, but this blog post describes how you can integrate with Sandstorm's login system (see the bullets about X-Sandstorm-Username, etc.):

https://blog.sandstorm.io/news/2014-08-01-mediagoblin.html

We often adapt apps to automatically create user accounts corresponding to each X-Sandstorm-User-Id they see. Keep in mind that the entire app instance is protected by Sandstorm's own sharing model, so you can assume that if a user gets access at all, the app instance owner shared the instance with them. Right now this sharing happens by simply copy/pasting the link, but this will get more sophisticated later.

Eventually, Sandstorm will have a sharing UI that supports assigning a permission level to each user. At the moment, though, this isn't implemented yet. You can distinguish between the instance owner and non-owners by defining a single "admin" permission -- see MediaGoblin's sandstorm-pkgdef.capnp for an example of defining such a permission.

https://github.com/jparyani/mediagoblin/blob/sandstorm-master/sandstorm-pkgdef.capnp

I assume ownCloud already implements its own permission model, though, which you could simply use together with Sandstorm's login.

setting up a user account and choosing/configuring the database

It's up to you to include whatever database you need inside your package. If at all possible, try to use sqlite. Sandstorm app instances are intended to be small, so MySQL and Postgres tend to be overkill. Among other problems, they tend to pre-allocate way too much disk space.

[JamborJan]

Hi @jancborchardt,

What is your status of porting ownCloud? I'm planning to look into this also but if you have already a usable status somehow it would be good to share your findings instead of starting from zero again.

Thanks!

[jancborchardt]

@JamborJan I didn’t look further into it yet. If you want to take it over please go ahead. :) For any questions about ownCloud, come join #owncloud-dev on IRC.

[JamborJan]

Ok I start working on that. I assume it will take some time but let's see.

@kentonv I'm not sure what you need to do here but from my point of view the issue can be closed. Unless you want to track the status of the ownCloud port with it.

[jancborchardt]

@JamborJan great stuff, thanks a bunch!

[paulproteus]

Hi JJ!​

Are you willing to test out using "vagrant-spk" for this?

https://github.com/zarvox/vagrant-spk + https://github.com/sandstorm-io/sandstorm/wiki/Porting-an-app-with-vagrant-spk

Drew did some work on the tool recently, and I'd be very grateful if you can try it and let us know how you like it.

It should resolve a lot of the issues that made your last package hard to create, so I'm very hopeful that you can test it out and see if it actually makes things easier.

Cheers!

[JamborJan]

Hi @paulproteus,

I plan to package the app sandstorm friendly and maybe will try to use sqlite as database to save ressources.

The vagrant-spk tool sounds good. I will check it out and hope that the mentioned focus on php and mysql is not limiting me now and I can use sqlite.

The process via Docker worked for me pretty well at the end with my last port. It was a normal learning process which delayed the whole thing.

[zarvox]

Hi! Just to let you know, due to technical limitations, I had to make some changes to how the tool works, which makes the wiki page instructions slightly out of date. I'll try adding a README to vagrant-spk and updating the wiki soon.

[JamborJan]

@paulproteus vagrant-spk setup was pretty easy so far. Will continue with that and report my experience and findings.

@zarvox it would be awesome to be able to select somehow which database engine to use and other options. I will fork your repo and try to figure something out and make some suggestions.

[classner]

Thumbs up for the project, I hope you succeed! :)

Just a note on the database engine: I see the advantage of sqlite, especially concerning data locality and portability. There are some Owncloud plugins, however, that only work with a MySQL backend. So it would be great to have the option to choose the MySQL backend, even it it's then necessary to do some additional steps and get the server configured...

[JamborJan]

@ChrislS do you know which plugins for example? I'm not done with the decision yet and if there are good reasons to not use sqlite then I don't see why not sticking to mysql.

[jancborchardt]

@JamborJan for performance reasons we recommend using MySQL for any setups with multiple users or syncing with multiple clients. SQLite is ok for single-user setups or dev setups, but when it’s on Sandstorm it’s best to just use MySQL.

[classner]

I am certainly no owncloud poweruser, so I don't have a good overview over the general requirements of plugins. I happen to know, though, that the Passman app (https://apps.owncloud.com/content/show.php/Passman?content=166285) only works with the MySQL backend.

[JamborJan]

OK, makes at least some steps easier as https://github.com/zarvox/vagrant-spk has mysql already by default.

[JamborJan]

Status: https://github.com/JamborJan/vagrant-spk https://github.com/JamborJan/owncloud-sandstorm

Nothing is working so far. When you build the app it actually wants to download the index php and the database is not up and running, I think I screwed up my vagrant-spk fork. I will check this in more detail later. Everybody who is interested can have a look.

[ocdtrekkie]

As a cool thing, Davros being compatible with the ownCloud client means we have a clear working pattern to ensure ownCloud's clients will work with ownCloud on Sandstorm as well. :)

[JamborJan]

I'm happy that Davros has been released as I'm not able to do further development / investigation at the moment. Maybe someone else (or I later) can pick up the work from here to ensure that there is more than one solution.

[paulproteus]

Hi all! I'm going to remove the documentation tag here, since with the existence of vagrant-spk and a PHP-specific packaging stack, I think that docs are not the main issue, or at least, if they are, I want someone to tell me how the docs didn't work for them.

Details on PHP packaging with vagrant-spk: https://docs.sandstorm.io/en/latest/vagrant-spk/packaging-tutorial/

[KuenzelIT]

Is there any progress for the owncloud app? Is there still someone trying to port it?

[ocdtrekkie]

Nobody I know of is working on it.

[jancborchardt]

If someone picks it up, note that Owncloud was forked by the majority of the core developers and community into Nextcloud. :) Our code is at https://github.com/nextcloud and if there's any questions, we hang out in #nextcloud-dev on freenode IRC.

[ocdtrekkie]

I think I am going to close these. I gave Nextcloud a try recently, and it's very, very much more than ideally should run inside a Sandstorm container, to the extent that it's largely an app platform of it's own at this point. I know it's a popular request, so if someone was to successfully package it, I think it'd be sensible to allow it in the market, but I don't think we want to consider it an issue that it's not present or dedicate the Sandstorm community's limited resources to packaging it.