General Overhead

[timkaechele]

The idea of using Docker.io to manage app dependencies is very appealing to me, but I am concerned about the overhead. Of course it's a beautiful concept for developers, they can choose their programming language, db, webserver, etc. freely, but not everybody uses a high end server to run its GDS instance. Most of them will be cheap linux boxes that don't run on 16gigs of RAM and the latest processors.

So why don't we use a generalized installation of programming languages, supporting the two latest versions of Ruby, Node, Python, PHP (no idea if there's an PHP equivalent to RVM or NVM), etc. It would force developers to stay up to date and update their apps from time to time, but at least all apps would benefit from the performance and security improvements new versions come with.

The same thing with databases. Install the latest versions of Postgres, Mysql, Couch, Mongo… The App could define it's database backend with the package.json and would get a unique username and password for a database and not an entire instance. Unused DB instances could be shut down.

By abstracting internals like users and contacts through an api nobody would need to interact directly with the specific db and would not be able to mess with things.

Same story with webservers, make a decision use nginx, apache or something lightweight like lighttpd, but don't create hundreds of webservers for the sake of freedom. Every major programming language is able to use some kind of FCGI or applicationserver reverse proxy setup to deliver it's content. Some apps like JABBER don't need a webserver at all. Specify it all in the package.json. If you need deeper webserver configuration for whatever reason use the internal GDS Webserver API to configure it and notify the user about these requirements. By doing so you would get more control over the platform itself and guarantee security.

I know everybody will be like: "Oh but that is evil that's not absolutely open”. Yep that's true, but personally I think it's better to draw a line than creating a big mess with hundreds of web/db servers. Developers still have the freedom to choose a lot of things and if a app/package really needs specific things it stil can l use docker.

[lukasbestle]

I totally agree with the argument that Docker.io or similar systems could be a bit too resource hungry for the purpose.

Still, I would not limit usage to a specific set of DBs, webservers etc., because some existing apps are depending on .htaccess files or a specific DB - we can't generally limit it.

I don't know how we can implement it, but the probably most flexible solution would be to provide the apps with different database and webserver packages (just like node modules work -> third-parties could offer theirs as well), but they would not be installed to the sandbox of the app but system-wide. The app would then be granted permission to use the DB or webserver using their own sandboxed system user.

[timkaechele]

I described this package system with the DBS the … was meant as an etc. :DD while looking at DBs I totally understand the differences between them, especially nosql and relational dbs differ in their approach to handle data, but Webservers don't they all send something a backend or static file provided to a client, the main difference between them is their configuration. It's just not very standardized, so why not creating an standardized API, that can easily be adapted. some small converter for htaccess and other config formats could translate them to API syntax.

[lukasbestle]

The important part you didn't write about: It has to be packages and not installing the latest versions from the beginning. So only install the stuff when required. :)

That's quite complicated. What if you just want to install Wordpress or any set of script files coming bundled with a .htaccess? It simply has to work. Additionally, each web server has features you can't easily convert - or it will add a lot of buggy overhead to the core doing that converting stuff.

As I wrote in services.md, we could use Pound as web frontend, which would make it easy to run multiple web servers in the backend at the same time all able to register at Pound via an API (which is not much of an overhead).

Although this server performance stuff seems like an important step, we should not forget that providing fresh infrastructure to every app is more secure and also more flexible (moving an app between systems would be extremely easy!) than using central installations of everything. Maybe Docker is not suited for this, but centralizing everything can't be the solution.

Also: We are discussing about DBs and web servers here, but that's not everything we could centralize, probably adding additional overhead we can't break down to "a package providing something centrally on the server" as every type of server is different.

[timkaechele]

Yeah actually I'm a bit confused by reading things like porting wordpress. that looks to me like trying to build a more user friendly server not a decentralized platform for apps. Where's the advantage of using bloated complex software like wordpress, that lead to this movement of making websoftware more accessible to the end user in the first place. I always thought about the GDS as an iOS for the web, a platform that just works. That includes a main system language. That every app should use, if possible. It's not a big deal if you don't, but well it is advised to do. If you need another programming language /db/caching server whatever, your app can require the requirement as a dependency similar to the Synology Package System. This system would have a main Appstore with a code review, so that end users are protected. A system that is accessible to the end-user and also to the developer community. That means to me that a developer doesn't have to care about a filesystem, a webserver or other services and interfaces, there should be a small but yet powerful api for that, that is implemented as open as possible. I assume that most apps get written for GDS from scratch anyway. By open sourcing things like the filesystem api every GDS App would be still useable outside of GDS.

[lukasbestle]

I totally agree with the idea of "just works™".

There should be no Wordpress app, but a webserver app supporting Wordpress installation. IMHO this is important, since not being able to install existing apps would encourage you to rent another server for the purpose - which is probably centralized again. Keeping everything on your own box can be a huge advantage.

The idea of having one programming language and a set of APIs is really great, that could make the whole ecosystem more flexible and less fragmented and therefore less power-hungry.

Still, there should be a Docker.io API providing Docker.io instances if required for apps like webservers.

[timkaechele]

Yeah totally agree, there should be a Docker.io API for things that are already written like Jabber Servers.

But I am not totally sure about the weberver thing, a webserver should already be built in so why not creating an app that puts a webinterface on it, you could use kind of inapp "purchases" to install existing webapplications. so you wouldn't have to deal with all the wacky things that confuse the average user. Just works. If you are more experienced you could still access the webserver manually.

I think of some kind of virtualhost setup that gives you the ability to install whatever you want in a clean environment, kind of like uberspace does.

[piranna]

The best "just works" experience I could achieve have been with PaaS platforms like Google AppEngine, Nodejitsu, AppFog, Heroku and similars, maybe we should forget about freedom of webserver or database, use them as comodities that anyone can use internally on their machine but being transparent to the user, and just offer high level APIs in Python, Javascript/Node.js or Java to the user. I think this would be the easiest way... Just develop and test in local environment (maybe a local instance of the platform installed with a .deb or NPM and configured to run your development files), git push the code to your own or cloud server and it gets running, no more steps.

Send from my Samsung Galaxy Note II El 14/04/2014 12:30, "Tim Kächele" notifications@github.com escribió:

Yeah totally agree, there should be a Docker.io API for things that are already written like Jabber Servers.

But I am not totally sure about the weberver thing, a webserver should already be built in so why not creating an app that puts a webinterface on it, you could use kind of inapp "purchases" to install existing webapplications. so you wouldn't have to deal with all the wacky things that confuse the average user. Just works. If you are more experienced you could still access the webserver manually.

I think of some kind of virtualhost setup that gives you the ability to install whatever you want in a clean environment, kind of like uberspace does.

— Reply to this email directly or view it on GitHubhttps://github.com/grand-decentral-station/concept/issues/41#issuecomment-40352139 .

[lukasbestle]

As I already wrote in services.md, we could use Pound for that to provide a port 80 (or 443) to any app. There could be an API to register an app/site with a domain and the internal port. SSL certs would be handled centrally in the system.

Pound would be installed by default (as it would be responsible for the admin interface as well), and the "webserver" app would be an additional LAMP installation managing all web apps with an app store and a "manual app" feature making it possible to define a path in the file system where an app is located. Instead of using virtual hosts, it would create actual hosts on different ports for each app to separate the hosts.

@piranna Pushing using git is a bit nerdy in my opinion. It has to work with existing apps. All the other apps get high level APIs, exactly.

[timkaechele]

Well I don't understand the "nerdy" argument (you are a developer that's always a bit nerdy :D) deploying via git would be really great. In the ruby/rails world it's somehow a defacto standard and works seamlessly. pushing updates would become really easy and installing an app would become more secure, because you wouldn't rely on strange developer provided install scripts you could analyze the app stack after receiving the code and install everything just like heroku does. Yeah I know that isn't that git specific but with the hook system provided by git it would become a lot easier. Of course only the new apps would profit, but it shouldn't be that hard to integrate existing apps into the system just like homebrew did. By doing so GDS would become a full featured PaaS Provider, that makes development a breeze.

[lukasbestle]

We have to keep in mind that our target group is not a developer like us, which is a person already able to use other PaaS services. We need to serve the average customer and provide a system which works like Facebook - no hassle involved.

If you tell them that they can "deploy" using "git", they won't understand that. They want an app store where you can click to install an app like with Synology DSM. :)

We can offer that feature, but we can't use it as a base for all app installations. Maybe we could implement it as a "deploy server app" for people who want to code their own apps?

[piranna]

If you tell them that they can "deploy" using "git", they won't understand that. They want an app store where you can click to install an app like with Synology DSM. :)

I don't see why that app store would not be able to work internally as a frontend to "git push" since there are also implementations of git in Javascript... :-)

[timkaechele]

Oh damn, you misunderstood it, i didn't mean to confront an average user with git and deploy, and hook and all these crazy words :D It would be the underlying system, the user would see a classical progress bar but under the hood all these awesome concepts would do their job :D

[lukasbestle]

Alright. :D Yeah, why not. :)

[jansauer]

Having a own box for GDS is a great idea but I don't see this box being a current Pi. With technology evolving rapidly, thinking of GDS to run on a medium box with approximately 8GB of ram shouldn't be unrealistic. This would allow for module separation with for example virtualisation and up to 10 users. One box per user doesn't scale well and you have less entry threshold if a user can get it's first account at a box of a friend he trusts. GDS should rather allow a user to moved to a other (own) box at a later point.

[piranna]

GDS should rather allow a user to moved to a other (own) box at a later point.

This reminds me about UnHosted remoteStorage concept...

[jansauer]

I really like the idea of unhosted. So far GDS and UnHosted have a lot in common. But I see GDS as a more intelligent central place for app with business logic.