PoC Brainstorming

[BernhardPosselt]

MVP should be the first target which should include the following

• Require users and devs to sign up:
  • e-mail
  • name or nick
  • password
  • captcha (re-captcha since everything else has been broken afaik)

API should be implemented as REST + JSON and require Basic Auth for the first iteration.

Later on most of the stuff will be parsed from the info.xml.

Let's assume I want to publish version 9.0.0 for the News app

Register App

An app id needs to be registered for an app developer before releases can be created, first come first serve (we will be able to intervene later on):

PUT /apps/news

{}

The JSON object is empty for now but will later on contain stuff like name, cateogry, description, links to homepages and issue trackers etc :)

Similarily an app developer can also unregister an app id (this will delete all releases as well):

DELETE /apps/news

Publish An App Release

This will contain much more in the end, namely dependencies, categories/tags etc. For the first try keep it small:

PUT /apps/news/9.0.0

{
    "download": "https://domain.com/news.tar.gz"
}

Delete An App Release

Straightforward: DELETE /apps/news/9.0.0

Get All Apps

In the future it should be possible to replace the call to this route by simply serving a continously updated JSON file (if ever needed because of performance issues). Therefore we do only minimal filtering based on the supported platform version GET /platform/9.1/apps.json

{
    "apps": {
        "news": {
            "9.0.0": {
                "download": "https://domain.com/news.tar.gz",
                "checksum": "asfasfafdasdf12312313123"
            }
        }
    }
}

The app will also contain more than the download link and checksum, like the name, dependencies etc

Packages Should Not Be Stored On The App Store

The app store should not host the packages, only download links. It happened in the past that the app store broke because disk space ran out and I don't think it can scale well enough.

So if you review apps, how do you make sure that the download link did not change? Simply provide a checksum (SHA256 sum).

The Case Against Git

In the beginning I thought it would be a cool idea to just base this on Git and GitHub like Bower, CocoaPods and many other package managers. The main issue with this is that most ownCloud apps have to be built first (mail, news, contacts, calendar?, notes?).

Building apps on the app store or a users installation is not a solution. It's too hard to sandbox and limited, but most importantly we would need to provide all sorts of dependencies (make, node and whatever people want).

Why not store everything on GitHub

GitHub has rate limiting in it's api so we can't hook up users

[jancborchardt]

cc @karlitschek @LukasReschke I guess

[LukasReschke]

Awesome, @BernhardPosselt. Thanks a lot for your thoughts on the possibility of modernizing the appstore. 🚀 🍻 👯 I have shared some of my thoughts below :)

So if you review apps, how do you make sure that the download link did not change? Simply provide a checksum (SHA256 sum).

Technically, I don't see any advantage in that. People can also just replace the content of the downloaded file.

To be honest, I'm seriously thinking whether we should just ditch the review business mostly. If somebody is a trusted developer just approve their app once and they can publish all updates they like in the future as well. Since the app releases are signed this also should be tamperproof enough.

• Require users and devs to sign up:
• e-mail
• name or nick
• password
• captcha (re-captcha since everything else has been broken afaik)

API should be implemented as REST + JSON and require Basic Auth for the first iteration.

Good call on going with Basic Auth for the first iteration. I'm wondering though if it would also be nice just to support something like "Login using GitHub" as well (of course any OAuth compatible thing might do). If somebody decides to have Two Factor Auth on their account there this also then is a free security gain for us.

Looking at the registrations on the forums I can tell you that more than 100 users just signed up there using GitHub. That seems to lower the barrier a lot.

Similarily an app developer can also unregister an app id (this will delete all releases as well):

I wouldn't delete the app per se and maybe just give it an "unmaintained" flag and show this then also in the GUI at some point. Also allows others to pick up the work and so.

Publish App Release

This will contain much more in the end, namely dependencies, categories/tags etc. For the first try keep > it small:

{
    "download": "https://domain.com/news.tar.gz",
    "checksum": "asfasfafdasdf12312313123"
}

I'd automatically generate the checksums when somebody uploads an app. Do a cURL request and calculate it server-side. Otherwise this is one more hard to understand step :-)

Packages Should Not Be Stored On The App Store

Not entirely sure about that one. I think the default should be "Host the stuff somewhere else" (e.g. GitHub releases) but we might as well offer somewhere an upload possibility for apps to share their apps. :-)

The Case Against Git

Absolutely with you on that. Also stuff like code-signing gets way harder in that case. :-)

[juliusknorr]

The Case Against Git

I kind of like the idea of using a git repository for managing the app store. It would make it possible to have any kind of review/moderation process also managed by the community. The app list could be managed in a simple JSON file, adding apps and pushing new releases can be done via pull requests.

In the beginning I thought it would be a cool idea to just base this on Git and GitHub like Bower, CocoaPods and many other package managers. The main issue with this is that most ownCloud apps have to be built first (mail, news, contacts, calendar?, notes?).

Just in case there will be no hosting of apps by nextcloud, the app developers need to build, package and upload their app anyways, so i don't see any issue there. They will just add the url of their prebuilt package to the repository.

The downside would be, that we don't get any kind of rating system with a git managed app repository.

But finally, as I always found the app ratings quite useful, I'm also fine with a dedicated app store.

[BernhardPosselt]

It would make it possible to have any kind of review/moderation process also managed by the community. The app list could be managed in a simple JSON file, adding apps and pushing new releases can be done via pull requests.

Exactly my thoughts prior to trying to find an API for it. In the end as a developer I want to push updates quickly and the pull request process does not scale that well. We've tried it with the review process prior to that and it was just too slow.

What I meant with building: Reviewing was one of the big advantages when using git. You could just diff tags :) However if a user uploads a package you can't be sure that it's built from the actual source which was reviewed. So you basically need to build it again.

Without these things you end up having to sync the app store and github which ends up doubling all the work. It's better to keep things simple first. Should we find out that we need it, we can always add it later on :)

[BernhardPosselt]

Technically, I don't see any advantage in that. People can also just replace the content of the downloaded file.

What I meant was that you sha256sum the package, so ownCloud would query the link and the package and compare the hash with the downloaded package. Sure code signing would also work but as you might know I couldnt get it working.

Good call on going with Basic Auth for the first iteration. I'm wondering though if it would also be nice just to support something like "Login using GitHub" as well (of course any OAuth compatible thing might do). If somebody decides to have Two Factor Auth on their account there this also then is a free security gain for us.

Exactly, this will also make it easier for users to file issues since they only need one account. I'm unsure however how well you can integrate it into a REST api, but we'll figure something out once we want to tackle that.

I wouldn't delete the app per se and maybe just give it an "unmaintained" flag and show this then also in the GUI at some point. Also allows others to pick up the work and so.

Sometimes you just want to delete things and calling a DELETE route should get rid of the resource. However the "looking for maintainer" use case could also be implemented at some point.

I'd automatically generate the checksums when somebody uploads an app. Do a cURL request and calculate it server-side. Otherwise this is one more hard to understand step :-)

Good call, it makes the API easier. If we limit the size to 100mb or something like that, storing it in a tmp folder should not be an issue. Plus you get the added benefit of checking if the certificate is valid (we should only accept https).

[BernhardPosselt]

Ok, took the time to work on it a bit today.

Curl get command works already:

curl http://localhost:8000/api/v1/platform/1.0/apps.json

But it's not in the designed format (yet) :P

[
  {
    "id": "news",
    "name": "News",
    "description": "# HI",
    "user_docs": "http:\/\/127.0.0.1:8000\/user",
    "admin_docs": "http:\/\/127.0.0.1:8000\/admin",
    "developer_docs": "http:\/\/127.0.0.1:8000\/dev",
    "issue_tracker": "http:\/\/127.0.0.1:8000\/issue",
    "website": "http:\/\/127.0.0.1:8000\/",
    "created": "2016-06-05T22:04:46.419739Z",
    "last_modified": "2016-06-05T22:04:46.419781Z",
    "categories": [
      "productivity",
      "tools"
    ],
    "authors": [
      1
    ]
  }
]

Full admin interface is already present (meaning you can edit everything already), so are translations (not for fields yet however). Getting started docs have been written. For a dev setup you only need make, Python 3.4+ and venv (which is bundled in some distros) and you can set up a dev install with make dev-setup.

I'll probably still have to adjust the database model and I will squash migrations, so expect migrations to break which will require you to resetup the db.

[BernhardPosselt]

Closing this ticket since #9 is ready and the PoC is basically finished :)

No we need some PHP guys that check out the API and try to implement it in ownCloud

[adsworth]

Great stuff!