flathub / org.DolphinEmu.dolphin-emu

https://flathub.org/apps/details/org.DolphinEmu.dolphin-emu
17 stars 18 forks source link

Making this Flatpak official #224

Open OatmealDome opened 3 months ago

OatmealDome commented 3 months ago

Hello - I'm a developer from the Dolphin project.

For the past 8 years, the Dolphin project has chosen not to create any official builds for Linux, instead opting to relegate that task to the distributions. However, with the rise of the Steam Deck, we've observed significant user demand for an official Flatpak version of Dolphin. As this unofficial Flatpak is being used by many people already, we think that collaborating with you and making this Flatpak official would be the best way to approach this. Is this something that everyone is interested in doing?

If so, then there are a few points we should probably discuss:

  1. We will likely be hosting our own Flatpak repo in addition to using Flathub, so we'll likely need to upstream some of the data here so we can create our own builds. I assume this will include the .yaml file at the very least. Is there anything else (i.e. what other PRs need to be made to the upstream repository)?
  2. The app ID utilizes a non-existent domain. Dolphin's official domain is dolphin-emu.org. We do not have any control over dolphinemu.org. This might pose a problem with verification on Flathub. Is it necessary to move to a new app ID?
  3. Control of the Flathub app should be transferred to an account owned by the Dolphin project. I believe there is a process for this that we can follow, but we won't do anything without your explicit approval.
  4. Is it possible for us to build Flatpaks on our own infrastructure and upload the artifacts to Flathub? We prefer having as much control over the build and distribution pipeline as possible. It would be nice to create one build that we can upload to both our own repo and to Flathub. (I think this might be a question better suited for Flathub, but I'm asking here just in case.)

Thanks for your time!

cpba commented 3 months ago

Hello - I'm a developer from the Dolphin project.

For the past 8 years, the Dolphin project has chosen not to create any official builds for Linux, instead opting to relegate that task to the distributions. However, with the rise of the Steam Deck, we've observed significant user demand for an official Flatpak version of Dolphin. As this unofficial Flatpak is being used by many people already, we think that collaborating with you and making this Flatpak official would be the best way to approach this. Is this something that everyone is interested in doing?

Yes indeed, I would very much love this.

If so, then there are a few points we should probably discuss:

1. We will likely be hosting our own Flatpak repo in addition to using Flathub, so we'll likely need to upstream some of the data here so we can create our own builds. I assume this will include the `.yaml` file at the very least. Is there anything else (i.e. what other PRs need to be made to the upstream repository)?

We have two downstream patches but only one is important, it changes the priority that dolphin uses to select its configuration folder to prefer xdg when inside a flatpak environment. This is to prevent issues that happen when users have leftover configuration files in directories that aren't writable from within the sandbox.

I have an open pull request:

https://github.com/dolphin-emu/dolphin/pull/10556

The second one is only a side-effect of the first, it clears the -dirty part of the version string so that users can still use netplay with each other (as the version string must match exactly) it won't be needed any longer if the first one is merged in upstream dolphin.

On the beta branch (the one that tracks your dev releases) there's another patch that forces the version string, this is needed because the builder does a shallow clone to improve build times (so the git tags are lost, and with them the version string). This can be prevented by doing a full clone, which I assume will be done when the builds are moved to the upstream pipeline.

Other than that, you will also want to merge the appdata/metainfo file. This is a metadata file that holds the information that users see when they install dolphin from app stores such as gnome-software, kde-discover (the default store on the steam deck) or the flathub website itself.

I keep meaning to update this file to get it into a more mergeable state but I haven't done so yet.

I'd like to help as much as possible, don't hesitate to ask.

2. The app ID utilizes a non-existent domain. Dolphin's official domain is `dolphin-emu.org`. We do not have any control over `dolphinemu.org`. This might pose a problem with verification on Flathub. Is it necessary to move to a new app ID?

I'm not completely sure about this yet but I hope it won't be necessary.

Other than the domain being wrong as you note, it has two uppercase letters, which flathub doesn't allow. Changing the appid is a hassle for users so an exception was created for this specific case, I believe another exception can be made for the missing hypen.

Changing the appid would involve creating a new app with the desired appid and closing this one by marking it as eol, from flatpak's point of view this is a completely separate application. There are ways to ease this for end users such as marking the old one as end-of-life-rebase which will prompt the user to uninstall the old one and install the new one but it's best to avoid this if not strictly necessary.

3. Control of the Flathub app should be transferred to an account owned by the Dolphin project. I believe there is [a process for this that we can follow](https://docs.flathub.org/docs/for-app-authors/submission#someone-else-has-put-my-app-on-flathubwhat-do-i-do), but we won't do anything without your explicit approval.

Yes, I know it sometimes doesn't look like it from the outside, but having the applications be directly in the hands of the upstream developers is the main goal of flathub.

Would that be just your account to start with? We could just open an issue asking for the accounts to be added.

Still, I'd like to be included in the process as I believe I can be of help.

4. Is it possible for us to build Flatpaks on our own infrastructure and upload the artifacts to Flathub? We prefer having as much control over the build and distribution pipeline as possible. It would be nice to create one build that we can upload to both our own repo and to Flathub. (I think this might be a question better suited for Flathub, but I'm asking here just in case.)

Yes.

Originally this wasn't allowed, but exceptions have been made for other projects, for instance, firefox is built by mozilla. I'm sure an exception can also be made for this particular case.

OatmealDome commented 3 months ago

OK, great! The first thing we should probably do is getting everything upstreamed so that builds can start being produced on our CI.

I have an open pull request:

dolphin-emu/dolphin#10556

Do you mind rebasing the PR? Once that's done, I can re-run CI and poke people to see if we can get it merged.

Other than that, you will also want to merge the appdata/metainfo file. This is a metadata file that holds the information that users see when they install dolphin from app stores such as gnome-software, kde-discover (the default store on the steam deck) or the flathub website itself.

I keep meaning to update this file to get it into a more mergeable state but I haven't done so yet.

I'd like to help as much as possible, don't hesitate to ask.

What needs to be updated in in the file to get it mergeable?

Would that be just your account to start with? We could just open an issue asking for the accounts to be added.

Having it under my account is fine for now, I think. Is there a way for multiple accounts to have shared ownership of a Flathub app (like an organization / team / whatever)?

OatmealDome commented 3 months ago

OK, I've merged dolphin-emu/dolphin#10556.

cpba commented 3 months ago

Thank you this is great news!

Sorry I haven't been responsive, it's been a stressful couple weeks. I have some free time tomorrow morning, I only need to drop the patches, test the build and open a new pr.

OatmealDome commented 3 months ago

No problem, take your time. We're not in a rush, anyway \:P

Just let me know when you've opened it.

cpba commented 3 months ago

There's good news.

It is also possible to build on the dolphin infrastructure and upload to flathub, but this is allowed at the discretion of the flathub admins. I was told to look at how OBS and Warzone2100 handle it for reference, we should set it up on the dolphin ci and once we get it to work they'll review it and give us a flathub upload token if they're happy with it.

This could take a bit so I'm thinking we should just do the verification first and keep building from here in the mean time, would that be alright?

OatmealDome commented 3 months ago

should have received an invite to take ownership of the repo.

Yes, I've accepted the invite. Thanks!

This could take a bit so I'm thinking we should just do the verification first and keep building from here in the mean time, would that be alright?

That sounds good to me. I think the next step would be to PR the YAML and the XML, so we can get builds set up on our CI.

barthalion commented 3 months ago

As Dolphin is one of the first 5 apps on Flathub, I'm fine with marking it as verified manually, similarly to Firefox/Thunderbird. While we have the technology for moving IDs around, I feel it's justified not to use it this time.

FWIW, I can handle it when you see it fit @OatmealDome.

cpba commented 2 months ago

@OatmealDome sorry for the delay, I've opened a pr here: https://github.com/dolphin-emu/dolphin/pull/13040

OatmealDome commented 3 weeks ago

Now that https://github.com/dolphin-emu/dolphin/pull/13040 has been merged and our CI infrastructure is set up, what would be our next steps?

cpba commented 3 weeks ago

This has only been done for a handful of projects so far so there isn't yet a set process.

The flathub admins will need to manually review the ci setup so the next step would be to open an issue here asking for it and providing the relevant information.

Since you did the work on the ci you're the best person to do this but I can help if you'd rather not.

I've been warned this step is going to take a while, but it's the last one ;)

OatmealDome commented 3 weeks ago

Sure, I can make the issue. How exactly should I phrase this request? Is there a template I need to follow?

cpba commented 3 weeks ago

There's no template, this is not commonly done.

I'd write something like this:

Transition org.DolphinEmu.dolphin-emu to upstream builds

I'm a member of the upstream dolphin-emu development team and owner of the org.DolphinEmu.dolphin-emu repo.

We have integrated flatpak builds into our ci system:

We'd like to replace the flatpaks currently being built on the flathub build system with the ones from our own infrastructure, in a similar way to how the OBS and warzone2100 projects currently do it.

You can review our current ci setup here:

add relevant links

maybe short explanation of how it works

any technical questions you may have

I can write it up a bit more if you need.

By the way, I understand that there aren't any yet, but is there a repo for stable builds? The beta branch is currently serving the dev builds but we'd need it for the stable branch.

OatmealDome commented 3 weeks ago

Ah, thank you. I'll follow your format when opening the issue.

By the way, I understand that there aren't any yet, but is there a repo for stable builds? The beta branch is currently serving the dev builds but we'd need it for the stable branch.

Yeah, the stable repo is currently at https://flatpak.dolphin-emu.org/releases.flatpakrepo. There is nothing published to it right now since we haven't made any releases after the Flatpak PR was merged.

cpba commented 3 weeks ago

In case it helps, this was the issue for the warzone2100 project:

https://github.com/flathub/flathub/issues/3652

OatmealDome commented 3 weeks ago

I've opened flathub/flathub#5808.