Audiveris / audiveris

Latest generation of Audiveris OMR engine
https://audiveris.github.io/audiveris
GNU Affero General Public License v3.0
1.53k stars 227 forks source link

Also make linux releases! #534

Open Altonss opened 2 years ago

Altonss commented 2 years ago

Hello, I just discovered the project and am very interested! The project seems amazing and I want to help bringing the software on linux. I saw we can already build it from source and that's already great. But It would be even better provide builds, for example through AppImages or Flatpaks :) If I find the time to help with this issue, I would be happy to contribute!

hbitteur commented 2 years ago

Sure, our small team would be happy to benefit from such contribution!

Personally I develop on Windows and my competency about Linux is close to zero. So I will make sure that Max, by far our most competent Linux guy, is aware of your proposal.

I know nothing about AppImages and Flatpaks. I think I remember that Oracle is to provide a general binary-packaging utility in some next Java release (perhaps based on jlink?). A track to explore. We just have a Windows-specific installer today, a more general approach would be interesting of course.

Altonss commented 2 years ago

I made some progress to package Audiveris as a flatpak. For the moment it is using the build obtained using gradle and wraps this into a flatpak. That's not very clean, but at least it's a beginning. Should I share my recipe? I could also try something with the jar files, but that's more complicated as I would need gradle to run these...Or am I wrong?

Altonss commented 2 years ago

Does Audiveris needs network access? Because I got some errors because of some lacking network access

hbitteur commented 2 years ago

Audiveris checks periodically for a potential new release. This is the only automatic network access I can think of (and you can change the polling period which is weekly by default). If there is no network, you should get just a warning like "Could not connect to Audiveris project".

Of course if you manually select, in the Help menu, items like Handbook, Wiki, Web Site, Check for Update, you will try to reach the information on the network.

hbitteur commented 2 years ago

Regarding flatpak, I can't help you. I will let Max answer your question if he can.

mwilck commented 2 years ago

I've created a flatpak recipe here. I created a helper tool that allows downloading all Java dependencies and using them for a clean flatpak build using grade --offline. I have created 4 branches for packaging 5.2.4, 5.2.5, and the current master and development branches, respectively. These branches contain all required dependencies, so they can be used for a simple flatpak-builder invocation. The bootstrapping procedure for downloading the dependencies is explained in the README. There are also two slim branches with no dependencies (scripts and scripts-17 for openJDK11 and openJDK17, respectively).

Network access works in my package. But opening a browser to read the docs currently doesn't work, this seems to be a generic Java/flatpak issue.

@Altonss, I just discovered this issue here after I was already done with my work. I hope you're not upset. Perhaps we can make a joint effort to get this ready for prime-time?

@hbitteur, would you agree to submitting these packages to flathub?

Altonss commented 2 years ago

Thanks a lot for your work @mwilck ! I already had tried to build the flatpak, and it kind of worked, but thanks for having made a clean version. Are all the functionnalities working? It would be sooo cool to be able to release some day on Flathub :)

mwilck commented 2 years ago

This is still in it's early stages. I haven't checked if everything works (as I said, launching a browser does not). I've opened some sample music sheets and run OMR on it; I was quite satisfied with the results so far. I'd be very grateful if you could give my packages a try. You should be able to build them fairly easily (clone my repo, checkout branch b5.2.5, and run flatpak-builder --repo /some/dir builddir org.audiveris.audivers.yml).

mwilck commented 2 years ago

I've created a short Wiki page explaining what the various branches are.

mwilck commented 2 years ago

The web browser problem is solved now due to help I got in the openJDK17 issue.

mwilck commented 2 years ago

@hbitteur: I'd appreciate a comment if you'd agree with submitting this to flathub, and if not, what you think I (we) should do beforehand. In particular, would you object me submitting this with the flatpak application ID org.audiveris.audiveris?

hbitteur commented 2 years ago

@mwilck I have no objection to your submitting. And I'll be out for a couple of days so don't wait for other comments from my side, simply keep going as you intend.

mwilck commented 2 years ago

I haven't submitted to flathub yet, but I've reworked the build scripts such that no dependencies need to be stored locally. Everything is now downloaded by flatpak-builder before gradle builds offline.

hbitteur commented 1 year ago

Any progress on this topic? We do need such Linux releases!

mwilck commented 1 year ago

Sorry I haven't been able to follow up here. I suppose my previous packaging work could be brought back to work with relatively little effort. I will try to find some time to work on it soon.

FYI, back in 2022 when I worked on this, I got frustrated because Audiveris would always crash in unpredictable ways in the text recognition stage, typically with references to freed memory, which AFAICT should never happen in Java. I am certain that this wasn't caused by the flatpak, it also happened with native binaries, so maybe it was an issue with Tesseract, or possibly with my JRE (actually various different JREs I tried). I didn't want to submit something I knew was going to crash on non-trivial scan attempts.

I guess I should have opened in issue here, but I thought that this can't be a generic issue but must be caused in some ways by my Linux environment. I didn't make any progress with the bug analysis though, and eventually gave up, especially as the project for which I had intended to use Audiveris was finished.

mwilck commented 1 year ago

So ... I had another look at my build environment for the flatpak, and to my surprise, I found that it still works smoothly on the lastest code. I have a branch for packaging Audiveris 5.3.1, and the development branch has all the meta data for packaging current Audiveris development. The other good news is that I couldn't reproduce the Tesseract crashes I'd previously seen.

So it seems that we're actually in quite decent shape. I'm sure there are still lots of issues (for example, the "About" page shows wrong versions, not idea why).

mwilck commented 1 year ago

@hbitteur:

The flatpak organization prefers that the author of an application submits the package to Flathub. Maybe you could consider doing the submission yourself? The procedure is described here.

You need a fork and a clone of flathub itself, which must be populated with the necessary files to build the Audiveris package. I have created a sample flathub fork here ("audiveris" branch). It was created from my Audiveris-flatpak repo, branch b5.3.1, using the script flathub-copy.sh.[^gen]

AFAICS you could use my flathub fork for a first submission attempt of Audiveris 5.3.1 without changes. However, please review it again, in particular the org.audiveris.audiveris.metainfo.xml file. If you find issues, or have any additional questions, contact me. Please also have a look at the Flathub submission requirements page, licensing etc.

Note that Flathub's requirements wrt licensing aren't overly strict. In my understanding, if we can package a Windows binary under some open-source license and distribute it, we should be able to publish on Flathub, too. But IANAL.

If you can't do this submission yourself for whatever reason, let me know, and I'll try to figure out how I can do it in your name.

[^gen]: The purpose of the mwilck/Audiveris-flatpak code is to create a build manifest for building Audiveris offline, as required by flatpak. The code that will eventually be submitted to Flathub is generated by the code in the mwilck/Audiveris-flatpak repo.

hbitteur commented 1 year ago

@mwilck

It's good news. Both for the Audiveris project and for all those who ask for easy installation on Linux.

We can add the fact that, thanks to recent PR from Peter Greth, Audiveris was able to replace JPodRenderer with Apache PDFBox. JPodRenderer still had a dependency on JAI (Java Advanced Imaging) which was not fully open-source. The road is now clear.

Today I went through the Flatpak and Flathub documentations that you pointed out in your post. I also read the 15 files contained in your Flathub fork. Should I say that I was far from understanding every line?

So, I think you, Martin, are the person who should take the lead on this Flatpak/Flathub submission. Indeed, I read that "some applications are distributed on Flathub without the involvement of their developers" and that Flathub "would prefer that these applications be controlled by their authors". But that's not our case. I think all Audiveris contributors, starting with me, would appreciate this submission.

This Audiveris project is open in all directions, all contributions are welcome. Baruch contributed with the Windows installer. This Linux distribution will be your contribution.

I will try to help you as much as I can. For example, I think we should aim for a new version (5.3.2), due to some bug fixes I'd like to include.

We'll get into more detail in the coming days, but I wanted to tell you that you should go for it. Thank you very much for your work. /Hervé

P.S. I need to resurrect my VirtualBox installation to get my hands on the Linux environment again

mwilck commented 11 months ago

Flathub submission created. Let's see how it goes. @hbitteur please have a look and let me know if anything is not as you would like it.

mwilck commented 11 months ago

@hbitteur, have you ever tried running Audiveris on other platforms than x86_64? The flatpak build fails for ARM because of missing tesseract libraries, and I wonder if it's worth the effort to try and make it build. For now I am just trying to disable the ARM build.

hbitteur commented 11 months ago

@mwilck No, I have never tried it myself. For what it's worth, have a look at issue #564

mwilck commented 11 months ago

We made it! Audiveris is now on flathub. :rocket:

I suppose this issue can be closed.

mwilck commented 11 months ago

@hbitteur, perhaps you want to add a note about this in your installation guide.

hbitteur commented 11 months ago

Sure, I need to do this! Let me try to run the installation on my PC.

Thanks a lot!

Altonss commented 11 months ago

Thanks a lot @mwilck ! Once it is in the installation guide, maybe it would be a good thing to verify it on flathub https://docs.flathub.org/docs/for-app-authors/verification

mwilck commented 11 months ago

I haven't double-checked, but I suppose verification can be only done for a single account. I can do this, but I'd leave the decision to @hbitteur whether he wants to link the app to his own account instead, which would increase credibility.

mwilck commented 11 months ago

@Altonss, perhaps you could give the package a test?

mwilck commented 11 months ago

About the verification: the process requires me create a file with an UUID-like token in a file on the audiveris.org web page (https://audiveris.org/.well-known/org.flathub.VerifiedApps.txt). I obviously can't do that myself. So @hbitteur, I'll need your cooperation for creating that file. Perhaps it's actually easier if you do this yourself. You can use github to identify to flathub.

(edited) Well it seems that audiveris.org doesn't currently do https at all. So I guess this needs more patience.

Altonss commented 11 months ago

@Altonss, perhaps you could give the package a test?

Tried it, and so far everything worked fine! Only issue is, by default Export book exports to /home/username/.var/app/org.audiveris.audiveris/data/AudiverisLtd/audiveris/piece/piece.mvt1.mxl, which isn't ideal for the end user I think (it would be better in documents for ex).

Altonss commented 11 months ago

@hbitteur, perhaps you want to add a note about this in your installation guide.

Wrote first little documentation for the flatpak installation https://github.com/Audiveris/audiveris/pull/687.

mwilck commented 11 months ago

Only issue is, by default Export book exports to /home/username/.var/app/org.audiveris.audiveris/data/AudiverisLtd/audiveris/piece/piece.mvt1.mxl, which isn't ideal for the end user I think (it would be better in documents for ex).

I agree that this path isn't ideal. I have kept changes to Audiveris's own source code to an absolute minimum. The only commit affecting it is 1f326d4.

Let me try to clarify my role here: I will will work on keeping the flatpak up to date and improving the code for creating the flatpak package. I don't feel responsible for other issues such as this one, even if they only occur on Linux / in the flatpak package. I will be happy to help with such issues, but I don't know the sources well enough to commit myself to fixing them.

In the concrete case here, I guess we need some indicator to figure out whether the app is being compiled for a) Linux and b) a flatpak environment, and set some compilation defaults accordingly. I'll try to figure something out, but it may take a while. I hope @hbitteur will be willing to take patches to this effect if they are cleanly written to change defaults only for Linux/Flatpak.

That said, I set the permissions of the flatpak such that it has access to $HOME/Documents and $HOME/Pictures. You should be able to use these folders. The awkward paths under $HOME/.var/app are part of the flatpak concept and should be familiar to most flatpak users.

mwilck commented 11 months ago

About the verification: I would need @hbitteur's assistance.

I think we should stop here. Adding more side issues to the "make linux release" topic is confusing. Let's discuss the verification process here, all else should go into separate issues.

hbitteur commented 11 months ago

Hi guys,

I hadn't used VirtualBox since about 10 years... So, I spent half a day fighting with that beast: downloading VirtualBox 7, installing Ubuntu unattended, discovering that my (french) keyboard was stuck to Qwerty, that I could not open a terminal with CTRL+ALT+T... Then a lot of googling to read somewhere that I should NOT use the default unattended way if I wanted to have the opportunity to correctly manage my Azerty keyboard. So I reinstalled everything, this time keyboard was OK and I could open a terminal window. I then installed flatpak, and was able to install audiveris from flathub like a charm.

Now add a couple of hours after dinner to succeed in sharing folders between my Windows host and the Ubuntu guest. And voila, Audiveris now runs correctly.

image

As you can see, I have been able to store a resulting .mxl file. The target is similar to what you obtained.

I'll see tomorrow if and how we can target other folders. This is enough for today.

hbitteur commented 11 months ago

Only issue is, by default Export book exports to /home/username/.var/app/org.audiveris.audiveris/data/AudiverisLtd/audiveris/piece/piece.mvt1.mxl, which isn't ideal for the end user I think (it would be better in documents for ex).

I agree with @mwilck: this has little to do with the main topic of this issue which is to provide a "Linux installer". I'd suggest we open a separate issue or discussion if needed on this question of target folder.

See the specific target folders article and the more general standard folders article in Audiveris handbook.

hbitteur commented 11 months ago

I then installed flatpak, and was able to install audiveris from flathub like a charm.

This is truly a significant success! Thank you all for this long-awaited achievement!

Just like we did with Windows installer, we now need to automate this publishing mechanism. Certainly by defining specific tasks in the Gradle script. We need to work with @mwilck on this.

hbitteur commented 11 months ago

I could not resist... Here below is Flathub web site today. Enjoy!

image

hbitteur commented 11 months ago

Just like we did with Windows installer, we now need to automate this publishing mechanism. Certainly by defining specific tasks in the Gradle script. We need to work with @mwilck on this.

Today, I tried to review and understand all the various pieces that Martin Wilck has worked upon to pave the way from Audiveris to Flathub. My motivation is double:

I first went through all the files, trying to understand their purpose together with the Flathub documentation in the other hand, then I stumbled on Martin's Audiveris-Flatpak README file which helped me a lot. I should have begun with it!

My understanding, please correct me if I am wrong, is that a submission to Flathub is implemented as a "pull request" on a forked Flathub dedicated project. And the PR must provide specifically populated files (a kind of manifest). The question is how to write these files and, even more importantly, how to keep them up-to-date with Audiveris evolutions. So that, for any new Audiveris release, we can easily publish the master source release on GitHub, the Windows installer, the documentation, etc ... and the Linux installer on Flathub.

Could we handle this (the process described in Martin's README file) via the build.gradle file hosted in Audiveris GitHub project? This is what we did for Windows installer, and for the schemas documentation as well (using a dedicated sub-project). Problem: here we need to use tools like flatpak-builder which, to my knowledge, need to be run on Linux.

Could someone infirm or confirm this possibility?

hbitteur commented 11 months ago

Minor suggestions for the Flatpak files:

mwilck commented 10 months ago
  * Instead of "Convert sheet music to **XML**", we should use "Convert sheet music to **MusicXML**". XML term is very broad and tells nothing about the contained information.

ok

  * I don't understand why `lilypond` appears in the keywords.

Well ... because my original, subjective motivation to try out Audiveris had been to create lilypond files from printed sheet music (which requires an additional conversion step). I agree it's kind of misleading. At least other music typesetting programs like MuseScore should be mentioned, too.

I'm happy to change this to whatever you suggest.

* orp.audiveris.audiveris.`metainfo.xml`:

  * `screenshots` element: we should provide better Audiveris images than these old ones.

Yes, and it would be wonderful if we had reliable permalinks to up-to-date screen shots on audiveris.org.

  * `release` element for 5.4: we should remove this element (content is almost void and 5.4 is not released yet) and rewrite it when we are ready to release.

Right, probably. This is one aspect of the flatpak submission / maintenance process I don't fully understand yet. In particular I am not sure if we could submit a 5.4 release to the flathub "beta" channel before it's officially released.

hbitteur commented 10 months ago

At least other music typesetting programs like MuseScore should be mentioned, too.

OK, let's mention Lilypond, Musescore and Finale

Yes, and it would be wonderful if we had reliable permalinks to up-to-date screen shots on audiveris.org.

I will take care of finding better images.

In particular I am not sure if we could submit a 5.4 release to the flathub "beta" channel before it's officially released.

I agree this is not very clear for me either.

In our case, we can consider that this very first submission to Flathub was a kind of beta-testing of our submission data and process.

I would suggest that the next Audiveris submission to Flathub will be tried only when the following conditions are both met:

mwilck commented 10 months ago

Just like we did with Windows installer, we now need to automate this publishing mechanism. Certainly by defining specific tasks in the Gradle script. We need to work with @mwilck on this.

Could we handle this (the process described in Martin's README file) via the build.gradle file hosted in Audiveris GitHub project?

My plan is actually to try to merge my https://github.com/mwilck/Audiveris-flatpak stuff into the Audiveris code base by creating a PR moving all of it into the dev/flatpak subdirectory. Perhaps after fixing some issues that have come up in the flathub review.

Integrating that into the gradle workflow would be the last step for me, because I (being a Linux person and not a Java person) have very little knowledge about gradle. So, help with gradle would be highly appreciated :grin:

This is what we did for Windows installer, and for the schemas documentation as well (using a dedicated sub-project). Problem: here we need to use tools like flatpak-builder which, to my knowledge, need to be run on Linux.

Right. I doubt flatpak-builder will ever run on Windows. But it might be possible to automate this using GitHub workflows, where using a Linux worker is no big issue. That should be the ultimate goal, once we have the manual flatpak creation working.

The current flatpak manifest for audiveris is generated from several parts. There is the (mostly static) .yml.in file, the list of tesseract languages to include, and most importantly, the list of dependencies, which is generated by a script. In the flathub review, I was asked to move this list auto-generated sources into a separate file. I eventually understood how this works, this is the next step I intend to do. It's just a minor change to my scripting.

Note that the "static" part isn't completely static; for example the flatpak runtime version needs to be updated from time to time. Also, as @hbitteur already noted, some information in the appstream data (.metainfo.xml), in particular the changelog, needs to be updated as well when a new release is attempted.

As you have seen, the current process runs the normal gradle build and collects the .jar files and the meta data that gradle has downloaded, creating a source list for flatpak. Because flatpak will download all these sources into a "flat" :grin: directory, we also need the creategradlrepo.sh script that will re-arrange them in a format the offline gradle build used by flatpak-builder can use. I bet my left foot that someone could come up with a gradle recipe to create these two files directly, without this clumsy "run-gradle-and-examine-what-it-did" procedure. (The python script used this stackoverflow response as a starting point).

mwilck commented 10 months ago

I would suggest that the next Audiveris submission to Flathub will be tried only when the following conditions are both met:

I think we should fix the keywords according to your suggestion (altough I don't think Finale supports Linux; thus unsure if it makes much sense mentioning it).

* We have converged between your ad-hoc repositories and Audiveris repository,

* Audiveris 5.4 is officially released on Github (a matter of a couple of months, it is still in alpha status on development branch).

Sounds like a good plan. I am not sure about the full automation yet, but it's at least a worthwhile goal.

hbitteur commented 10 months ago

Right. I doubt flatpak-builder will ever run on Windows. But it might be possible to automate this using GitHub workflows, where using a Linux worker is no big issue.

Very good idea!

Note that the "static" part isn't completely static; for example the flatpak runtime version needs to be updated from time to time.

Yep. I noticed that flathub supports Java LTS versions 8, 11 and 17. But not Java 21 yet, even though it's the latest LTS Java version today. I had the intention to move to Java 21 very soon, at least on the development branch. We may have to wait.

hbitteur commented 10 months ago

I don't think Finale supports Linux

I'm afraid you are right. I just checked on the Internet.

hbitteur commented 10 months ago

@mwilck Hi Martin, I just redesigned the Gradle sub-project that builds the Windows installer. I also did something similar to automate the generation of Audiveris handbook as a PDF.

Now I feel brave enough to start a new Gradle sub-project dedicated to your Linux installer. I will certainly need your help here and there, but let's go! :-) /Hervé

hbitteur commented 10 months ago

@mwilck I would like to get all your items from https://github.com/mwilck/Audiveris-flatpak into a dedicated folder within Audiveris Linux installer sub-project.

I plan to use /linux-installer to host the needed build.gradle file And your material (perhaps modified for Gradle use) would be put into /linux-installer/dev folder

How could we do this? Should I manually copy every item. Or is there a more official way to proceed (I'd like to keep the link to your work, so that you get the credit).

hbitteur commented 10 months ago

@mwilck I have cloned your repository https://github.com/mwilck/Audiveris-flatpak. Now, I'm trying to proceed manually as described in the README.md file, before wrapping the actions as Gradle tasks. I have faced a few problems:

  1. languages.sh uses several "curl -s ..." which fail for some security reason. I modified these calls as "curl -sk ..." and the script now works correctly. The file lang_sources.yml got appended with new languages, however I noticed the insertion of a blank line between the old ones and the new ones. I could not fix this, I simply hope this has no impact.
  2. Running build for dummy.yml, by the command: flatpak-builder --force-clean --disable-cache --keep-build-dirs dummy dummy.yml I confess that I don't understand why we need to build some dummy stuff, perhaps you could explain this. Anyway, I installed flatpak-builder (on my virtual Ubuntu machine), I then ran this dummy build, which failed for lack of freedesktop, which I had to install (the obsolete 21.08 was required, perhaps I could change it for 23.08 in the .yml file). Then I could see the flatpak-builder downloading gradle-7.3-all.zip, before issuing this error message: Failed to download sources: module dummy: Error renaming temporary file: Text file busy

At this point I do need some help. What is this text file? And why is it busy?

I apologize for these manually typed messages, I still can't make any cut and paste working with my Ubuntu virtual machine. Life can get hard sometimes... :-)

mwilck commented 10 months ago

@hbitteur,

sorry I missed these updates because I was sick and there is some confusion with email delivery from github to my various email accounts.

I suppose you've seen my PRs #704 and #705 by now. The dummy.yml stuff is history now, as you may have seen.

languages.sh uses several "curl -s ..." which fail for some security reason

That's strange. Were you using some proxy or the like? Or maybe your Linux installation was missing the standard set of trusted root certificates? The package is typically called ca-certificates.

hbitteur commented 9 months ago

@mwilck Sorry for this late answer. I had to address a couple of bug-fixes. I'm back to the flatpak/flathub topic today.

I have pulled your PR 704 in a local folder of mine to study its content before merging it into the development branch. In particular, I'm trying to understand what you are doing and how this could be more of less automated in Gradle.

I have to say I'd never heard of git submodules before I read your README.md recipe file. So, please be kind with my naive remarks.

Once pulled, to initialize the submodule, the recipe says we have to run:

git submodule init --all

The --all option is not recognized by git. So, I removed it.

I expected to see git-managed items in folders like "dev" and then the resulting (git-ignored) artefacts written somewhere under the "build" folder. This does not seem to be the case. For instance, the dev/flathub folder seems to gather files like the manifest org.audiveris.audiveris.yml file, or the appstream meta data org.audiveris.audiveris.metainfo.xml file with other files that are auto-generated (lang_sources.yml, dependencies.yml and mkgradlerepo.sh). I checked the .gitignore file in the same dev/flathub folder (submodule?). It does not ignore the auto-generated files. Why? In short, I think we should more clearly separate the input files and the output files. Only the input files should be managed by git, and all the output files should be "cleanable" by gradle.

Focusing on the input files of Audiveris project as a whole (that is not only the dev/flathub portion),I think we should avoid any information duplication between them. Here are some obvious examples (as of this writing):

These constants should be defined only once, typically as symbolic constants in the build.gradle file. Then any other input file should refer to the symbol name, not to its value.

For instance, "openjdk17" appears several times in org.audiveris.audiveris.yml file. Is there a way in such a YAML file to include symbolic names? I don't know yet.

mwilck commented 9 months ago

I have to say I'd never heard of git submodules before I read your README.md recipy file. So, please be kind with my naive remarks.

I know, git submodules are somewhat difficult to get used to. But when you have overcome the first parts of the learning curve, they can be very useful. If you're interested, have a look at the chapter in the git book.

I understand very well that if you are uncomfortable with git submodules, you don't want to start messing with them. But' I'd like to ask you to give them a fair chance :smile: You can work just normally with the audiveris repos without checking out the submodule, as long as you don't work on the flatpak.

git submodule init --all

The --all option is not recognized by git. So, I removed it.

My bad, sorry. It's just git submodule init; git submodule update. --all is just for git submodule deinit, which removes the contents of the submodule directory.

I checked the .gitignore file in the same dev/flathub folder (submodule?). It does not ignore the auto-generated files. Why?

Because for flathub, these files are part of the submission. We must have them under version control (a flathub PR is nothing but a github PR, so all files that are part of the flathub submission must be in git). It doesn't matter that they are autogenerated from the audiveris code.

IOW, these file are ignored in the audiveris git repo, but not in the flathub repo. git is smart enough to understand that and not show them when you work in the main repo.

In short, I think we should more clearly separate the input files and the output files. Only the input files should be managed by git, and all the output files should be "cleanable" by gradle.

I can do it like this if you prefer. We could have dev/flathub (or some other path that you might want) simply as an output directory under audiveris version control, git-ignored. But it has disadvantages compared with the submodule approach.

  1. The autogenerated files need to be copied to a checked-out flathub/audiveris repository (which wouldn't be the same in every work environment) and added and commited there before pushing / submitting to flathub. By using the submodule, the output path is fixed, and the copying is unnecessary. The autogenerated files are placed into the submodule directory directly, and a quick git -C dev/flathub status shows us if anything has changed and needs to be committed.
  2. It's less obvious how to handle the non-autogenerated flathub files like org.audiveris.audiveris.yml and org.audiveris.audiveris.metainfo.xml. They are only strictly required for flathub, but it would be kind of awkward not to have them in the audiveris repo. I guess with the discussion below about using variable substitution, this argument is resolved; we'd keep the template in the audiveris repo and the generated output in the flathub repo.

Anyway, maybe we should move this discussion into #704 ?

... These constants should be defined only once, typically as symbolic constants in the build.gradle file. Then any other input file should refer to the symbol name, not to its value.

Yes of course. Would it be a problem for gradle to simply update the files in the dev/flathub submodule directory, using some sort of text substitution?

For instance, "openjdk17" appears several times in org.audiveris.audiveris.yml file. Is there a way in such a YAML file to include symbolic names? I don't know yet.

In this specific case, I think the answer is "no". YAML-based languages support variables, like GitHub actions with it's ${{ variable }} syntax, but it isn't standardized, and YAML itself doesn't support it. Flatpak's YAML is just plain YAML, unless I am mistaken.

We need to create a template and generate the .yml file from it. As a Unix guy, I'd think of some sed script, but I guess there are equivalent and probably more powerful means available in the gradle ecosystem.