[idea] Publish a flatpak on Flathub

denilsonsa commented 2 years ago

Flatpak is a distribution-agnostic package system where the application is shipped with all the dependencies and runs inside some kind of sandbox. It's relatively new, and is gaining quite a lot of popularity.

Additionally, Valve's Steam Deck supports Flathub out-of-the-box. In fact, its main OS is mounted read-only, so people must either use flatpak or manually install it themselves in their home directory. The easiest way to run anything on Steam Deck is through Flathub. (Well, the easiest is actually through Steam store itself.)

ssokolow commented 2 years ago

I'd be willing to write up the files necessary to submit it if that'd get the ball rolling.

I don't have time to take on full-blown maintainership, but Flathub does have a build bot which can automatically generate ready-to-approve PRs and run test builds when new releases are made or dependencies need to be upgraded.

joeraz commented 2 years ago

As the one who currently handles the main releases, I don't mind adding another release script to update a flathub package when I do a main release, to keep it up to date.

Unfortunately, I do not know much about flathub or flatpak, so I would not be able to create the initial scripts to generate the packages. But if someone is able to provide me a script and instructions, I can ensure that the package stays up-to-date.

ssokolow commented 2 years ago

OK. Assuming nothing unexpected comes up, I'll give it a try tomorrow.

Do you already have an AppStream XML (Distro-agnostic catalogue entry definition for things like GNOME Software and KDE Discover, which Flathub also uses for its web UI) that you want me to use or should I just make one of those as part of the process?

(For the record, the Flatpak build manifests try to be as declarative as possible, so it's closer to what your .travis.yml looks like than a script. For example, to build a module that that uses a common build system they explicitly support like autotools, cmake, meson, etc., you just specify the build system and they do the rest.)

EDIT: Speaking of which, I can do it in YAML, but do you mind if I use JSON? The degree to which whitespace is significant in complex mixes of lists and dicts in YAML makes me feel very nervous and I'm not a fan of how many footguns it has around optionally quoted strings. Both big maintainership gotchas.

joeraz commented 2 years ago

No appstream XML - if I remember correctly, most of the information that goes in it is stored in an object inside the setup.py file, if that helps any.

JSON is fine with me.

ssokolow commented 2 years ago

I got the OK to submit another game this morning and I've already learned that there are some flaws in the instructions I was following, so I think I'll wait until that submission process has finished before I offer anything up for PySolFC to avoid needlessly making the same mistakes twice.

EDIT: Done. I'll try to find time for this tomorrow.

ssokolow commented 2 years ago

OK, I've been familiarizing myself with the build process. Here's what I have so far:

I haven't yet gone source diving to investigate, but the docs don't seem to cover installing the cardsets in a system location rather than under $HOME. If that's more than just a docs oversight, I may need to grab a patch from a distro build.
I've decided to try the Kivy version first. If it's too alien to what I'm used to as a PySolFC user, I may switch to working on the Tkinter version (I'm stubborn that way), but I imagine the Kivy version is probably more ready to support native Wayland operation. (Plus, pysol-sound-server's makefile ignores the install prefix flatpak-builder sets and then dies with a "read-only filesystem" error, I'd prefer not to patch it if I can avoid it, and PyGame's pile of dependencies is overkill.)
According to the build instructions for the solvers, I'm going to need to google up the recommended way to add things to CMake's search path under flatpak-builder.
While I haven't tried setting up PySolFC itself yet due to that cardsets thing, I have module definitions for tkinter, Pillow, ttkthemes, Kivy, and PySolFC-Cardsets which finish without failing the build.

joeraz commented 2 years ago

This info might help:

The way cardsets are handled differs between distro builds, but the simplest way to add them is to simply copy the contents of the cardsets package (either the full one or the minimal one) from SourceForge to a data/cardsets folder.
The Kivy version is largely intended for Android use, though it can be run in other environments. Because of this, its interface is quite a bit different.
Pysol Sound Server hasn't been updated in a while (don't think I've ever touched it personally), largely because PyGame supports more types of audio files and that's what was configured in the other official releases.
The pillow and ttkthemes libraries are both optional libraries. Pillow is used for advanced graphics options, and if it's not included, many graphics options/features will be disabled - though I might want to make Pillow a required library in the future, as it's getting more difficult to work around. Ttkthemes just adds additional tkinter theme options to the options/set theme option - omitting this library just means there will be fewer options in that menu.

ssokolow commented 2 years ago

Thanks. I'll switch back to the Tkinter version then.

Pillow and ttkthemes were trivial to get building. A simple python3 flatpak-pip-generator ttkthemes generated a build plan fragment that pulls them both in and doesn't error out. (Though, as I said, since I haven't tried building PySolFC itself yet, there's always the possibility of subtle failures.)

ssokolow commented 2 years ago

Ugh. A compiler update in the Freedesktop SDK 22.08 broke the ready-made build plan for PyGame.

Given that one of the failures is in Fluidsynth, does PySolFC need MIDI support or can I try pref-ing it off?

Also, I just noticed that the instructions don't say anything about what I'm supposed to do with pysol-music-4.50.tar.xz.

ssokolow commented 2 years ago

Also, I'm continuing to look through the instructions and there's a lot of inconsistency I'm trying to reconcile:

Why do the Mageia instructions say to install PyGTK but not Perl while the Debian/Ubuntu instructions say to install Perl but not PyGTK?
Why do the "from source without installation" instructions make no mention of pip installing six, random2, and pysol-cards? I can understand six being present by default, but the other two?
Why is ln -s data/images images necessary?
Where Makefile and ./contrib/install-pysolfc.sh differ, which one should I treat as authoritative?
Why do the instructions use gmake without mention that GNU Make is only present as make on *buntu-family distros? Were they written for a BSD?

EDIT: And, judging by setup.py, there are also un-mentioned dependencies on attrs,configobj, and pycotap.

ssokolow commented 2 years ago

OK, I decided to set sound aside initially and just get PySolFC working as-is first.

I'm to the point where I've borrowed the pip3 install command preferred by flatpak-pip-generator to run setup.py and that completed successfully, but I'm getting this error now.

Traceback (most recent call last):
  File "/app/bin/pysol.py", line 41, in <module>
    sys.exit(main(sys.argv))
  File "/app/lib/python3.10/site-packages/pysollib/main.py", line 437, in main
    r = pysol_init(app, args)
  File "/app/lib/python3.10/site-packages/pysollib/main.py", line 199, in pysol_init
    app.dataloader = DataLoader(args[0], f)
  File "/app/lib/python3.10/site-packages/pysollib/util.py", line 146, in __init__
    raise OSError(str(argv0)+": DataLoader could not find " +
OSError: /app/bin/pysol.py: DataLoader could not find ('html/license.html',)

I'll look into why it's not finding /app/share/PySolFC/html/license.html after I've slept.

joeraz commented 2 years ago

Okay, a lot to answer here. First, I did not write most of those documents, I just took over the release process after the previous developers left - I do agree they could use some revisions, I just haven't gotten around to that yet. But I can't really answer as to the original intentions of the people who wrote them, or which distributions they were using. But this info might help:

The current music is in s3m, it, and mod formats, and the sound samples are all wav files, so no midi files are currently used. However, the support for midi may still be used by some users, as users can drop audio files in the data/music folder to add their own soundtracks, in addition to the PySol music repo.
The html/license.html file is used by Pysol to identify the data directory. It being missing can complicate access to the data directory - I believe linking the images directory in the data folder is intended as a failsafe in this case. It's probably not needed under most circumstances (my install isn't set up with that link). As for the license.html file itself, can you confirm if you ran (g)make rules? That should generate the license.html as part of the process.
When I push new releases, I follow the instructions described in Contributing.md to generate the new package. As this makes use of the makefile over the other scripts, I consider the makefile the more definitive source.

ssokolow commented 2 years ago

I'm not asleep yet, so I'll reply now.

The current music is in s3m, it, and mod formats, and the sound samples are all wav files, so no midi files are currently used. However, the support for midi may still be used by some users, as users can drop audio files in the data/music folder to add their own soundtracks, in addition to the PySol music repo.

Noted. I'll see if following the deprecation message suggestion to replace G_STATIC_MUTEX_INIT with g_mutex_init solves the "extra brace group at end of initializer" error that's killing the FluidSynth build under the version of GCC used for the 22.08 Freedesktop runtime.

The html/license.html file is used by Pysol to identify the data directory. It being missing can complicate access to the data directory - I believe linking the images directory in the data folder is intended as a failsafe in this case. It's probably not needed under most circumstances (my install isn't set up with that link). As for the license.html file itself, can you confirm if you ran (g)make rules? That should generate the license.html as part of the process.

Yes, I did. /app/share/PySolFC/html/license.html exists... PySolFC just didn't find it. I was planning to reverse-engineer the relevant bits of DataLoader after I've slept to make sense of what it was intended to be doing and why it was failing.

When I push new releases, I follow the instructions described in Contributing.md to generate the new package. As this makes use of the makefile over the other scripts, I consider the makefile the more definitive source.

OK. Aside from the mo task and make rules, it looks like it's just handing off to setup.py. So far, I've been using a make rules followed by a pip3 install command adapted from what flatpak-pip-generator generates, since that handles the whole /app vs. /usr thing for me.

ssokolow commented 2 years ago

I think I found the problem.

It appears that DataLoader uses this line to add /usr/... to the search path...

path.extend(DATA_DIRS)

...but DATA_DIRS is hard-coded to use /usr rather than the --prefix passed to setup.py.

# data dirs
DATA_DIRS = []
# you can add your extra directories here
if os.name == 'posix':
    DATA_DIRS = [
        '/usr/share/PySolFC',
        '/usr/local/share/PySolFC',
        '/usr/share/games/PySolFC',
        '/usr/local/share/games/PySolFC',
        ]
if os.name == 'nt':
    pass

Luckily, rg -tpy '[^! ]/usr' says that should be the only place that hard-codes /usr outside of install scripting I'm not using like setup_osx.py.

The quick fix would be to add /app/share/PySolFC to the list, but I'll look into a proper fix that respects --prefix.

ssokolow commented 2 years ago

I decided to try patching in /app/share/PySolFC to make sure that a test with a proper solution would directly result in a working install (change just one variable at a time), and it does... though it also prompted me to file bug #273.

ssokolow commented 2 years ago

Hmm. It's actually GLib 2.70 that's breaking FluidSynth and it's doing so because it's an unmaintained FluidSynth 1.x that, according to one bug report I read, also has unpatched vulnerabilities.

Given that PyGame 1.9.6 itself also has a build failure I'd have to diagnose and patch, and that Ubuntu is moving from PyGame 1.x to 2.x as of Jammy, I think the only fix is going to be porting PySol's audio support to PyGame 2.x, trying to use PySol-Sound-Server for now, or disabling audio.

I tried building PySol-Sound-Server against sdl12-compat (the compatibility wrapper which allows SDL 1.2.x programs to build against SDL 2.x) and it did build and install once I bypassed the makefile which was discarding the --prefix=/app flatpak-builder was sending it but you were right about it being effectively abandoned.

PySol failed to talk to it with this error:

Traceback (most recent call last):
  File "/app/lib/python3.10/site-packages/pysollib/pysolaudio.py", line 70, in connectServer
    if self._connectServer():
  File "/app/lib/python3.10/site-packages/pysollib/pysolaudio.py", line 197, in _connectServer
    r = self.cmd("protocol 6")
  File "/app/lib/python3.10/site-packages/pysollib/pysolaudio.py", line 193, in cmd
    return self.audiodev.cmd(cmd)
SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

(Something to investigate. If that happens on more than just Flatpak's build of Python 3.10, then it meets the Linux kernel's "support for it broke and nobody noticed" standard for deleting dead code.)

Even if my goal weren't to "do it properly" with this Flatpak build to make it easy to maintain going forward, I'm hitting a wall of technical debt that needs to be paid down.

That said, I have a working-but-silent Flatpak build of PySol that should be submittable as soon as the Appstream XML file is done, so at least there's that.

Speaking of which...

How many of the release descriptions, either on https://pysolfc.sourceforge.io/ or elsewhere, are released under a license compatible with one of the licenses on this list? I need to know what I can add to the Appstream XML file.
I've already defined homepage, bugtracker, help, vcs-browser, and contribute links in the Appstream XML. Is there anything you'd like me to put for the faq, donation, translate, or contact link types?
Given how much information can be stored in Appstream XML, and how complicated your build process seems, if I'm still feeling motivated once this is all done, would you be interested in me whipping up a Python script to try to reduce the amount of duplicated information and the number of manual steps in your release process?

Also, your existing screenshots don't meet the guidelines (eg. "Screenshots should be taken with English as the display language.", "Screenshots should not show anything outside the application's windows") and I'd prefer to specifically screenshot the theme that the Flatpak release will default to, as well as showing a wider variety of supported games.

(I was thinking Klondike, Clock, Bits n Bytes, a photogenic Hanafuda game, Mahjongg Taipei, and Shisen-Sho 14x6 with a variety of table tiles and cardback/tileset settings... though I will admit that's because, aside from Hanafuda games, those are all games that are familiar to me.)

As the Appstream XML requires screenshots to be specified as publicly available URLs that software repositories are then responsible for downloading and caching, I'll either need you to add them to some PySolFC-related hosting once you approve them, or maybe they could be hosted in the repository Flathub will create for you once the submission is approved.

I suppose the next thing I could do without taking it upon myself to modernize PySol's sound support would be to add the solvers to the Flatpak manifest. Then, maybe go back and implement proper lookup for --prefix in pysollib.settings.DATA_DIRS.

joeraz commented 2 years ago

Okay.

I will look into PyGame 2.0 to see how difficult it would be to upgrade.
I wrote the release descriptions for the versions I published, which would be everything from 2.12.0 onward. As such, I can re-license those under FSFAP for this purpose. As I did not write the older release notes, they would be under the GPL v3 like the rest of the website/app code. You'd have to contact shlomif and skomoroh if you'd them under a different license.
Anything that can simplify the release process would be greatly appreciated.
Getting new screenshots is on my to-do list. In recent versions, I've enhanced PySolFC's support for higher resolution cardsets and display in larger screens, so I've been wanting to get new screenshots that show this off. I was just waiting a couple releases in case there were any major bugs. But if you have screenshots to offer, I can add them to the website and SourceForge as part of the upcoming 2.18.0 release.

joeraz commented 2 years ago

I just checked. PySolFC works fine with PyGame 2.0.1.

ssokolow commented 2 years ago

I just checked. PySolFC works fine with PyGame 2.0.1.

OK. I'll start work on adding PyGame 2.x to the Flatpak manifest

As such, I can re-license those under FSFAP for this purpose.

Give me a sec to read up on the FSFAP. Aside from seeing it in that list, I'd never heard of it before and I used MIT for the last Flatpak manifest I wrote.

EDIT: Ahh. It's to the BSD0 as the BSD0 is to other permissive licenses. Yeah, I have no problem with that.

As I did not write the older release notes, they would be under the GPL v3 like the rest of the website/app code. You'd have to contact shlomif and skomoroh if you'd them under a different license.

I'll have to either omit them or contact shlomif and skomoroh. GPLv3 isn't on the list of licenses the Appstream XML validator accepts because, if something is included in the Appstream XML, it must be permissively licensed so it can be freely embedded in "app store"-style interfaces without complicating the legal situation.

Getting new screenshots is on my to-do list. In recent versions, I've enhanced PySolFC's support for higher resolution cardsets and display in larger screens, so I've been wanting to get new screenshots that show this off. I was just waiting a couple releases in case there were any major bugs. But if you have screenshots to offer, I can add them to the website and SourceForge as part of the upcoming 2.18.0 release.

I'll take some once I've finished getting the Flatpak build up and running, so they can be as representative as possible.

ssokolow commented 2 years ago

EDIT: Ahh. It's to the BSD0 as the BSD0 is to other permissive licenses. Yeah, I have no problem with that.

OK, I've incorporated the changes from the website for 2.12.0 and beyond. Since this is intended for a distribution channel where the cardsets are bundled in with the base game, I've tweaked any mention of the cardsets so it doesn't imply that the user should go searching for a separate download.

I also flattened the outline list in the v2.15.0 release notes because the Appstream XML spec explicitly says "Do not assume the format is HTML. This list contains all currently supported formatting options:" and "Nested lists are not supported".

I'll take some once I've finished getting the Flatpak build up and running, so they can be as representative as possible.

On second thought. I'm close enough. How do these look? (I took the liberty of using wmctrl to ensure identical 1280x960 window dimensions every time, despite how this new version of PySol is so obsessed with changing its window dimensions every time I change settings like card set, table background, and game type that I'm probably going to set a KWin rule to lock it to the dimensions I prefer.)

EDIT: Uploaded newer copies with optimized compression

(If you add them to the website, please use PNG. I don't like working with WebP, it's not guaranteed to be universally supported yet, and they'll be converted to PNG anyway by consumers that do support ingesting WebP.)

denilsonsa commented 2 years ago

SystemError: PY_SSIZE_T_CLEAN macro must be defined for '#' formats

I'm probably late to the party, but I've seen this bug in other Python applications on Arch Linux / Manjaro Linux. It's related to using the Python C API to compile some C code to be executed within the Python application. This "error" was introduced in Python 3.10.

denilsonsa commented 2 years ago

I took the liberty of using wmctrl to ensure identical 1280x960

If you think it is worth doing, maybe you could wrap up a script to automatically take PySolFC screenshots. Maybe a setting and a command-line parameter to prevent auto-resizing and sticking to a specific window size. But I'm feeling this will likely be more trouble than needed.

...but DATA_DIRS is hard-coded to use /usr rather than the --prefix passed to setup.py.

I agree the proper fix should be to respect the --prefix, and if possible this proper fix should be implemented.

Another less-hacky-than-hardcoding-yet-another-path solution would be to add some relative paths. So, the code could try to find where it is itself located, and then look at ../foobar or ../../share/foobar or similar.

ssokolow commented 2 years ago

I'm probably late to the party, but I've seen this bug in other Python applications on Arch Linux / Manjaro Linux. It's related to using the Python C API to compile some C code to be executed within the Python application. This "error" was introduced in Python 3.10.

Read more about it: https://bugs.archlinux.org/task/73372#comment210456

I just got PyGame2 building when you'd posted that, so I've got sound without PySol-Sound-Server... but not music.

I configured flatpak-builder to build a replacement for the libSDL2_mixer.so included in the Freedesktop runtime that has libmikmod enabled, but still no music, so, when I'm not so tired, I'll need to diagnose why. (eg. is the library path causing it to ignore the one I built in favour of the one provided by the Freedesktop Runtime? Did I make a mistake while installing the music? etc.)

If you think it is worth doing, maybe you could wrap up a script to automatically take PySolFC screenshots. Maybe a setting and a command-line parameter to prevent auto-resizing and sticking to a specific window size. But I'm feeling this will likely be more trouble than needed.

Probably quicker and "good enough" to just have a Python script which copies a PySolFC config file into place, starts it, abuses wmctrl to poll for the main window having appeared, repositions it, waits a couple of seconds to ensure PySolFC had time to rescale the cards, takes a screenshot, and then kills it and iterates onto the next stored config file to take another screenshot.

On a related note, if anyone wants to try automating testing the validity of the suggested package manager dependency install commands in the README, Vagrant is probably the simplest way to automate downloading and starting a VM of a given version of a given distro, running a few commands, and then leaving the window open for a human to judge.

For example, since the Rust cross tool's support for producing FreeBSD binaries from Linux for my shared web host varies from poor to nonexistent, this Vagrantfile is all I needed to spin up a FreeBSD box to do a non-cross compile in and copy in a file to work around cargo build and Vagrant's shared folders implementation not quite playing nicely together.

Vagrant.configure("2") do |config|
  config.vm.box = "freebsd/FreeBSD-12.3-RELEASE"
  config.vm.box_version = "2021.12.02"

  # Downloading the FreeBSD package updates on `vagrant up` takes time
  config.vm.boot_timeout = 600

  config.vm.provider "virtualbox" do |vb|
    # Display the VirtualBox GUI when booting the machine
    vb.gui = true

    # Customize the amount of memory on the VM:
    vb.memory = "2048"
  end

  config.vm.provision "shell", inline: <<-SHELL
    pkg install -y lang/rust
    cp /vagrant/vagrant_share_workaround.sh $HOME
    chmod +x $HOME/vagrant_share_workaround.sh
  SHELL
end

vagrant_share_workaround.sh just contains this:

#!/bin/sh

mkdir -p $HOME/proj
cd $HOME/proj
rsync -rav /vagrant/Cargo.* /vagrant/src .
cargo build --release
rsync -rav $HOME/proj/target/release/spam_prefilter /vagrant/
chmod +x /vagrant/spam_prefilter

(/vagrant/ is automatically mapped to the folder containing the Vagrantfile on the host system, so that means that provisioning is as simple as running vagrant up after the git clone and making builds is just ./vagrant_share_workaround.sh.)

I agree the proper fix should be to respect the --prefix, and if possible this proper fix should be implemented.

Another less-hacky-than-hardcoding-yet-another-path solution would be to add some relative paths. So, the code could try to find where it is itself located, and then look at ../foobar or ../../share/foobar or similar.

I asked in the Python IRC channel and, apparently, picking apart the path in __file__ to derive the install prefix is still the best way to do it.

Maybe PySolFC can grow a "check paths and then exit with a return code indicating success or failure" option that could be used to test that such code is working properly on a CI server.

ssokolow commented 2 years ago

Probably quicker and "good enough" to just have a Python script which copies a PySolFC config file into place, starts it, abuses wmctrl to poll for the main window having appeared, repositions it, waits a couple of seconds to ensure PySolFC had time to rescale the cards, takes a screenshot, and then kills it and iterates onto the next stored config file to take another screenshot.

...and I got nerd-sniped, so here's a proof of concept for actually taking the screenshots. I'll leave it to you to save some PySolFC configuration files which remember different "game/card set/card back/table tile" combinations as the last thing you played and plumb in another argument so you can start each PySolFC with a different HOME via subprocess.Popen's env argument to point it at each one in turn without having to copy files around and risk losing your original config.

https://gist.github.com/ssokolow/8f4062cf39d2accbae20b5cc1c9b9621

It currently hard-codes an assumption about where on my desktop is the best place to put the window, but that could probably be factored out easily by using the Xephyr commands I shared in #273 (or, to go fully headless so it could even be done in a GitHub Action, Xvfb) to take the screenshots on a desktop of predictable size that it doesn't have to share with any other applications. Just remember to launch a window manager like kwin_x11 or you won't have titlebars and borders.

Yeah, it's a little hackier than I'd like, personally, but I don't want to risk burning out my interest in this learning how to add a "take screenshot" command-line option to PySolFC itself when I haven't even finished packaging it for Flatpak yet, let alone cleaning up the build instructions if I still have the motivation for that.

joeraz commented 2 years ago

Is this main window resizing you describe happening when Auto-Scaling is enabled, or is this just referring to #273? If so, that's a bug, though not one I've been able to reproduce. When auto-scaling is enabled, the window should remain a constant size, but if it's disabled, it should resize as you change game or cardset (of course, if the window's maximized, it should remain that way regardless).

Those screenshots work. I'll add them to the main site with the next release, thanks.

ssokolow commented 2 years ago

Status update: I found why it's not playing music in the Flatpak build using a minimal "play one of PySolFC's music files using PyGame" script.

pygame.error: Failed loading libmodplug.a: libmodplug.a: cannot open shared object file: No such file or directory

I didn't even know you might not want to put *.a in the list of unnecessary files to be cleaned from a Flatpak build.
Why the heck does PyGame want the .a file rather than/in addition to the .so?

I rebuilt with modplug.a still exhibited the same problem, so I used flatpak run --devel --command=sh to mount strace into the sandbox and give myself a shell to poke at.

It looks like I'm going to have to go digging into PyGame to figure out how to tell it to look at /app/lib/libmodplug.a rather than /usr/lib/x86_64-linux-gnu/libmodplug.a.

Is this main window resizing you describe happening when Auto-Scaling is enabled, or is this just referring to https://github.com/shlomif/PySolFC/issues/273? If so, that's a bug, though not one I've been able to reproduce. When auto-scaling is enabled, the window should remain a constant size, but if it's disabled, it should resize as you change game or cardset (of course, if the window's maximized, it should remain that way regardless).

With auto-scaling enabled.

It appears to be discarding my preferred dimensions and switching to some PySol-provided default whenever changing games requires a change in cardset. (Which prompts the main window in my dimensions to be hidden, the progress dialog to be re-displayed, and then the main window re-shown at PySol-specified dimensions.)

Speaking of which, is there a reason it uses that progress dialog rather than launching the main window ASAP and then displaying a progress indication in the playfield? (Maybe cache the table tile used last time so it can be loaded and displayed behind the progress indication while the rest are being enumerated?)

Those screenshots work. I'll add them to the main site with the next release, thanks.

I'm going to need them at a public URL for the Appstream XML to reference before I can finish the Flathub stuff. Would it be possible to upload them sooner but not use them in the site until the next release?

ssokolow commented 2 years ago

With auto-scaling enabled.

It appears to be discarding my preferred dimensions and switching to some PySol-provided default whenever changing games requires a change in cardset. (Which prompts the main window in my dimensions to be hidden, the progress dialog to be re-displayed, and then the main window re-shown at PySol-specified dimensions.)

Correction: It doesn't do it every time. Just when it needs to load a new cardset, so switching from Klondike to Bits n Bytes and then back to Klondike will only discard and redisplay the main window on the first switch, but then switching to Shisen-Sho after that will discard it again.

It also may be that it's trying to preserve the correct dimensions but getting forced by the WM to go smaller when its "discard, progress, redisplay" dance forces the window to jump to a different monitor with a usable (i.e. non-taskbar) region smaller than its former size on the larger monitor.

(As soon as you're destroying and creating windows, you subject yourself to things like "force/advise new windows to appear on the monitor containing the cursor/previous active window" and other stuff like that.)

ssokolow commented 2 years ago

Also, I got a response from shlomif on the release notes.

hi Stephan!

All my changes to PySol FC [or similar copyleft/etc. codebases] are under CC0/MITL/etc.:

https://github.com/shlomif/shlomif-computer-settings/blob/master/shlomif-settings/git/commit-messages/cc0-copyright-disclaimer.txt

Now I just need to make time to do some git blame-ing or similar and figure out which ones are his. (Or figure out how to contact skomoroh and get permission first, so it doesn't matter which are which.)

ssokolow commented 2 years ago

Given the amount of work I've done, and since it does produce a working build (just one without music or solvers), I decided to make public what I've got so far.

https://github.com/ssokolow/PySolFC_flatpak

(I'll also be busy much of tomorrow, so I might as well give people something to look at in advance.)

joeraz commented 2 years ago

I uploaded the screenshots to the site. They should be accessible in the same directory as the others as PySolFC_2.18_1.png through PySolFC_2.18_6.png. Will that do?

For the other release notes, if I remember correctly, shlomif did everything from 2.1 through 2.10. Skomoroh did the earlier versions, and I did the later ones.

Regarding the window sizing, each game in PySolFC does have a pre-defined size, which determines the minimum amount of space needed to display the layout for that game, leaving a reasonable amount of room for longer card stacks. The card size naturally plays into that formula. If auto-scaling is disabled, the window can automatically resize to that size when changing game or cardset. Auto-scaling being enabled would override this logic, but you're right that there could be complications with the windows being recreated.

Will review the repo, thanks.

ssokolow commented 2 years ago

I uploaded the screenshots to the site. They should be accessible in the same directory as the others as PySolFC_2.18_1.png through PySolFC_2.18_6.png. Will that do?

Should do just fine. I've pushed an update to the Appstream XML file that now passes validation.

For the other release notes, if I remember correctly, shlomif did everything from 2.1 through 2.10. Skomoroh did the earlier versions, and I did the later ones.

Looks about right in my quick browse with tig. I'll double-check that 2.1 should be included in that span and then add the release notes to the XML.

ssokolow commented 2 years ago

Music is now working (I had to set a CMake flag to get libmodplug's build process to generate a .so file and add a wrapper script to set LD_LIBRARY_PATH so SDL_mixer can find it under /app/lib).

I've pushed the changes to ssokolow/PySolFC_flatpak, and I've filed a bug on SDL_mixer over its build scripting mis-identifying libmodplug.a as a valid .so file.

Next up, the solvers.

EDIT: Actually, probably next up is getting shlomif's release notes copied into the Appstream XML file. I'm not feeling especially motivated to get shlomif/shlomif-cmake-modules integrated into the Flatpak build at this point in my day, but copy-pasting and adjusting the markup on some release notes is something I don't mind.

ssokolow commented 2 years ago

OK. shlomif's release notes have been added to the XML and the changes have been pushed.

ssokolow commented 2 years ago

...and, fool that I was, I decided to try it today because I thought I was in the home stretch for packaging up the solvers after noticing a variable name that'd let me tell CMake where to look for shlomif/shlomif-cmake-modules.

...I spent nearly two hours learning the hard way that despite it being named CMAKE_MODULE_PATH, it's a -D variable, not an environment variable, and then another 15 minutes staring at its source to discover that I'm going to have to patch it because it hard-codes copying itself from /usr/share (which is read-only and can't be installed into) to the temporary build location of each package that uses it for some unknown reason.

*sigh I've never even used* the solvers. I just want to give users a good, future-proof first impression.

joeraz commented 2 years ago

I haven't dealt much with the solvers myself, so I can't help much with that. But shlomif has worked with them quite a bit.

ssokolow commented 2 years ago

Sorry for going silent. After how frustrated I felt with CMake, I needed some time refactoring a Rust project to de-stress. I've filed shlomif/shlomif-cmake-modules#1 and, if he doesn't make the effort unnecessary within the next day or two, I'll write a patch to switch the hard-coded /usr to /app to get back to making progress.

shlomif commented 2 years ago

@ssokolow : Shlomif_Common.cmake shud be part of the source tarballs

ssokolow commented 2 years ago

I prefer to work from the GitHub repositories. Working from source tarballs makes me feel that the configuration for auto-detecting new releases will be more fragile.

shlomif commented 2 years ago

I prefer to work from the GitHub repositories. Working from source tarballs makes me feel that the configuration for auto-detecting new releases will be more fragile.

Hi.

building fc-solve from git is less supported. you reap what you sow - https://twitter.com/shlomif/status/1330457762191511554

ssokolow commented 2 years ago

I prefer to work from the GitHub repositories. Working from source tarballs makes me feel that the configuration for auto-detecting new releases will be more fragile.

Hi.

building fc-solve from git is less supported. you reap what you sow - https://twitter.com/shlomif/status/1330457762191511554

It also serves as a familiar-to-people-other-than-shlomif form of documentation for how the binaries relate to the in-repo upstream source when someone wants to contribute a patch upstream.

(As opposed to making the patch against the release tarball and letting you figure it out because it's too much work to figure out how to make a test build against the upstream source.)

shlomif commented 2 years ago

hi,

On Sat, 24 Sep 2022 01:05:28 -0700 Stephan Sokolow @.***> wrote:

I prefer to work from the GitHub repositories. Working from source tarballs makes me feel that the configuration for auto-detecting new releases will be more fragile.

Hi.

building fc-solve from git is less supported. you reap what you sow - https://twitter.com/shlomif/status/1330457762191511554

It also serves as a familiar-to-people-other-than-shlomif form of documentation for how the binaries relate to the in-repo upstream source when someone wants to contribute a patch upstream.

(As opposed to making the patch against the release tarball and letting you figure it out because it's too much work to figure out how to make a test build against the upstream source.)

this is one reason we have CI (eg : gh actions)

--

Shlomi Fish https://www.shlomifish.org/ Original Riddles - https://www.shlomifish.org/puzzles/

Tech needs less wizards, ninjas, and rockstars, and way more sociologists. — https://twitter.com/nslater/status/545592700289155072

Please reply to list if it's a mailing list post - https://shlom.in/reply .

ssokolow commented 2 years ago

I don't know about you, but I would never contribute to something where I couldn't vet my contribution locally before exposing it to public scrutiny, CI or not.

It's a matter of reputation if nothing else. It would make me look irresponsible and/or lazy to submit something without being sure that it builds against what i'm submitting to on my system.

shlomif commented 2 years ago

On Sat, 24 Sep 2022 01:05:28 -0700 Stephan Sokolow @.***> wrote:

I prefer to work from the GitHub repositories. Working from source tarballs makes me feel that the configuration for auto-detecting new releases will be more fragile.

Hi.

building fc-solve from git is less supported. you reap what you sow - https://twitter.com/shlomif/status/1330457762191511554

It also serves as a familiar-to-people-other-than-shlomif form of documentation for how the binaries relate to the in-repo upstream source when someone wants to contribute a patch upstream.

(As opposed to making the patch against the release tarball and letting you figure it out because it's too much work to figure out how to make a test build against the upstream source.)

I can rework / forwardport patches against the source tarballs.

--

Shlomi Fish https://www.shlomifish.org/ Funny Anti-Terrorism Story - https://shlom.in/enemy

Chuck Norris wrote a complete Perl 6 implementation in a day, but then destroyed all evidence with his bare hands, so no‐one will know his secrets. — https://www.shlomifish.org/humour/bits/facts/Chuck-Norris/

Please reply to list if it's a mailing list post - https://shlom.in/reply .

ssokolow commented 2 years ago

OK then. Do you have something like an RSS feed, JSON file, or "redirect to newest version" URL I can point flatpak-external-data-checker at so it can automatically open "bump dependency version" PRs (and, in the process, trigger test builds) when you publish updated versions?

I don't feel comfortable using the HTML scraper mode outside of stuff I trust to be sufficiently regular and machine-generated and I'd like to have that functionality.

shlomif commented 2 years ago

hi.

On Sat, 24 Sep 2022 22:52:00 -0700 Stephan Sokolow @.***> wrote:

OK, then. Do you have something like an RSS feed, JSON file, or "redirect to newest version" URL I can point flatpak-external-data-checker at so it can automatically open "bump dependency version" PRs (and, in the process, trigger test builds) when you publish updated versions?

I don't feel comfortable using the HTML scraper mode outside of stuff I trust to be sufficiently regular and machine-generated and I'd like to have that functionality.

https://fc-solve.shlomifish.org/download.html is valid XHTML5, and there r also the git tags and the fedora rawhide package.

--

Shlomi Fish https://www.shlomifish.org/ https://www.shlomifish.org/humour/Summerschool-at-the-NSA/

There are only two hard things in Computer Science: cache invalidation and naming things. — Phil Karlton

Please reply to list if it's a mailing list post - https://shlom.in/reply .

ssokolow commented 2 years ago

The git checker is what I'm currently using but, as far as I can tell, it only supports pulling the tagged revision of the repository... probably because git itself doesn't have a provision for release artifact hosting and they don't want to be GitHub-specific about it.

I'd prefer not to depend on a specific distro as the upstream, so Fedora Rawhide is out.

As long as the old direct-to-tarball URLs remain valid until I decide to do a version bump, the site should be OK if you care that much about validity, but it's late in my day, so I'll have to try building from that release tarball tomorrow.

joeraz commented 2 years ago

@ssokolow How are things coming here? I'm currently hoping to release a new version next weekend, but I can delay it for a couple weeks if you have something that needs to go in the release.

ssokolow commented 2 years ago

It's been a bad week for me, so I haven't made any progress. Assuming using the release tarballs for fc-solve and the black hole solver clears up the complications with getting things to build with an immutable /usr, I may have a finished manifest for you within a day or two of resuming work, so I'll try to get back to it tomorrow or Tuesday.

ssokolow commented 2 years ago

@shlomif

========================================================================
Building module fc-solve in /home/ssokolow/src/PySolFC_flatpak/.flatpak-builder/build/fc-solve-16
========================================================================
-- The C compiler identification is GNU 12.1.0
-- The CXX compiler identification is GNU 12.1.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found Perl: /usr/bin/perl (found version "5.36.0") 
Cloning into 'rinutils'...
fatal: unable to access 'https://github.com/shlomif/rinutils.git/': Could not resolve host: github.com
CMake Error at cmake/rinutils_bootstrap.cmake:24 (MESSAGE):
  Could not find rinutils anywhere - it should have been bundled in the
  releases' source tarball.

  You can try installing it from a source release or from its repository:
  https://github.com/shlomif/rinutils

  Also see: https://github.com/shlomif/fortune-mod/issues/44
Call Stack (most recent call first):
  CMakeLists.txt:222 (RINUTILS_SET_UP_FLAGS)

Looks like https://fc-solve.shlomifish.org/downloads/fc-solve/freecell-solver-6.6.0.tar.xz has the same problem I was having to hack around with the git repositories. For reasons of reproducibility, flatpak-builder separates the build process into distinct download and build phases and no custom code is allowed during the download phase.

...and I can't just pull in rinutils manually, because that's what I've been trying to do and what's been ignoring my -DCMAKE_MODULE_PATH=/app/lib/cmake and pitching a fit because flatpak-builder doesn't let me install things to /usr/share/cmake/Modules/Shlomif_Common.cmake.

@joeraz How would you feel about releasing a Flatpak without the solvers initially, to not delay the release, then adding the solvers later once this mess gets resolved somehow? ...because aside from those solvers, what I've pushed does produce a build that's fully functional under every test I tried and meets or exceeds every goal I'd set myself for a release-ready Flatpak package.

You could add a note about the solvers to the description that gets displayed on Flathub. I've seen other packages do that to communicate differences between the Flathub release and other releases.

denilsonsa commented 2 years ago

IMHO, given all the trouble, I think it's worth releasing the flatpak without the solver. PySolFC has 1160 available games, while the solver is for 18 games.

If needed, someone could put a little info dialog on PySolFC (flatpak version) where the solver usually is. The dialog could be something like this:

This version of PySolFC doesn't include the solvers. If you think they are important, can you please vote for this feature at https://example.com/somewhere ?

This could be a way to gather the amount of interest in bundling the solvers.

Alternatively, since the solver is also available online, PySolFC could have a link to the online solver pre-populated with the current deal, like this: https://fc-solve.shlomifish.org/js-fc-solve/text/?stdin=4C%202C%209C%208C%20QS%204S%202H%0A5H%20QH%203C%20AC%203H%204H%20QD%0AQC%209S%206H%209H%203S%20KS%203D%0A5D%202S%20JC%205C%20JH%206D%20AS%0A2D%20KD%20TH%20TC%20TD%208D%0A7H%20JS%20KH%20TS%20KC%207C%0AAH%205S%206S%20AD%208H%20JD%0A7S%206C%207D%204D%208S%209D&preset=lg&deal_number=561&one_based=1&unicode_suits__ascii=1&unicode_suits__unicards=0&unicode_suits__unisuits=0&string_params=&game_type=freecell&num_freecells=default&num_columns=default (And showing this link could be added also to the vanilla PySolFC, in case it detects the solver is not found.)

And thank you for all the work so far!

shlomif / PySolFC

[idea] Publish a flatpak on Flathub #256