NixOS / nixpkgs

Nix Packages collection & NixOS
MIT License
16.4k stars 12.9k forks source link

FOSS music trackers packaging effort #81815

Open fgaz opened 4 years ago

fgaz commented 4 years ago

I recently took an interest in the world of tracker music and discovered that there are a lot of FOSS trackers, but that packages for them are scattered across distros (Debian has quite a lot though).

So i decided to package all of them for NixOS :D

This issue tracks ~my~our packaging effort. Suggestions are welcome!

In no particolar order (name - {Repology, Homepage, pull request #} - additional details):

Related stuff which we could maybe package

There are definitely historical, long unmaintained, no longer used trackers in this list that probably belong to a NUR. Still, it'd be nice to have as much as possible in nixpkgs.

Resources
OPNA2608 commented 4 years ago
  • [ ] bambootracker

I have been sitting on a derivation for BambooTracker since the big Qt infrastructure awhile ago, been meaning to open a PR for it once the ever-approaching 0.4 release is out. I can open a PR for the current state later today, you can give it a look if you want.

  • [ ] 0cc-famitracker

FT and any fork thereof is Windows-only, though some forks improve Wine support by alot. I don't think packaging Wine-based applications is something we do in nixpkgs. In any case, you should give j0CC-FamiTracker a look instead for your personal use.

  • [ ] schismtracker - was already in nixpkgs, needs a .desktop file

I still have an update PR outstanding for schismtracker, can look into adding a .desktop file while I'm at it. https://github.com/NixOS/nixpkgs/pull/76725

Something that's missing from your list would be CheeseCutter, which is stuck waiting for GDC to get packaged (https://github.com/NixOS/nixpkgs/pull/69144 needs to get merged first) - I tried building it with the other Dlang compilers already in nixpkgs but it relies on GDC's support of legacy syntax to compile.

deliciouslytyped commented 4 years ago

Tangentially related, if anyone is interested in this issue, they may also be interested in the various JACK related issues.

fgaz commented 4 years ago

@OPNA2608 thanks for the interest! I added your prs and cheesecutter to the list

FT and any fork thereof is Windows-only

oops, I thought I filtered out all of those, thanks

OPNA2608 commented 4 years ago
  • [ ] deflemask?

While it's a very popular chiptune tracker, DefleMask is closed-source and far from FOSS-happy. See the VICE emulator's Hall of Shame section on DefleMask & reSID. The latest distributed non-reSID binary could be patched & wrapped but I get abit of a stomach ache thinking about it.

fgaz commented 4 years ago

That wasn't intended, thanks again for catching it! I removed it and added a bunch more from other repos and some nice resources I found (1 2 3 4).

37 trackers! That's a lot!

OPNA2608 commented 3 years ago
  • [ ] bambootracker

Merged! Can finally be ticked off the list. :smile:


Quite new, and I'm talking to the dev about what can be done to improve packaging. It does some awful things:

Once these problems are solved, I plan to open a PR.

fgaz commented 3 years ago

Great! I Updated the list

Compiling, including and unconditionally expecting unrar at runtime

ow. that's unfree... what does it need it for? The project looks cool though

OPNA2608 commented 3 years ago

Really only for convenience, apparently packs of SPC files (SNES music code) are usually in the RAR format. I have open issues for those problems:

fgaz commented 3 years ago

@OPNA2608 should I also ping you in my prs?

OPNA2608 commented 3 years ago

Feel free to do so!

OPNA2608 commented 3 years ago

Can you do me a favour and put

[ ] Fix macOS build

under CheeseCutter? I have been meaning to tackle that for awhile but never got around to test it properly and keep forgetting about it.

There's an updated commit on the repo that introduces Darwin-related compilation fixes and the SDL loading module requires some patching on our side to handle our non-standard path straight to the dylib, as opposed to an SDL.framework in the Library.

I'll hopefully remember it and make a PR later this week.

fgaz commented 3 years ago

Done! I wonder if there's a way to let you edit the OP too

OPNA2608 commented 3 years ago

I don't think there is.

For reference, a failed darwin evaluation on Hydra: https://hydra.nixos.org/build/126193708

OPNA2608 commented 3 years ago

Talked to the developer abit to fix some installation problems over at https://github.com/yuxshao/ptcollab/issues/3, will make a PR after testing the online part with some friends.

fgaz commented 3 years ago

Added to the list :+1:

fgaz commented 3 years ago

I found another great resource: https://wiki.thingsandstuff.org/Tracker

dukzcry commented 3 years ago

@OPNA2608 I really liked BambooTracker for these features: 1) Modern interface 2) MIDI keyboard support and jam mode 3) FM synthesis

What other trackers with same features would you recommend? 3rd point is not necessary, since its quite niche

fgaz commented 3 years ago

@dukzcry if I remember correctly, of those packaged soundtracker has the modern interface and midi support, and klystrack has all three but in limited form

fgaz commented 3 years ago

I merged all resources and there are ~60 entries now... There's a lot of cool stuff in there! (and also a fair amount of abandoned projects)

OPNA2608 commented 3 years ago

  • [x] hivelytracker {R H #95797}
    • [ ] unbreak darwin (relevant comments in the init pr)

I remembered this was still open. I hacked together a Darwin build, derivation needs cleanup & some more lib.optional* usage to not break Linux building. Has some problems like missing graphics on my end that may be a problem with my system, or with the packaging. But at least it builds & doesn't crash, so it's an improvement I guess? :sweat_smile:.

Screenshot_macOS_2021-04-17_22:32:20

Don't have the time rn to flip between machines and fix everything up, but I'll review a PR if you clean it up & CC me.

Clickme for patch ```patch @@ -3,6 +3,7 , fetchFromGitHub , pkg-config , makeWrapper +, xcbuildHook , SDL , SDL_image , SDL_ttf @@ -24,34 +25,63 stdenv.mkDerivation rec { nativeBuildInputs = [ pkg-config makeWrapper - ]; + ] + ++ lib.optional stdenv.hostPlatform.isDarwin xcbuildHook; buildInputs = [ SDL SDL_image SDL_ttf - gtk2 - glib + # gtk2 + # glib ]; - makeFlags = [ - "-C sdl" - "-f Makefile.linux" - PREFIX=$(out)" - ]; + # makeFlags = [ + # "-C sdl" + # "-f Makefile.linux" + # PREFIX=$(out)" + # ]; # TODO: try to exclude gtk and glib from darwin builds NIX_CFLAGS_COMPILE = [ - "-I${SDL}/include/SDL" + "-I${SDL.dev}/include/SDL" "-I${SDL_image}/include/SDL" "-I${SDL_ttf}/include/SDL" - "-I${gtk2.dev}/include/gtk-2.0" - "-I${glib.dev}/include/glib-2.0" + # "-I${gtk2.dev}/include/gtk-2.0" + # "-I${glib.dev}/include/glib-2.0" ]; + preBuild = '' + cd sdl/osx/xcode + # xcode project expects SDL Frameworks for linking, we only have dylibs + substituteInPlace Hivelytracker.xcodeproj/project.pbxproj \ + --replace 'lastKnownFileType = wrapper.framework; name = SDL.framework; path = /Library/Frameworks/SDL.framework' 'lastKnownFileType = "compiled.mach-o.dylib"; name = libSDL.dylib; path = ${SDL}/lib/libSDL.dylib' \ + --replace 'lastKnownFileType = wrapper.framework; name = SDL_image.framework; path = /Library/Frameworks/SDL_image.framework' 'lastKnownFileType = "compiled.mach-o.dylib"; name = libSDL_image.dylib; path = ${SDL_image}/lib/libSDL_image.dylib' \ + --replace 'lastKnownFileType = wrapper.framework; name = SDL_ttf.framework; path = /Library/Frameworks/SDL_ttf.framework' 'lastKnownFileType = "compiled.mach-o.dylib"; name = libSDL_ttf.dylib; path = ${SDL_ttf}/lib/libSDL_ttf.dylib' + substituteInPlace ../../../hvl2wav/Makefile \ + --replace 'gcc' '${stdenv.cc.targetPrefix}cc' + ''; + # Also build the hvl2wav tool postBuild = '' - make -C hvl2wav + make -C ../../../hvl2wav + ''; + + installPhase = lib.optionalString stdenv.hostPlatform.isDarwin '' + runHook preInstall + install -Dm755 Products/Release/Hivelytracker.app/Contents/MacOS/Hivelytracker $out/lib/Hivelytracker + cp -R Products/Release/Hivelytracker.app/Contents/Resources/* $out/lib/hivelytracker + # Crashes if fonts are not in ttf/ subdirectory + mkdir $out/lib/hivelytracker/ttf + mv $out/lib/hivelytracker/{*.,}ttf + mkdir $out/bin + runHook postInstall ''; postInstall = '' @@ -59,11 +89,11 @@ stdenv.mkDerivation rec { # Ideally we should patch the sources, but the program can't open # files passed as arguments anyway, so this works well enough until the # issue is fixed. - wrapProgram $out/bin/hivelytracker \ - --run "cd $out/share/hivelytracker" + makeWrapper $out/lib/hivelytracker/Hivelytracker $out/bin/hivelytracker \ + --run "cd $out/lib/hivelytracker" # Also install the hvl2wav tool - install -Dm755 hvl2wav/hvl2wav $out/bin/hvl2wav + install -Dm755 ../../../hvl2wav/hvl2wav $out/bin/hvl2wav ''; meta = with lib; { @@ -83,7 +113,6 @@ stdenv.mkDerivation rec { license = licenses.bsd3; platforms = platforms.all; maintainers = with maintainers; [ fgaz ]; - broken = stdenv.isDarwin; # TODO: try to use xcbuild }; } ```

FT and any fork thereof is Windows-only

oops, I thought I filtered out all of those, thanks

Actually, nesicide includes a version of FamiTracker that's cross-platform, via an MFC-over-Qt compatibility layer. It has some problems but it can launch and produce some sound. I haven't tried packaging the whole IDE yet though.

grafik

OPNA2608 commented 3 years ago

I picked afew projects from the list that I've heard of before / seemed interesting and hacked-together overlays:

Chibitracker

Requires some patching to build for x86_64-linux, builds fine out-of-the-box for i686-linux (via pkgsi686Linux.chibitracker).

package-chibitracker.txt

grafik

i686 versions works mostly fine, only crashes when exiting. x86_64 can't load a module most of the time and crashes with a bus error. It's an interesting little project but it seems abandoned and abit broken, not sure if it makes sense to include this one.

AdlibTracker2

x86_64 support requires replacing x86-exclusive inline assembly with equivalent x86_64 assembly… :confused:

package-adlibtracker2.txt

Compiles on x86_64-linux but crashes immediately. This may be "normal" however because I heard that the AUR's package (which installs the prebuilt Linux version) also crashes for some people in a similar fashion. Not sure what's up with that.

  Sample buffer size: 1024 samples (requested 2048)
  Sampling rate: 49716 Hz (requested 49716)
Reading configuration file ... not found!
An unhandled exception occurred at $00000000004B200E:
EAccessViolation: Access violation
  $00000000004B200E

i686-linux doesn't build because i686 fpc fails to build for me. The project might still be active.

Arkostracker2

The project is still binary-only for now but I tried to package it regardless.

package-arkostracker2.txt

It "builds" but fails on me when it tries to draw via libGL. Maybe this works on an actual NixOS machine.

bt1cn@pcclu01:/dev/shm$ nixGL ./result/bin/ArkosTracker2 
./result/bin/ArkosTracker2() [0x7c8ce7]
./result/bin/ArkosTracker2() [0x70c009]
./result/bin/ArkosTracker2() [0x7bab0d]
/nix/store/sbbifs2ykc05inws26203h0xwcadnf0l-glibc-2.32-46/lib/libpthread.so.0(+0x13700) [0x7f17fd936700]
/nix/store/hpnazqkp4zbm2pkp1xj4whmld2zhwv0r-libglvnd-1.3.3/lib/libGLX.so.0(+0xa415) [0x7f17fd30b415]
/nix/store/hpnazqkp4zbm2pkp1xj4whmld2zhwv0r-libglvnd-1.3.3/lib/libGLX.so.0(glXChooseVisual+0x13) [0x7f17fd304633]
./result/bin/ArkosTracker2() [0xa67d16]
./result/bin/ArkosTracker2() [0x9620d7]
./result/bin/ArkosTracker2() [0x9a5147]
./result/bin/ArkosTracker2() [0x6c4f53]
./result/bin/ArkosTracker2() [0x70cf1b]
./result/bin/ArkosTracker2() [0x85984a]
./result/bin/ArkosTracker2() [0x4e8b1b]
/nix/store/sbbifs2ykc05inws26203h0xwcadnf0l-glibc-2.32-46/lib/libc.so.6(__libc_start_main+0xed) [0x7f17fd3c9ded]
./result/bin/ArkosTracker2() [0x4eace9]
/home/bt1cn/.nix-profile/bin/nixGL: line 6:  8477 Killed                  "$@"

Last release was on March 21, 2021 so it's very much in development still.

psycle

The qpsycle project isn't available on their SVN repository anymore. There is a new cpsycle rewrite that's in active development but it only has a build script for Windows. psycle-core etc are still available and buildable on Linux, albeit with some fixes.

package-psycle.txt

This is only a command-line-only player however, and so far I've been unable to get any sound out of it. Psycle modules embed the names of the plugins they need… with a DLL extension. I don't know if that affects the plugin loading. I saw in strace that it also tries to find a config.xml at some point. ALSA also doesn't seem to work & causes an abortion.

1tracker

I gave this a quick attempt. I think it's trying to test something by writing to the nix store at runtime - which obviously fails - so it instantly errors. I'll try to look into this more. I think it's still active.

package-1tracker.txt

(Excuse the monitor photo, any button press will close the window) IMG_20210622_134625


ZorroTracker is a very new, WIP and active project (since there's some minor financial backing behind it) but it's Electron-based. I have no clue how to build that from source and gave up on it.

fgaz commented 3 years ago

Wow, nice job!

AdlibTracker2

I also tried building it but gave up shortly after. Does the cross32 makefile help?


1tracker

I tried stracing it and it looks like it's missing some files: $out/lib/1tracker/engines/{test.1te,list,hash}.

If you add test.1te (even just by copying one of the other engines) it starts, but fails if you try to select a different engine.

If you write the list of engines in the list file, the selection works and you can make some sounds. I have yet to figure out the specific format though. I tried one filename per line and only the first entry seems to work.

I have no idea what the hash file does.

EDIT: By looking at the readme, it seems like those files are caches, which explains the error EDIT2: yup. when running it from a writable directory, I get "updating engines"... and apparently better graphics too?!? EDIT3: okay, the cache files are a biit more complicated than I thought: list.txt hash.txt

EDIT4, screenshots:

In the store: 2021-06-23-135844_1366x768_scrot

In the writable src directory: 2021-06-23-135556_1366x768_scrot 2021-06-23-135601_1366x768_scrot

the different graphics are probabaly caused by 1tracker.cfg

OPNA2608 commented 3 years ago

okay, the cache files are a biit more complicated than I thought:

.1te files seem to be C/C++ source code, with a function void Info(void) that prints the title & about stuff from list (i.e. ZX|1BIT|AntEater etc). Maybe we could write a bash script that scans and collects the engines' filename, title and about texts to build the list file.

hash looks to be all the file names, truncated / left-aligned and padded to 8 characters & appended, with a \0-terminator.

Launching & changing engines with in-store version ![grafik](https://user-images.githubusercontent.com/23431373/123105178-df534c00-d437-11eb-9a55-d4ac15ed7e61.png) ![grafik](https://user-images.githubusercontent.com/23431373/123105749-5852a380-d438-11eb-8f75-bf243da146dd.png) ![grafik](https://user-images.githubusercontent.com/23431373/123105776-5dafee00-d438-11eb-959f-3ac90be12e4c.png)
  installPhase = ''
    runHook preInstall

    mkdir -p $out/{bin,lib/1tracker}

    mv colors/ docs/ engines/ examples/ fonts/ 1tracker $out/lib/1tracker/
    ln -s $out/{lib/1tracker,bin}/1tracker

    # Write list and hash files for the tracker
    pushd $out/lib/1tracker/engines
    for engine in *; do
      echo $engine
      # Ignore non-engines for list, but not for hash
      if [[ "$engine" == *.1te ]]; then
        echo $engine >> list
        grep 'SetTitle' $engine | cut -d'"' -f2 >> list
        grep 'SetAbout' $engine | cut -d'"' -f2 | sed -e 's/\\n/\n/g' >> list
        echo "###" >> list
      fi  
      printf '%-8s' ''${engine:0:8} >> hash
    done
    printf '\0' >> hash
    ln -s ./{anteater,test}.1te
    popd

    runHook postInstall
  ''; 

I think we can fix the cfg path problem by patching in SDL_GetPrefPath.

fgaz commented 3 years ago

.1te files seem to be C/C++ source code

angelscript actually! (which looks very nice, I might use it for some of my own projects...) so we could use the angelscript package in nixpkgs to run them

fgaz commented 3 years ago

Though the best long-term solution is probably to ask the author to use XDG for cache and configuration (and maybe to pick up additional engines). Should we contact them?

OPNA2608 commented 3 years ago

angelscript actually! (which looks very nice, I might use it for some of my own projects...) so we could use the angelscript package in nixpkgs to run them

I think devendoring angelscript would be a good idea if it's feasible, yeah.

Should we contact them?

We can certainly ask! :smile:


I patched in SDL_GetPrefPath and wrote a wrapper that copies the example cfg to the appropriate place if it's missing. It should work "from the store" now. LMK what you think.

package-1tracker.txt

fgaz commented 5 months ago

https://github.com/djdiskmachine/LittleGPTracker is a LGPT fork with a more active maintainer. I may switch the package to it if https://github.com/djdiskmachine/LittleGPTracker/issues/71 is resolved.

P.S.: wow, 2 years since the last tracker addition! Time flies! I need to finish that tiatracker pr...

ScottFreeCode commented 4 months ago

If I wanted to get QTFamitracker (or any other Linux-compatible Famitracker fork, if there are any) in, how involved is that likely to be? I've never packaged a QT app for Nix before, but I've done a little bit of C building in Nix and used to use C and C++ for whole projects back in the day (Nix can't be more of a slog than CMake's multiple steps and interminable build time).

fgaz commented 4 months ago

@ScottFreeCode Qt isn't too bad, see the relevant section in the manual. qmake or cmake is used automatically if you put it in nativeBuildInputs.