search

Thefrank / jellyfin-server-freebsd

jellyfin-server component for freebsd
185 stars 16 forks source link

automate builds with gh actions #78

Open Migelo opened 3 months ago

Migelo commented 3 months ago

This draft pull request aims to automate the builds using gh actions vmactions/freebsd-vm@v1 and renovate bot to trigger the builds when https://github.com/jellyfin/jellyfin publish a new release.

I need help with the building part using dotnet.

  1. As per your comment https://github.com/dotnet/source-build/issues/1139#issuecomment-1943360539 I tried the simple pkg install dotnet to get dotnet in the FreeBSD VM,
  2. download the latest jellyfin_10.9.6-amd64.tar.xz
  3. TODO build

I don't have any experiences building with dotnet, so if you could give me some pointers, I am happy to spend time to get this automated build process going.

Thefrank commented 3 months ago

Oh nice!

I added my WIP/experimental ports build to repo. You can check it out here.

I forgot to upload it last week :(

I can add this (and something for the UI part) if I am unable to get a ports version working

Migelo commented 3 months ago

Thank you! I will have a look and then try to make it work, but ofc feel free to push commits yourself. :)

Migelo commented 3 months ago

I managed to get the environment more or less set up, at least how I understand it should be set up. The build fails tho, can you please have a look? The error is here step 3, line 4472 and beyond.

Here are the first few lines for convenience.

  cd /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/ &&  dotnet restore Jellyfin.Server --runtime freebsd-x64 --packages /usr/ports/multimedia/jellyfin/work/packages &&  dotnet publish Jellyfin.Server --configuration Release --self-contained --runtime freebsd-x64 --packages /usr/ports/multimedia/jellyfin/work/packages -o bin/publish "-p:DebugSymbols=false;DebugType=none;UseAppHost=true;PublishReadyToRun=false"

  Welcome to .NET 8.0!
  ---------------------
  SDK Version: 8.0.100

  ----------------
  Installed an ASP.NET Core HTTPS development certificate.
  To trust the certificate, view the instructions: https://aka.ms/dotnet-https-linux

  ----------------
  Write your first app: https://aka.ms/dotnet-hello-world
  Find out what's new: https://aka.ms/dotnet-whats-new
  Explore documentation: https://aka.ms/dotnet-docs
  Report issues and find source on GitHub: https://github.com/dotnet/core
  Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
  --------------------------------------------------------------------------------------
    Determining projects to restore...
    Restored /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/src/Jellyfin.MediaEncoding.Keyframes/Jellyfin.MediaEncoding.Keyframes.csproj (in 506 ms).
  /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/src/Jellyfin.Networking/Jellyfin.Networking.csproj : error NU1603: Warning As Error: Microsoft.AspNetCore.HttpOverrides 2.2.0 depends on Microsoft.Extensions.Options (>= 2.2.0) but Microsoft.Extensions.Options 2.2.0 was not found. An approximate best match of Microsoft.Extensions.Options 8.0.0 was resolved. [/usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Jellyfin.Server/Jellyfin.Server.csproj]
  /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/src/Jellyfin.MediaEncoding.Hls/Jellyfin.MediaEncoding.Hls.csproj : error NU1603: Warning As Error: Microsoft.AspNetCore.HttpOverrides 2.2.0 depends on Microsoft.Extensions.Options (>= 2.2.0) but Microsoft.Extensions.Options 2.2.0 was not found. An approximate best match of Microsoft.Extensions.Options 8.0.0 was resolved. [/usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Jellyfin.Server/Jellyfin.Server.csproj]

It goes on and on with the final lines being

  /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Emby.Naming/Emby.Naming.csproj : error NU1603: Warning As Error: Microsoft.Extensions.FileProviders.Abstractions 2.2.0 depends on Microsoft.Extensions.Primitives (>= 2.2.0) but Microsoft.Extensions.Primitives 2.2.0 was not found. An approximate best match of Microsoft.Extensions.Primitives 8.0.0 was resolved. [/usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Jellyfin.Server/Jellyfin.Server.csproj]
  /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Emby.Naming/Emby.Naming.csproj : error NU1603: Warning As Error: Microsoft.Net.Http.Headers 2.2.0 depends on Microsoft.Extensions.Primitives (>= 2.2.0) but Microsoft.Extensions.Primitives 2.2.0 was not found. An approximate best match of Microsoft.Extensions.Primitives 8.0.0 was resolved. [/usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Jellyfin.Server/Jellyfin.Server.csproj]
  /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Emby.Naming/Emby.Naming.csproj : error NU1603: Warning As Error: Microsoft.AspNetCore.Http.Abstractions 2.2.0 depends on System.Text.Encodings.Web (>= 4.5.0) but System.Text.Encodings.Web 4.5.0 was not found. An approximate best match of System.Text.Encodings.Web 8.0.0 was resolved. [/usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Jellyfin.Server/Jellyfin.Server.csproj]
    Failed to restore /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Emby.Naming/Emby.Naming.csproj (in 31 ms).
    Failed to restore /usr/ports/multimedia/jellyfin/work/jellyfin-10.9.2/Emby.Server.Implementations/Emby.Server.Implementations.csproj (in 129 ms).
  *** Error code 1

  Stop.
  make: stopped in /usr/ports/multimedia/jellyfin

Thank you for your help!

Thefrank commented 3 months ago

So NuGet has this long standing history of looking for or restoring NuGets that are not needed during a specific code path. This can (rarely) happen if there is mismatch between the the requested SDK and the resolved SDK. This can make the determining the FULL list of depends rather challenging.

Some of the ones listed were also stored and restored from local cache when I built jellyfin outside of ports this which is why they got missed here.

I will work on getting that updated with the actual list

On the plus side:

the recently released FreeBSD 14.1 includes ZFS based VM images which will make troubleshooting inside a proper build environment easier

the UI looks like it built without issue

Migelo commented 3 months ago

Thank you for the help. From what you wrote I understand that you'll work on an updated list of NuGets that should fix this build.

In the meantime, how would I go about building jellyfin outside of ports? Might be the simpler way in the end.

Excellent news, I love myself some zfs goodness :) I am currently running a FreeBSD VM through quickemu on an Ubuntu 24.04 host.

Thefrank commented 3 months ago

The method that I currently use to build Jellyfin is to use Azure Pipelines Agent (I publish a build here) and use a locally hosted instance.

Thefrank commented 3 months ago

I have updated the NuGet list. That should be all of them. This is also more than what is actually consumed by the build but instead everything that is explicitly referenced, transitively referenced, and things that NuGet wants restored anyways.

Thefrank commented 3 months ago

fixed an extra file in pkg-plist poudriere testport -b latest -o multimedia/jellyfin completes without errors. I am not brave enough to try and have it build: node, npm, llvm, and dotnet but it does work with current packages

Migelo commented 3 months ago

The first run ran just short of 2h and then the runner crashed...so I first assumed we ran out of ram or disk space, but reading the gh runner docs they mention 16GB of ram and 14GB of disk, so I do not believe we could be crashing due to RAM running out.

Did you run the poudriere command on the port from this repository? I assume so, but I am new to all of this FreeBSD packaging.

Migelo commented 3 months ago

The run finished successfully with this!

I think the last thing we need to do now is to extract the artifacts and we're good to go!

Thefrank commented 3 months ago

The first run ran just short of 2h and then the runner crashed...so I first assumed we ran out of ram or disk space, but reading the gh runner docs they mention 16GB of ram and 14GB of disk, so I do not believe we could be crashing due to RAM running out.

I think most runners are capped at 2h. If poudriere has to build a large dependence tree it can take hours/days on first run.

Did you run the poudriere command on the port from this repository? I assume so, but I am new to all of this FreeBSD packaging.

The command issued was against ports tree with the jellyfin port files replaced with ones from this repo. This is why poudriere reports an unclean checkout in the log.

log: jellyfin-10.9.2.log

Migelo commented 3 months ago

The command issued was against ports tree with the jellyfin port files replaced with ones from this repo. This is why poudriere reports an unclean checkout in the log.

Great, thank you for the explanation.

I think most runners are capped at 2h. If poudriere has to build a large dependence tree it can take hours/days on first run.

It seems the runner runs out of space during the build, see this run for example.

I switched over to ubicloud ubicloud-standard-4 runner. It ended up working and I built 10.9.2 here. I tried running it and it did not work unfortunately, but maybe that's because I was running 10.9.4 on my server and downgrading can cause issues.

I also implemented renovate so that it automatically compares the current release of jellyfin to the version string in the Makefile and then updates the value with the new one and submits a pull request, like this one for 10.9.6.

I noticed the version string is also present in

I guess we need to fix those as well.

Thefrank commented 3 months ago

I used 10.9.2 as a framework and to wait for @mvanbaak feedback on it. I was able to get it to build in poudriere so it should work as a drop in replacement for the current ports version which downloads a prebuilt binary from here.

The upside of it being a source built from ports is people won't complain about it being "binary only" :)

I noticed the version string is also present in

This is working as designed. Any time Jellyfin gets a bump the NuGet files (tedious), Node files (trivial), and pkg-plist (easy) files need to be redone. This also means regenerating the distinfo (easy). Patches are another item that will likely need to be redone often as they are applied without fuzzing.

Migelo commented 3 months ago

This is working as designed.

Oki, great! Then what do you recommend? That I switch to poudriere or keep my current setup as defined in the workflow?

Thefrank commented 3 months ago

poudriere is the best way to reproduce the FreeBSD packaging process as that is what FreeBSD uses. Anything submitted to port must pass poudriere.

A single dotnet line is the best way to just build the server and a single line npm is the best way to build the web-ui. Until now, this has been my method as I can just run a single YML and have it all done.

AFAIK poudriere lacks a method of automatically bumping/editing files. This is typically the job of the port maintainer (or the person building something locally).

Migelo commented 3 months ago

Great, I have been switching to poudriere and compiling on my machine to test. Hopefully I can get a quick build automated here soon.