Radarr fails to handle Gzip encoded requests

[janstenpickle]

Describe the bug

On Nixos, the current version of Radarr (4.0.5.5981) fails to accept or make HTTP requests due to a missing (I think) Dotnet dependency (liblibSystem.IO.Compression.Native).

Making requests without compression enabled does work, however search inside of Radarr doesn't as the Dotnet HTTP seems to also depend on Zlib.

Steps To Reproduce

Steps to reproduce the behavior:

1. Set up Radarr with with services.radar... options
2. Ensure services.radar.openFirewall is set to true
3. Rebuild the system
4. Browse to <hostname>:7878
5. See error in systemd journal for radarr.service

Expected behavior

Radarr should be able to respond to and make HTTP requests with zlib compression enabled.

Screenshots

Technically not a screenshot, but a stacktrace

[v3.2.2.5080] System.IO.Compression.ZLibException: The underlying compression routine could not be loaded correctly.
 ---> System.DllNotFoundException: Unable to load shared library 'libSystem.IO.Compression.Native' or one of its dependencies. In order to help diagnose loading problems, consider setting the LD_DEBUG environment variable: liblibSystem.IO.Compression.Native: cannot open shared object file: No such file or directory
   at Interop.zlib.DeflateInit2_(ZStream& stream, CompressionLevel level, CompressionMethod method, Int32 windowBits, Int32 memLevel, CompressionStrategy strategy)
   at System.IO.Compression.ZLibNative.ZLibStreamHandle.DeflateInit2_(CompressionLevel level, Int32 windowBits, Int32 memLevel, CompressionStrategy strategy)
   at System.IO.Compression.Deflater..ctor(CompressionLevel compressionLevel, Int32 windowBits)
   --- End of inner exception stack trace ---
   at System.IO.Compression.Deflater..ctor(CompressionLevel compressionLevel, Int32 windowBits)
   at System.IO.Compression.DeflateStream.InitializeDeflater(Stream stream, Boolean leaveOpen, Int32 windowBits, CompressionLevel compressionLevel)
   at System.IO.Compression.DeflateStream..ctor(Stream stream, CompressionMode mode, Boolean leaveOpen, Int32 windowBits, Int64 uncompressedSize)
   at System.IO.Compression.GZipStream..ctor(Stream stream, CompressionMode mode, Boolean leaveOpen)
   at Radarr.Http.Extensions.Pipelines.GzipCompressionPipeline.WriteGZipStream(Action`1 innerContent, Stream targetStream) in D:\a\1\s\src\Radarr.Http\Extensions\Pipelines\GZipPipeline.cs:line 73
   at Radarr.Http.Extensions.Pipelines.GzipCompressionPipeline.<>c__DisplayClass6_0.<CompressResponse>b__1(Stream responseStream) in D:\a\1\s\src\Radarr.Http\Extensions\Pipelines\GZipPipeline.cs:line 51
   at Nancy.Owin.NancyMiddleware.RequestComplete(NancyContext context, IDictionary`2 environment, Func`2 performPassThrough, Func`2 next)
   at Nancy.Owin.NancyMiddleware.<>c__DisplayClass2_1.<<UseNancy>b__1>d.MoveNext()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Owin.WebSocketAcceptAdapter.<>c__DisplayClass6_0.<<AdaptWebSockets>b__0>d.MoveNext()
--- End of stack trace from previous location ---
   at Radarr.Host.Middleware.SignalRMiddleware.<Attach>b__7_0(HttpContext context, Func`1 next) in D:\a\1\s\src\NzbDrone.Host\WebHost\Middleware\SignalRMiddleware.cs:line 51
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests[TContext](IHttpApplication`1 application)

Additional context

I tried both inside and outside of a Nix container, which yielded the same result.

Notify maintainers

Metadata

Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.

 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.89, NixOS, 21.11 (Porcupine)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.3.16`
 - channels(chris): `""`
 - channels(root): `"nixos-21.11.336824.ccb90fb9e11"`
 - nixpkgs: `/nix/var/nix/profiles/per-user/root/channels/nixos`

[illiusdope]

Came to report this. It started after the zlib update for 21.11, however on the unstable channel it works again. This suggests that some other library hasn't been updated for 21.11 (though I'm struggling to find it because both mono and radarr don't seem to have any recent updates in either branch).

[GoogleBot42]

The same issue seems to affect prowlarr. It might affect other *arr as well.

Edit: Looks like updating to master doesn't fix prowlarr. Maybe this is because it is using dotnet core instead of mono. I'll investigate more later.

[janstenpickle]

I doesn't seem to affect Sonarr, but that runs under Mono instead of the Dotnet runtime.

[GoogleBot42]

Looks like arr services like to use the latest dotnet core versions. Since they seem to only test with and use dotnet I think we should stick with that. Converting the arr services to use dotnet-runtime should keep it up to date.

Edit: mono support was dropped https://github.com/Radarr/Radarr/commit/cc572729ff0ce6585f6737c5e4dbd692c2b18fa6

[GoogleBot42]

@illiusdope radarr does not work for me in nixos-unstable.

[GoogleBot42]

ok some progress I think. The following two seem to work correctly in nixos 21.11 (and probably unstable as well)

Nevermind. prowlarr and radarr both seem to work except when executed by the nixos module for some reason

If I try these, I get some certificate errors but it at least serves the webpages.

nix-shell --pure -p prowlarr --run Prowlarr
nix-shell --pure -p radarr --run Radarr

[GoogleBot42]

Not why it isn't already included somehow but adding zlib to LD_LIBRARY_PATH seems to fix it for me. After someone else confirms it works for them, I'll open a PR. I also set it to use latest dotnet-runtime instead of the 3.1 runtime since upstream seems to test on the latest.

  nixpkgs.overlays = [
    (final: prev: {
      radarr = prev.radarr.overrideAttrs (old: rec {
        installPhase = ''
          runHook preInstall
          mkdir -p $out/{bin,share/${old.pname}-${old.version}}
          cp -r * $out/share/${old.pname}-${old.version}/.
          makeWrapper "${final.dotnet-runtime}/bin/dotnet" $out/bin/Radarr \
            --add-flags "$out/share/${old.pname}-${old.version}/Radarr.dll" \
            --prefix LD_LIBRARY_PATH : ${lib.makeLibraryPath [
              final.curl final.sqlite final.libmediainfo final.mono final.openssl final.icu final.zlib ]}
          runHook postInstall
        '';
      });

      prowlarr = prev.prowlarr.overrideAttrs (old: {
        installPhase = ''
          runHook preInstall
          mkdir -p $out/{bin,share/${old.pname}-${old.version}}
          cp -r * $out/share/${old.pname}-${old.version}/.
          makeWrapper "${final.dotnet-runtime}/bin/dotnet" $out/bin/Prowlarr \
            --add-flags "$out/share/${old.pname}-${old.version}/Prowlarr.dll" \
            --prefix LD_LIBRARY_PATH : ${lib.makeLibraryPath [
              final.curl final.sqlite final.libmediainfo final.mono final.openssl final.icu final.zlib ]}
          runHook postInstall
        '';
      });
    })
  ];

[NULLx76]

I can confirm that @GoogleBot42's fix works for me as well

[andir]

FWIW I just dropped this in my configuration:

systemd.services.prowlarr.environment.LD_LIBRARY_PATH="${lib.getLib pkgs.zlib}/lib"

Good enough until we have a fix in nixos.

[MatthiasBenaets]

Thank you for the temporary fix @GoogleBot42.

For those who are waiting it out until a solution is merged, don't forget to stop radarr.serivce, otherwise stuff like bazarr will not fully function or exit out.

[janstenpickle]

Confirmed that @GoogleBot42's fix works in a Nix container too. Thank you @GoogleBot42!

[GoogleBot42]

PRs https://github.com/NixOS/nixpkgs/pull/167916 https://github.com/NixOS/nixpkgs/pull/167917

[GoogleBot42]

the fix is now merged in master

[NULLx76]

seems fixed, closing the issue