Cannot connect to Deemix as indexer or download client

[kaibagley]

Hey, thanks for trying to fix the upstream issues! However im still failing to get Deemix to connect with Lidarr.

When adding as indexer/download client, using localhost:port I immediately get this error: Test was aborted due to an error: Connection refused (localhost:6595), and a timeout error when using the IP address of the server rather than localhost.

Here are the docker logs for Lidarr:

[v2.1.1.3878] FluentValidation.ValidationException: Validation failed: 
 -- : Test was aborted due to an error: Connection refused (localhost:6595)
   at Lidarr.Api.V1.ProviderControllerBase`4.VerifyValidationResult(ValidationResult validationResult, Boolean includeWarnings) in ./Lidarr.Api.V1/ProviderControllerBase.cs:line 276
   at Lidarr.Api.V1.ProviderControllerBase`4.Test(TProviderDefinition definition, Boolean includeWarnings) in ./Lidarr.Api.V1/ProviderControllerBase.cs:line 267
   at Lidarr.Api.V1.ProviderControllerBase`4.Test(TProviderResource providerResource) in ./Lidarr.Api.V1/ProviderControllerBase.cs:line 212
   at Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncObjectResultExecutor.Execute(IActionResultTypeMapper mapper, ObjectMethodExecutor executor, Object controller, Object[] arguments)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync()
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Rethrow(ActionExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeNextResourceFilter>g__Awaited|25_0(ResourceInvoker invoker, Task lastTask, State next, Scope scope, Object state, Boolean isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Rethrow(ResourceExecutedContextSealed context)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(State& next, Scope& scope, Object& state, Boolean& isCompleted)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync()
--- End of stack trace from previous location ---
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.<InvokeAsync>g__Awaited|17_0(ResourceInvoker invoker, Task task, IDisposable scope)
   at Microsoft.AspNetCore.Routing.EndpointMiddleware.<Invoke>g__AwaitRequestTask|6_0(Endpoint endpoint, Task requestTask, ILogger logger)
   at Lidarr.Http.Middleware.BufferingMiddleware.InvokeAsync(HttpContext context) in ./Lidarr.Http/Middleware/BufferingMiddleware.cs:line 28
   at Lidarr.Http.Middleware.IfModifiedMiddleware.InvokeAsync(HttpContext context) in ./Lidarr.Http/Middleware/IfModifiedMiddleware.cs:line 41
   at Lidarr.Http.Middleware.CacheHeaderMiddleware.InvokeAsync(HttpContext context) in ./Lidarr.Http/Middleware/CacheHeaderMiddleware.cs:line 33
   at Lidarr.Http.Middleware.StartingUpMiddleware.InvokeAsync(HttpContext context) in ./Lidarr.Http/Middleware/StartingUpMiddleware.cs:line 38
   at Lidarr.Http.Middleware.UrlBaseMiddleware.InvokeAsync(HttpContext context) in ./Lidarr.Http/Middleware/UrlBaseMiddleware.cs:line 27
   at Lidarr.Http.Middleware.VersionMiddleware.InvokeAsync(HttpContext context) in ./Lidarr.Http/Middleware/VersionMiddleware.cs:line 29
   at Microsoft.AspNetCore.ResponseCompression.ResponseCompressionMiddleware.InvokeCore(HttpContext context)
   at Microsoft.AspNetCore.Authorization.Policy.AuthorizationMiddlewareResultHandler.HandleAsync(RequestDelegate next, HttpContext context, AuthorizationPolicy policy, PolicyAuthorizationResult authorizeResult)
   at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
   at Microsoft.AspNetCore.Diagnostics.ExceptionHandlerMiddleware.<Invoke>g__Awaited|6_0(ExceptionHandlerMiddleware middleware, HttpContext context, Task task)

[Warn] LidarrErrorPipeline: Invalid request Validation failed: 
 -- : Test was aborted due to an error: Connection refused (localhost:6595) 

[codefaux]

Can I see the container info for both containers? Without it, I'm kind of guessing.

Either the docker run command used to create them, the docker-compose.yaml they use (hint: if you use docker-compose.yaml you can put them both in the same file for better interoperability) or the screenshots of the Unraid template you're using would be best.

lol, THAT SAID. Here's my guess;

I believe you might be running into an edge issue where containers sometimes cannot connect to a container-created port on the same host, on the same network interface. In my personal setup, I'm using docker's container name resolution feature when I need a container to speak to another container. Docker - wether or not you're using docker-compose service groups - will translate a container name to a DNS entry. So, if your containers are named "lidarr" and "deemix" you would point Lidarr to "deemix:6595" instead of "localhost:6595" and that may solve it. This is also helpful with regards to containers which require a separate database container, for example.

[ad-on-is]

I started experiencing the same issue.

For context: I've updated my deezer account, so I can download FLACs. Therefore, I'm going through my current artists, and I'm re-downloading everything again, hence I'm extensively using deemix now.

I'm very familiar with Docker, and I have everything in one single docker-compose.yaml and the containers talk to each other with their names.

Here are my findings regarding this issue. Note: These either occur frequently or just one time.

• When the error happens, deemix loses the account information, and refuses to log in again (with username/pw or ARL)
• Deemix shows the following warning: The app can't reach Deezer. Check your internet connection, your firewall or your antivirus.

Errors thrown within deemix:

As can be seen, a successfully downloaded album was removed from the queue Further down, a bunch of 403 errors happen. It looks like as if deemix is restarting itself, due to errors, indicated by: Listening on 0.0.0.0:6595

2024-04-10T14:20:47.437Z [info] [album_39924621_9] Removed from the queue

[ERROR] deezer.gw deezer.pageAlbum { ALB_ID: '9535918', lang: 'en', header: true, tab: 0 } HTTPError Response code 403 (Forbidden)

2024-04-10T14:21:17.595Z [error] unhandledRejection: deezer.pageAlbum [object Object]:: HTTPError: Response code 403 (Forbidden)

GWAPIError: deezer.pageAlbum [object Object]:: HTTPError: Response code 403 (Forbidden)

    at GW.api_call (/snapshot/deemix-gui/server/dist/app.js)

    at runMicrotasks (<anonymous>)

    at processTicksAndRejections (node:internal/process/task_queues:96:5)

GWAPIError: deezer.pageAlbum [object Object]:: HTTPError: Response code 403 (Forbidden)

    at GW.api_call (/snapshot/deemix-gui/server/dist/app.js)

    at runMicrotasks (<anonymous>)

    at processTicksAndRejections (node:internal/process/task_queues:96:5)

[ERROR] deezer.gw deezer.pageAlbum { ALB_ID: '9451006', lang: 'en', header: true, tab: 0 } HTTPError Response code 403 (Forbidden)

[ERROR] deezer.gw deezer.pageAlbum { ALB_ID: '9451018', lang: 'en', header: true, tab: 0 } HTTPError Response code 403 (Forbidden)

[ERROR] deezer.gw deezer.pageAlbum { ALB_ID: '9594408', lang: 'en', header: true, tab: 0 } HTTPError Response code 403 (Forbidden)

[ERROR] deezer.gw deezer.pageAlbum { ALB_ID: '9591936', lang: 'en', header: true, tab: 0 } HTTPError Response code 403 (Forbidden)

-- Using user ID 1000, group ID 1000, umask 002

-- Effective ownership downloaded files will be 1000:1000 u=rwx,g=rwx,o=rx

(node:1) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.

(Use `deemix-server --trace-deprecation ...` to show where the warning was created)

2024-04-10T14:21:21.381Z [info] Listening on 0.0.0.0:6595

/search?term=michael%20jackson%20the%20ultimate

2024-04-10T14:21:23.736Z [info] Currently running deemix-gui version 2024.2.25-r222.5d447b6035

2024-04-10T14:21:23.737Z [info] deemix-lib version 3.6.14

2024-04-10T14:21:23.931Z [error] Response code 403 (Forbidden)

HTTPError: Response code 403 (Forbidden)

    at se.<anonymous> (/snapshot/deemix-gui/server/dist/app.js)

    at processTicksAndRejections (node:internal/process/task_queues:96:5)

[ERROR] deezer.gw deezer.pageSearch {

  query: 'michael jackson the ultimate',

  start: 0,

[codefaux]

I started experiencing the same issue.

No, you are not. They received connection refused from the container connection attempt, you are receiving 403 forbidden from what appears to be the Deezer gateway. Your error says 403, theirs says Connection Refused. They are not seeing container restarts, you are. You are experiencing a somewhat similar symptom.

For context: I've updated my deezer account, so I can download FLACs. Therefore, I'm going through my current artists, and I'm re-downloading everything again, hence I'm extensively using deemix now.

I've spent literally the last three days doing the same between Lidarr+Deemix for acquisition and Picard for tagging, using 320kbit MP3 instead of FLAC, because my own library has become an absolute mess. 50k+ tracks so far.

I'm very familiar with Docker, and I have everything in one single docker-compose.yaml and the containers talk to each other with their names.

Your problem is not Docker related, to my understanding the Deezer gateway is sending 403 to your API attempt. That indicates Lidarr is speaking to Deemix successfully, and Deemix has issued a successful DNS resolution, connection, handshake, and communication of API request to Deezer, but the Deezer server said "You are forbidden from accessing this page." I've perused the Deemix source code, they don't reference the capacity to issue a 403 so I'm concluding that it must be from Deezer.

Here are my findings regarding this issue. Note: These either occur frequently or just one time.

* When the  error happens, deemix  loses the account information, and refuses to log in again (with username/pw or ARL)

I'm not a Deemix dev so I can't guarantee the accuracy of this statement, but this is likely because Deezer is issuing a temporary ban due to bandwidth issues -- I really don't know. My strong suspicion is that this is a Deezer issue, not a Deemix issue (see reports of 403 errors in nearly all popular Deezer downloaders via Google) BUT it may be a Deemix issue. It is not a Deemix-for-Lidarr issue.

* Deemix shows the following warning: **The app can't reach Deezer. Check your internet connection, your firewall or your antivirus. **

Further evidence that Deezer is issuing a temporary ban ... see above.

Errors thrown within deemix:

As can be seen, a successfully downloaded album was removed from the queue Further down, a bunch of 403 errors happen. It looks like as if deemix is restarting itself, due to errors, indicated by: Listening on 0.0.0.0:6595

I can't say why Deemix is crashing and restarting itself, likely it isn't properly prepared to handle the 403 or its 403 handling causes an error/exit. This is the only part of this Issue which I can guarantee is a bug, and it is a Deemix bug caused by Deezer's 403.

This is not a Deemix-for-Lidarr issue.

Please feel free to raise an issue at the appropriate repository if they are accepting issues.

@kaibagley You have not responded since the initial issue was opened. I am left to assume either you have fixed the problem, or you no longer seek to fix the problem. I am closing this issue.

[ad-on-is]

Hey, thanks for the insight, I really appreciate it!

I was going through the issues and saw this one, so didn't want to raise a duplicate.

A temporary ban was my first assumption too, but surprisingly the 403s stop after I restart the deemix container. Maybe just coincidence.

I've spent literally the last three days doing the same between Lidarr+Deemix for acquisition and Picard for tagging, using 320kbit MP3 instead of FLAC, because my own library has become an absolute mess. 50k+ tracks so far.

Lol... Glad to hear I'm not the only one. Btw, you might check out my service that uses deemix for metadata, like missing albums and artists, that are not in musicbrainz.

https://github.com/ad-on-is/lidarr-deemix

[ad-on-is]

Please feel free to raise an issue at the appropriate repository if they are accepting issues.

Oh, one more thing... the project is abandoned. So I don't think raising issues there makes any sense.

[codefaux]

Hey, thanks for the insight, I really appreciate it!

No worries, I'm happy to help how I can. Thank you for not taking my tone negatively, I have issues coming across like an ass, lol.

I was going through the issues and saw this one, so didn't want to raise a duplicate.

I can understand the motivation.

A temporary ban was my first assumption too, but surprisingly the 403s stop after I restart the deemix container. Maybe just coincidence.

I can't speak to that, perhaps the Deemix container bugged and was making malformed requests.

Lol... Glad to hear I'm not the only one. Btw, you might check out my service that uses deemix for metadata, like missing albums and artists, that are not in musicbrainz.

I just took a quick look -- mitm is an interesting approach I had not considered. I may very well wish to set that up -- right now I'm using Deemix as an Indexer/Downloader to pull tracks for Lidarr, and correcting the tags with Picard (mostly because it adds lyrics) where I can force it to match an album. I'm about ready to pull my hair out. Still hundreds of albums with no direct match that I'm basically just walking away from. I'm getting fatigued with the whole "music" thing right now, but I'll add your project to my next pass if I don't feel satisfied with the results.

Oh, one more thing... the project is abandoned. So I don't think raising issues there makes any sense.

OK fair, but it makes more sense than raising them here, lol.

[ad-on-is]

I have issues coming across like an ass, lol.

All good, man! My tone depends on my day too, sometimes.

I'm about ready to pull my hair out.

Yeah, I'm starting to feel that Lidarr just isn't for me.

I thought it'd go through the artists and monitor for new qualities automatically, but it doesn't.

I'm at the point where I'm thinking about writing something on my own, that grabs artists/albums from spotify/lastfm (and manually adding them), fetches metadata from deezer, and uses streamrip for the downloads. I think this would cover my usecase completely.

[codefaux]

All good, man! My tone depends on my day too, sometimes.

This is about as good as my tone gets, unfortunately. I have a very specific way of communicating and it comes across badly.

Yeah, I'm starting to feel that Lidarr just isn't for me.

Definitely starting to get there myself. Deemix would be great if it had a way to track your local library and a feed of releases, the only alternative provided by the Deezer API would be to regularly poll for albums we don't know about with artists we do..

I thought it'd go through the artists and monitor for new qualities automatically, but it doesn't.

If you have Quality Profiles set up properly, it should upgrade qualities of existing album releases if it finds one, up to your configured limit. The problem is, Lidarr's metadata and music releases have so many variants that it often can't say for certain "this is the same album" even if it can say for certain "this is higher quality". You can even configure which order different qualities will be shown in the search results for Manual searches. For example; my searches will show, from the top down, MP3-320, MP3-256, FLAC, and then the rest of the stuff. Automatic searches will accept MP3-256 or MP3-320, nothing else. It will* auto-upgrade to MP3-320 if anything lower on the list (including a previously-imported FLAC) exists.

I'm at the point where I'm thinking about writing something on my own, that grabs artists/albums from spotify/lastfm (and manually adding them), fetches metadata from deezer, and uses streamrip for the downloads. I think this would cover my usecase completely.

I think the solution is to use metadata and file soruces from the same place.

I'll explain; The single largest problem with Lidarr is that it can't differentiate between all of the release versions available.

If its indexer were Deezer, and its metadata provider were Deezer, and its file provider were Deezer, everything would match every time. Same if it only used Spotify.

As soon as you start grabbing files and metadata from different places, you run into exactly what we have now. I think the best approach - aside from a full scratch project - would be to bend Lidarr away from Musicbrainz, or bend Deemix into a better Lidar-alike.

[ad-on-is]

If its indexer were Deezer, and its metadata provider were Deezer, and its file provider were Deezer, everything would match every time. Same if it only used Spotify.

Hmmmmmm... if that's true. Then I'd just need to rewrite my service in a way that it ditches the results from musicbrainz and uses the ones from deemix instead.

Right now, my service works as follows:

• Lidarr makes request to api.lidarr.audio
• The request gets redirected to my service
• my service passes the request to api.lidarr.audio and makes an additional request to deemix
• it then merges the albums from lidarr and demix and sends them back
• Lidarr thinks it got all albums from it's API

So, the only thing I'd have to do is to never include anything from api.lidarr.audio but only from deemix.

[codefaux]

I would be quite interested in hearing how this goes, if you choose to chase it down.