search

advplyr / audiobookshelf

Self-hosted audiobook and podcast server
https://audiobookshelf.org
GNU General Public License v3.0
6.86k stars 485 forks source link

[Bug]: Fails to match when importing larger podcasts #3298

Open Bigmack3000 opened 3 months ago

Bigmack3000 commented 3 months ago

What happened?

Hey, I've uploaded about 60 podcasts now. All but 2 went fine. The two I'm having trouble with are both over 500 episodes. abs is able to find a match, but when i click to actually match it, abs fails to complete the match.

Even if i skip matching and just try to set a rss url, abs still fails.

This does not happen for any podcast under 500 episodes.

What did you expect to happen?

for abs to successfully match the podcast

Steps to reproduce the issue

  1. load a podcast with over 500 episodes. (the two in question for me are "Business Wars" and "Behind the Bastards".
  2. try to match

Audiobookshelf version

2.12.3

How are you running audiobookshelf?

Docker

What OS is your Audiobookshelf server hosted from?

Other (list in "Additional Notes" box)

If the issue is being seen in the UI, what browsers are you seeing the problem on?

Chrome

Logs

No response

Additional Notes

running off synology. 2024-08-17.txt

logs uploaded here.

nichwall commented 3 months ago

To clarify, you are doing the following:

  1. Have the files already downloaded
  2. Manually add to the library through the file system
  3. Scan the podcast library
  4. Add an RSS feed (for the podcast)
  5. Match episodes (this is when the problem occurs?)

When you say "only more than 500 episodes", do you mean you have more than 500 files stored locally, or just that there are more than 500 episodes in the RSS feed?

Bigmack3000 commented 3 months ago

Steps 1-3 yes. I normally try matching the episode first since it tends to be the 1st result nearly every time anyway, and then matching adds the rss feed for me.

These two, failed when matching. Then also failed when just trying to add an rss feed.

Yes more than 500 stored locally.

Right now, I tested starting smaller for one of them. I only added 1 episode of "Behind the Bastards". that was able to match without issue. Now I'm adding 50 at a time to see if there's a point where the problems start.

Bigmack3000 commented 3 months ago

So it's more than 500 episodes before the problem hits. Somewhere around 600-650, but it happens even if you import in chunks. Once, I hit that number, "quick match" started failing. and something about a socket disconnecting pops up. But then it connects again right away.

updated logs attached. 2024-08-17.txt

nichwall commented 3 months ago

Thanks for clarifying. Can you enable Debug logs (in the server settings under "Logs") and then try matching again? That will give more information as to what is going on.

Bigmack3000 commented 3 months ago

2024-08-17.txt ok updated logs here.

Bigmack3000 commented 3 months ago

just curious if anyone was able to look into this?

Bigmack3000 commented 3 months ago

so it seems that even if you download the podcasts, instead of uploading from a folder, you'll still run into this problem after a certain number.

Bigmack3000 commented 2 months ago

Has anyone else been able to recreate this?

nichwall commented 2 months ago

I don't know if anyone has looked into it yet. I'll see if I can recreate the problem tonight

nichwall commented 2 months ago

I wasn't able to recreate the problem last night (found other issues that needed to be addressed).

Advplyr made a nice writeup of some ongoing memory issues, which may be related (without digging more into the problem).

https://github.com/advplyr/audiobookshelf/issues/3338#issuecomment-2323382459

Bigmack3000 commented 2 months ago

Thanks, i will look into that solution.

out of curiosity, when trying to replicate it, what did you do? you added a podcast with over 650 episodes?

nichwall commented 2 months ago

The only thing I have tried so far is adding "Business Wars" to a new podcast library, then going to the Find Episodes modal (the magnifying glass), selecting "Download All" and successfully downloading all episodes. I then made a new test server, added the podcast through the file system, then successfully matched the entire podcast to the RSS feed.

This may be due to using test servers and not having as much stuff in memory, though.

Edit to add: there were 590 episodes at the time of this test

Bigmack3000 commented 2 months ago

ok yea, that makes sense for business wars. when i originally brought it over from itunes, there were quite a few advertisements for other shows with it. So that has less than 650 when trimmed down. "Behind the bastards" should actually have over 650 episodes to download.

Bigmack3000 commented 2 months ago

Hey, I wanted to say that expanding the memory fixed the problem! At least up until 850 episodes. We'll see if it comes back at a certain size. Thanks!