Hangs on scraping

[NicoVisagie]

The program runs perfectly, and it identifies the added content that has not been scraped before. It also shows when you click on the content, but there is some sort of delay from the time you click on the content and when it displays. When you push the scrape button, it shows you all the relevant content it as found, but it is when you start the actual scraping process that it hangs. It shows you the little window that it is doing something, but shortly thereafter, everything stops working and nothing more happens. The only way out is to press CTRL ALT DEL and then end the process in Windows. Restarting the application sometimes fixes the bug, but not always.

I have now loaded V2.10.2 and that works perfectly. No hanging at all.

Just thought you would like to know...

MediaElch Version:

• 2.10.4 stable

Operating System:

• Windows 11 (Build 22621)

[bugwelle]

Thank you for reporting this issue!

I will have to debug this. I don't know what change is responsible, yet.

[dsiggi]

Same problem here on gentoo linux. MediaElch build from source.

[bugwelle]

No success, yet. Is it any specific scraper or all movie scrapers?

What Qt version do you use?

Regards, Andre

[GarryS543]

I am experiencing same thing with two of the adault movie scrapers. it apears thatthe Poster and plot fails to be retrieved. Unsing 2.10.4 on win10 machine. QT vertion 6.5.1

all other movie scrapers apear to work well.

[dsiggi]

Hi, QT version is 5.15.10. I'm using the "The Movie DB" scrapper. Today the program is working normally.

[Stis]

MediaElch v2.10.4 on Win10, first scraping of movies today with this version of MediaElch. Whatever the scraper, hangs... With Shows, no problem.

Dit a last try just now, liike 10 min after, everything's working as expected. ^^'

[bugwelle]

Thank you all for confirming this bug. Seems unrelated to the Qt version (which is a good thing) and also unrelated to the operating system.

The only thing I noticed is a short period after clicking "Scrape" where MediaElch is unresponsive. But that is only <5s (which isn't good, I know, but not the freeze you seem to be seeing).

To summarize:

• this happens when scraping a single movie
• happens with The Movie DB, but probably also other scrapers
• the scraped preview is fast, there is no lag when clicking on a movie in the list

Is that correct? Also, does MediaElch actually freeze for >5min?

Thank you all for being patient.

Regards, Andre

[bugwelle]

Ok, kind of got it reproduced. But only for certain movies. In my case it was "Verblendung" (German title), which took >10s until it was responsive again. And I have quite the beefy computer.

I think I need to profile MediaElch (performance measurements), but my current guess is (after a quick debugging session), that the Ignore articles when sorting feature is poorly implemented.

@ all: Could you try following and tell me if it's gotten slightly better?

Go to MediaElch->Settings->Uncheck "Ignore articles when searching"

Regards, Andre

[bugwelle]

I was able to reproduce it on my Windows machine. I really have no idea why it just froze... However, this seems to be related to https://github.com/Komet/MediaElch/issues/1640

A new nightly for Windows and Linux is now available (or in the next 30min): https://mediaelch-downloads.ameyering.de/snapshots/

Now, MediaElch doesn't freeze, but should be at most 2s unresponsive (not good either, but does not freeze).

It would be awesome to get feedback from you.

[GarryS543]

Just got back home to test again 2.10.4No more hang on the adult dvd empire scraper.It still does not return the poster and description. I suspect the website made changes.Garry sSent from my iPhoneOn Oct 1, 2023, at 10:25 AM, Andre Meyering @.***> wrote: I was able to reproduce it on my Windows machine. I really have no idea why it just froze... However, this seems to be related to #1640 A new nightly for Windows and Linux is now available (or in the next 30min): https://mediaelch-downloads.ameyering.de/snapshots/ Now, MediaElch doesn't freeze, but should be at most 2s unresponsive (not good either, but does not freeze). It would be awesome to get feedback from you.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

[GarryS543]

Just tried dev 2-10.5In adult DVDFab empire, the preview actually worked, retrieved descriptions and poster. clicked scrape status bar froze at half way mark it’s been 2 minutes plus and still frozen.Thank you,Garry sSent from my iPhoneOn Oct 1, 2023, at 10:25 AM, Andre Meyering @.***> wrote: I was able to reproduce it on my Windows machine. I really have no idea why it just froze... However, this seems to be related to #1640 A new nightly for Windows and Linux is now available (or in the next 30min): https://mediaelch-downloads.ameyering.de/snapshots/ Now, MediaElch doesn't freeze, but should be at most 2s unresponsive (not good either, but does not freeze). It would be awesome to get feedback from you.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

[bugwelle]

Thanks. I'll consider this issue fixed, but will leave it upon for others to reply as well. :-)

[ScoobyD00M]

For me it still hangs when using MovieDB with the latest nightly for Windows. But only on my Laptop, on my Desktop PC everything is working. Both are using Windows 11 and same configuration for MediaElch. The problem doesn't occur on version 2.10.2 stable though.

[Stis]

Well well well... Created a dummy file tetris.avi 2.10.4 took almost a minute but scraped it. 2.10.2... had to be killed. 2.10.5 642ff197 got it perfectly. 2.10.5 3f02c75f... had to be killed.

Did jumanji.mp4 Every version worked flawlessly.

Tested a tetris.mkv Every version workeds flawlessly.

By curiosity, re-did a tetris.avi Every version worked... but not as smooth.

Finally did Gladiator.mkv, Jumanji.avi, tetris.avi Everything was ok.

Each time it was an empty file of 0 octet.

Will retest later...

[GarryS543]

Used 2.10.5 642ff197 last night and is working great, can even do batch scraping with zero problems

Thank you,

Garry S

From: Stis @.> Sent: Wednesday, October 4, 2023 7:56 AM To: Komet/MediaElch @.> Cc: GarryS543 @.>; Comment @.> Subject: Re: [Komet/MediaElch] Hangs on scraping (Issue #1648)

Well well well... Created a dummy file tetris.avi 2.10.4 took almost a minute but scraped it. 2.10.2... had to be killed. 2.10.5 https://github.com/Komet/MediaElch/commit/642ff197f451b34dfdbde5256fec81ffea630757 642ff19 got it perfectly. 2.10.5 https://github.com/Komet/MediaElch/commit/3f02c75ff040361fafb3a1542b553c22628c3ed9 3f02c75... had to be killed.

Did jumanji.mp4 Every version worked flawlessly.

Tested a tetris.mkv Every version workeds flawlessly.

By curiosity, re-did a tetris.avi Every version worked... but not as smooth.

Finally did Gladiator.mkv, Jumanji.avi, tetris.avi Everything was ok.

Each time it was an empty file of 0 octet.

Will retest later...

— Reply to this email directly, view it on GitHub https://github.com/Komet/MediaElch/issues/1648#issuecomment-1747018475 , or unsubscribe https://github.com/notifications/unsubscribe-auth/BALHF7UD4JDJ6JXAZUPYCZDX5V2G7AVCNFSM6AAAAAA43NP6Q6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTONBXGAYTQNBXGU . You are receiving this because you commented.Message ID: @.***>

[Stis]

New test. 2.10.4 was perfect. 2.10.2 was long. 2.10.5 needed to be kill.

I just don't understand and think I'll let you do the good job. ^^'

[bugwelle]

Thank you all. Ok, I really don't get how https://github.com/Komet/MediaElch/commit/642ff197f451b34dfdbde5256fec81ffea630757 worked perfectly whereas https://github.com/Komet/MediaElch/commit/3f02c75ff040361fafb3a1542b553c22628c3ed9 had to be killed. The latter commit is only used when movies are reloaded. Not when scraping a movie.

---

@ScoobyD00M Does one have an SSD and the other a HDD?

[bugwelle]

Ok. This is getting ridiculous :smile:

It works fine if compiled with clang+asan, but crashes if compiled with gcc.

After profiling it, there is nothing suspicious with clang, but on gcc, I can clearly see that the amount of "movie changed" signals is one of the reasons everything stops working.

But without the debugger enabled (and after some further fixes), everything works just fine and with the debugger, it stops working again.

With 72ba334fd1e455168e1de329ed95ca9f220d7022 it seems to get better. Could you try that one out? A new nightly will be available within the next hour. I will delete all old snapshots to avoid confusion.

[ScoobyD00M]

@ScoobyD00M Does one have an SSD and the other a HDD?

Both got NVMe, but on the Laptop the CPU comes pretty much to the limit. I killed the process after 3-4 minutes. I also noticed that MediaElch freezes briefly when I edit the title of a movie.

Edit: Todays nightly did the job. Only the progress bar was stuttering a bit and the program became unresonsive for a few seconds.

[bugwelle]

Edit: Todays nightly did the job. Only the progress bar was stuttering a bit and the program became unresonsive for a few seconds.

Really? It actually works now? If that stays true for a few days, I will publish a new release. The few seconds are expected and I know why that's the case, but it's nothing I can quickly fix.

Thank you very much for your response. :)

[ScoobyD00M]

Thank you very much for maintaining. ;)

I tested a few more movies. 95 movies on my network drive in a row without any problems and a few single files... no problems so far.

[thoroc]

I've had the same issue intermittently for the past few release (didn't keep track tbh). I am currently on the latest main release (2.10.4).

[bnerickson]

I had this issue when scraping a film with TMDb on 2.10.4. Running the nightly (https://mediaelch-downloads.ameyering.de/snapshots/Windows_10_or_later/MediaElch_win_Qt6_2.10.5-dev_2023-10-15_00-36_git-master-62d5356c.zip) fixed the issue for me.

[bugwelle]

Thank you all. I plan to release a new version this weekend. I'll consider this issue fixed. :-)

(@thoroc or did you run into issues in the latest nightly version?)

[thoroc]

I'll have to try the new version. I wasn't using nightly.