search

DIGITALCRIMINAL / ArchivedUltimaScraper

Scrape content from OnlyFans and Fansly
GNU General Public License v3.0
946 stars 39 forks source link

It is extremely slow now #919

Open twist3dimages opened 1 year ago

twist3dimages commented 1 year ago

I will provide as much information as I can. It used to be incredibly fast and take maybe an hour to deal with thousands of files but it just hangs. I'll let it run before I go to bed and wake up and it's maybe done a few hundred. Often times hanging on renaming. What info can I give you to help?

All my files are on a network share nad it is slow regardless of what program I use to call start_us.py

Scontoni commented 1 year ago

I am having similar issues now after upgrading to the UltimaScraper version (and a brand-new installation of the latest version) The script takes 2 minutes for the script to even start, and then it hangs on the first model

Arko109 commented 1 year ago

Same here, before the script took like ~140 min to run (single thread), after updating to latest commit - I let it run 3 DAYS straight and it didn't finish. I increased the number of threads to 8, it completed once in about 800 min, but now seems even slower

Arko109 commented 1 year ago

@twist3dimages @Scontoni try the workaround in #902 , it fixed it for me

twist3dimages commented 1 year ago

@twist3dimages @Scontoni try the workaround in #902 , it fixed it for me

I increased the threads but it's always stuck on the same person. I think it has to deal with the Blacklists option not working

Arko109 commented 1 year ago

@twist3dimages @Scontoni try the workaround in #902 , it fixed it for me

I increased the threads but it's always stuck on the same person. I think it has to deal with the Blacklists option not working

I mean this one https://github.com/DIGITALCRIMINALS/OnlyFans/issues/902#issuecomment-1501225584

twist3dimages commented 1 year ago

@twist3dimages @Scontoni try the workaround in #902 , it fixed it for me

I increased the threads but it's always stuck on the same person. I think it has to deal with the Blacklists option not working

I mean this one #902 (comment)

I mainly use it in a docker container but I tried adding it anyway and it still does the same thing. This may have fixed it for me

I think it's mainly the blocklists don't work, the infinite loop being false and the timeout don't work, and when I specify only 1 model it downloads everyone