search

datawhores / OF-Scraper

A completely revamped and redesigned fork, reimagined from scratch based on the original onlyfans-scraper
MIT License
643 stars 53 forks source link

Statistics not reset on next model #454

Closed Windoze345 closed 2 days ago

Windoze345 commented 3 weeks ago

Describe the bug

When downloading from multiple models, the statistics (count of downloaded files) don't reset when the model has nothing to download.

To Reproduce

Download from multiple models. Model 1 has 2 videos -> count shows 2 videos 0 photos Model 2 has nothing new -> count also shows 2 videos 0 photos Model 3 has 3 photos -> count shows 0 videos 3 photos ...

Expected behavior

The count should get reset on every model like before

Screenshots/Logs

[final_log.final_log:42] [wandldoelltorky][Action Download] (0 bytes) (0 downloads total [0 videos, 0 final_log.py:42 audios, 0 photos], 0 skipped, 0 failed))
[final_log.final_log:42] [] final_log.py:42 [final_log.final_log:42] [welcome.to.jezebels][Action Download] (16.88 MB) (2 downloads total [1 videos, final_log.py:42 0 audios, 1 photos], 0 skipped, 0 failed))
[final_log.final_log:42] [] final_log.py:42 [final_log.final_log:42] [wetnesday][Action Download] (16.88 MB) (2 downloads total [1 videos, 0 audios, final_log.py:42 1 photos], 0 skipped, 0 failed))
[final_log.final_log:42] [] final_log.py:42 [final_log.final_log:42] [xisabellaabrown][Action Download] (16.88 MB) (2 downloads total [1 videos, 0 final_log.py:42 audios, 1 photos], 0 skipped, 0 failed))
[final_log.final_log:43] final_log.py:43

In this case only "welcome.to.jezebels" had 1 video & 1 photo

Config

ofscraper --config ./config --username ALL --posts all --action download

System Info

datawhores commented 3 weeks ago

I'm having a hard time see this can you show a full log file?

Windoze345 commented 3 weeks ago

full log is here: https://privatebin.io/?7c03ffc8fe580857#4u97P6wct9s6qs6tCy492BQuynJrWSaTLvpaWtMX7R48 in this case only "amateuraussie" had 1 video & 4 photos and "welcome.to.jezebels" had 1 video but the statistics show for every model

tenseiken commented 4 days ago

This is still happening in 3.12.4. Redacted log here. The easiest way to see it is by looking at the "Final Results Logs" section:

 2024-09-09 14:20:00:[final_log.final_log:42]  [profile1][Action Download] (439.88 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile2][Action Download] (439.88 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile3][Action Download] (439.88 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile4][Action Download] (439.88 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile5][Action Download] (439.88 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile6][Action Download] (439.88 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile7][Action Download] (23.85 MB) (2 downloads total [1 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile8][Action Download] (23.85 MB) (2 downloads total [1 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile9][Action Download] (23.85 MB) (2 downloads total [1 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile10][Action Download] (23.85 MB) (2 downloads total [1 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile11][Action Download] (23.85 MB) (2 downloads total [1 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile12][Action Download] (23.85 MB) (2 downloads total [1 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile13][Action Download] (23.85 MB) (2 downloads total [1 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile14][Action Download] (1.01 MB) (3 downloads total [0 videos, 0 audios, 3 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile15][Action Download] (1.01 MB) (3 downloads total [0 videos, 0 audios, 3 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile16][Action Download] (2.5 MB) (7 downloads total [0 videos, 0 audios, 7 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile17][Action Download] (2.5 MB) (7 downloads total [0 videos, 0 audios, 7 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile18][Action Download] (2.5 MB) (7 downloads total [0 videos, 0 audios, 7 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile19][Action Download] (2.5 MB) (7 downloads total [0 videos, 0 audios, 7 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile20][Action Download] (2.5 MB) (7 downloads total [0 videos, 0 audios, 7 photos], 0 skipped, 0 failed))
 2024-09-09 14:20:00:[final_log.final_log:42]  [profile21][Action Download] (2.5 MB) (7 downloads total [0 videos, 0 audios, 7 photos], 0 skipped, 0 failed))

In the above, I had already run the scraper once in the same day, so there wasn't much new to get. You can see that profile1 had just a single file to download. Then, on profile1 through profile6, there was nothing new to download, so it repeated the exact same values (counts and sizes) for those profiles until it came to profile7 which had something new to download. Then it repeats those same values again for profile8 through profile13. You can see it happening again at profile14 and again at profile16.

It's not that it can't display a zero value, though. If the first profile has zero new things to download, it will say that. It just won't put a zero after anything else. Here is an example from yesterday showing that:

 2024-09-08 06:08:17:[final_log.final_log:42]  [profile1][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile2][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile3][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile4][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile5][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile6][Action Download] (3.99 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile7][Action Download] (3.99 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile8][Action Download] (3.99 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile9][Action Download] (3.99 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile10][Action Download] (402.25 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile11][Action Download] (402.25 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile12][Action Download] (402.25 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile13][Action Download] (402.25 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile14][Action Download] (456.75 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile15][Action Download] (36.22 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile16][Action Download] (36.22 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile17][Action Download] (36.22 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile18][Action Download] (36.22 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile19][Action Download] (1.93 MB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile20][Action Download] (1.93 MB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-08 06:08:17:[final_log.final_log:42]  [profile21][Action Download] (1.93 MB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
datawhores commented 4 days ago

Okay I will take a look again

DonaldTPP commented 3 days ago

Having the same problem, says the stats are the same as the user you scrape the first time and then if you choose another user it will say the scrape was finished with the exact same amount of stats as the other user beforehand and just skip the content.

tenseiken commented 3 days ago

I saw commit bd8502b referring to this issue, so I pulled the 3.12.5 docker image and did another run a few minutes ago. Looks like it's still happening:

 2024-09-10 15:58:45:[final_log.final_log:29]  Final Results Logs
 2024-09-10 15:58:45:[final_log.final_log:42]  Normal Mode Results
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile1][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile2][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile3][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile4][Action Download] (0 bytes) (0 downloads total [0 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile5][Action Download] (259.63 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile6][Action Download] (259.63 KB) (1 downloads total [0 videos, 0 audios, 1 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile7][Action Download] (38.81 MB) (4 downloads total [2 videos, 0 audios, 2 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile8][Action Download] (291.23 KB) (2 downloads total [0 videos, 0 audios, 2 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile9][Action Download] (291.23 KB) (2 downloads total [0 videos, 0 audios, 2 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile10][Action Download] (291.23 KB) (2 downloads total [0 videos, 0 audios, 2 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile11][Action Download] (291.23 KB) (2 downloads total [0 videos, 0 audios, 2 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile12][Action Download] (291.23 KB) (2 downloads total [0 videos, 0 audios, 2 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile13][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile14][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile15][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile16][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile17][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile18][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile19][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile20][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
 2024-09-10 15:58:45:[final_log.final_log:42]  [profile21][Action Download] (14.71 MB) (1 downloads total [1 videos, 0 audios, 0 photos], 0 skipped, 0 failed))
datawhores commented 2 days ago

Can you try with 3.12.7

Windoze345 commented 2 days ago

Seems to work for me

tenseiken commented 2 days ago

Yeah, I meant to post earlier. Looks like it's fixed in 3.12.7.