search

Hashirama / OFDL

Onlyfans content downloader with graphical user interface
231 stars 40 forks source link

Incorrect Sizes #14

Closed DonaldTPP closed 4 years ago

DonaldTPP commented 4 years ago

The last few days I've been having errors when scraping new content for somebody. I have to scrape multiple times and most of the times I restart it and end up with this. It also takes forever to scrape at an average speed of 2.1 to 3.6 kbps. Is there a way to get it to go faster?

Messages: 0 Highlights: 0 Images: 2 Videos: 0 Stories: 0 Archived: 0 Audio: 0 Total size: 220B

Files to be downloaded: 2 Download Size: 220B

Hashirama commented 4 years ago

Do you have the incorrect file count with one model or more than one and who?

DonaldTPP commented 4 years ago

The scraper seems to be struggling over the last three days. Before it was faster that I do the first initial scrape one at a time and then I would do one or two models at a time and it would do it in about 3 to 5 minutes, now it takes hours just for two at a time and then when it's done scraping it gives the wrong size amounts. Multiple times.

DonaldTPP commented 4 years ago

Still having the same problems even after a fresh folder and just using my config.json

laavenderbxby: Messages: 0 Highlights: 0 Images: 0 Videos: 1 Stories: 0 Archived: 0 Audio: 0 Total size: 110B

Files to be downloaded: 1 Download Size: 110B

DonaldTPP commented 4 years ago

laavenderbxby: Messages: 0 Highlights: 0 Images: 0 Videos: 1 Stories: 0 Archived: 0 Audio: 0 Total size: 110B dontslutshame: Messages: 1 Highlights: 0 Images: 0 Videos: 1 Stories: 0 Archived: 0 Audio: 0 Total size: 220B

Files to be downloaded: 3 Download Size: 330B

DonaldTPP commented 4 years ago

Just tested by subscribing to a completely new model that I haven't subscribed to before, same problem.

: Messages: 1 Highlights: 0 Images: 553 Videos: 82 Stories: 0 Archived: 0 Audio: 1 Total size: 68.32KB Files to be downloaded: 637 Download Size: 68.32KB
DonaldTPP commented 4 years ago

It worked 7 tries later. The scraping and collecting of links is faster than usual but this time around it collected them properly.

: Messages: 1 Highlights: 0 Images: 553 Videos: 82 Stories: 0 Archived: 0 Audio: 1 Total size: 9.56GB

Files to be downloaded: 637 Download Size: 9.56GB

Hashirama commented 4 years ago

In the script there is code to check whether the content length of the files is available and if not the size is kept as zero.

If some of the headers aren't returning content lengths then this might explain the download size being wrong but the "files to be downloaded" being right.

Either way it should still download all 637 files, the "download size" only helps the progress bar and nothing more. The program goes by the "files to be downloaded" count.

DonaldTPP commented 4 years ago

Can confirm it downloads all the files but they're basically empty.

DonaldTPP commented 4 years ago

Messages: 0 Highlights: 0 Images: 257 Videos: 69 Stories: 0 Archived: 1 Audio: 0 Total size: 35.13KB

Files to be downloaded: 327 Download Size: 35.13KB

https://ibb.co/7VG73y8 https://ibb.co/LdtwfQq

DonaldTPP commented 4 years ago

And, just like clockwork, the 6th time, it works.

Messages: 0 Highlights: 0 Images: 257 Videos: 69 Stories: 0 Archived: 1 Audio: 0 Total size: 1.70GB

Files to be downloaded: 327 Download Size: 1.70GB

Hashirama commented 4 years ago

Ah alright, I'll look into it again.