search

Kagami / tistore

:camera: Tistory photo grabber
24 stars 10 forks source link

Fix aborted downloads #5

Open Kagami opened 6 years ago

Kagami commented 6 years ago

See https://github.com/Kagami/tistore/issues/4#issuecomment-411875935 Might be caused by high concurrency level and expiration of default aria's download timeout.

Kagami commented 6 years ago

@Lumerica, yea, I've seen few aborted downloads while testing too, but thought it were just dead links and left unattended. But you're right, it might be caused by high concurrency level too.

Can you reproduce issue with concurrency set to 1?

Lumerica commented 6 years ago

Thanks, I am now trying with: beautifulmirinae.tistory.com which I had lots of aborted downloads before I will let you know tomorrow, because its going to take some time to download.

there was also a tistory site that took me 2 full days with 16 threads set about 75 aborted downloads.

Lumerica commented 6 years ago

the download hasnt finished yet after 8 hours looks like it needs another 7 hours. But so far no aborted downloads just that it takes long time.

Kagami commented 6 years ago

Any news?

Lumerica commented 6 years ago

Oops I thought that concluded the test, but I would say its OK, just that it took 8 hours and would have taken 15 hours for 1 site.

In the 8 hours 5730 files have been downloaded successfully without any aborted downloads.

Kagami commented 6 years ago

Nice. I will try to increase timeouts then and check if that helps.

Lumerica commented 6 years ago

thank you for your hardwork :)