search

TumblThreeApp / TumblThree

A Tumblr and Twitter Blog Backup Application
https://TumblThreeApp.github.io
MIT License
623 stars 75 forks source link

Blog unqueues after a short time of doing nothing #485

Closed DaymareChimera closed 1 year ago

DaymareChimera commented 1 year ago

Describe the bug I'd been spending a few days downloading a Twitter blog (authenticated), and all was going well up until yesterday. I tried to queue up the blog again and resume the download process, but after about 30 seconds it evaluated 20 post URLS and then automatically un-queued itself. A few times, I managed to get it to successfully download a few posts, but now it does the above behavior and downloads nothing new. Checking the blog's online status hasn't resulted in any blog offline errors while this has been happening. This continues happening after updating to the most recent version of the application.

Steps to reproduce the behavior:

  1. Enter a Twitter URL and click Add Blog.
  2. Select the blog from the list and right-click.
  3. Click "Enqueue selection."
  4. Click the Download button.
  5. Go to the queue tab.
  6. See error

Expected behavior I expected it to continue evaluating post urls and downloading new posts until the connection timed out or it finished. Previously, the blog had stayed in the queue even when encountering connection timeouts, and would wait until it was able to start downloading again.

Screenshots image image

Desktop (please complete the following information):

Additional context OS running in VirtualBox. Still a problem after restarting the OS, restarting VirtualBox, and restarting my computer.

thomas694 commented 1 year ago

Please look up the value after the field "LastId" in the .twitter file inside the Index folder.

DaymareChimera commented 1 year ago

It's this: 1721225519453077824

thomas694 commented 1 year ago

I already assumed. The whole blog was already scanned once. From now on it only scans new posts since the last successful scan. After a few hours or a day, it may download a couple of (new) posts. If you add it to the queue again before the blog has new posts, it just finishes after a short while. You should be able to check that in your explorer folder by sorting on modified date.

DaymareChimera commented 1 year ago

Oh okay, thank you! I'll check that out and report back!

DaymareChimera commented 1 year ago

Okay, update: I tried with a different account, but ran into the same issue where I only got ~6k posts off an account that Twitter shows as having ~81k posts. Interestingly, this only seemed to start after I checked "Download replies" in the Details section after it started running. I deleted the files downloaded, tried again with checking the box before I queued it up, and it worked fine.