bbolli / tumblr-utils

Utilities for dealing with Tumblr blogs, Tumblr backup
GNU General Public License v3.0
667 stars 124 forks source link

Saving my tumblr's likes produces an endless loop #235

Closed twilight-flower closed 1 year ago

twilight-flower commented 1 year ago

As of early January—some time between 10:00 EST on the 4th and 10:00 EST on the 5th—saving my tumblr's likes no longer works. After going through all my actual likes collecting all the posts (filling up the media, and posts directories in up-to-date fashion), it then proceeds to loop infinitely through attempts to retrieve nonexistent posts (e.g. posts #1100-1149 out of 1007 expected, then #1150-1199, and so forth), eventually running into the tumblr API's rate limit, never actually terminating and putting together the HTML index, until I ctrl-C out.

A minimal invocation which will successfully reproduce the issue is:

py -2 tumblr_backup.py -l moonlit-tulip

...however, it's not universal across blogs. If I run the same post on a different blog:

py -2 tumblr_backup.py -l gogcom

...the backup works fine, without issue.

(I have tested whether unliking the post I liked on the 4th—this one—works to make backing up my likes function properly again. It doesn't. So it's not as simple as "one particular post somehow broke everything and removing it from my likes will unbreak things", although it wouldn't shock me if liking that post were still somehow related, given the timing.)

All of the above has been done with the most recent version of tumblr-utils (caa912f8bf49237c5aaf0150510998b101ff4a67), running on Windows 10 with Python 2.7.18.

bbolli commented 1 year ago

HI, thanks for this bug report. What is the py -2 command that you use to start the script doing?

twilight-flower commented 1 year ago

py is the Windows Python launcher. The -2 flag indicates that I'm having it use my system's copy of Python 2 rather than Python 3. The actual path of the binary being invoked by py -2 is C:\Programs\Python27\python.exe, and everything after that invocation is passed into that binary, so e.g. py -2 --version runs the same as python --version would on Linux, py -2 -i example_filename.py runs the same as python -i example_filename.py would on Linux, et cetera,

(If I run the launcher without the -2 flag, just as py, then it instead goes to my Python 3.10 binary, since that's my default one for normal day-to-day use. And py -3 does the same.)

bbolli commented 1 year ago

On moonlit-tulip, the API keeps returning the same 7 posts (likes 1001-1007), even though the request specifies an offset >= 1050. On gogcom, the API cleanly returns an empty list starting at offset 50. I wonder if there's a maximal offset of 1000. We'll see once you have more than 1050 likes...

For now, I'll break the loop as soon as I see the same list of posts as in the previous request.

bbolli commented 1 year ago

See also #219.