cwoac / thingy_grabber

Script for archiving thingiverse things
MIT License
39 stars 11 forks source link

Follow next links and collect all items in Groupings #21

Closed unknownlamer closed 3 years ago

unknownlamer commented 3 years ago

Greetings.

I made an attempt to fix issue #20; this just loops over the next Links sent by the API response until all pages have been visited. Seems to work fine here (if a bit brute force). I didn't add anything to handle pagination in other responses; at least for now it looks like Thingiverse isn't paginating anything else (I did not find a user profile with more than 30 collections to test the code that finds user collections however).

Flyingvette commented 3 years ago

Here is one with over 2,500 things and over 60 collections: https://www.thingiverse.com/syzguru11/designs

This one has 400 designs and 26 collections: https://www.thingiverse.com/walter/designs

Lmk if I can help you find more. I'll try to test this version out tonight and report back.

unknownlamer commented 3 years ago
$ ./thingy_grabber.py collection syzguru11 timeWarner
requesting:https://api.thingiverse.com/collections/10329199/things/?access_token=***
Found 16 things.
Downloading 0 thing(s).
Downloading thing 0 - ThingLink(thing_id=3711730, name=' Dr. who telephone booth (quick and dirty)', api_link='https://api.thingiverse.com/things/3711730')
...

Confirmed the list of collections for a user is not paginated at this time, and there was no problem finding and fetching a random collection from page 4 of this user's collections list. Double checked manually as well: $ curl -siL https://api.thingiverse.com/users/syzguru11/collections/all?access_token=$(cat api.key) | less

Flyingvette commented 3 years ago

Unfortunately I was using the compiled executable and don't know how to do that manually. I'll have to wait until the master absorbs the update 👍 Thank you for looking at this! Excited to give it a go.