Bookmark images pulls only most recent

yellowbluesky / PixivforMuzei3

Best Pixiv plugin for Muzei 3

GNU General Public License v3.0

173 stars 15 forks source link

Bookmark images pulls only most recent #176

Closed misternem0 closed 2 years ago

misternem0 commented 2 years ago

Not sure this is an issue, or just a limitation? I pull bookmarked images and it seems to grab mostly within the most recent 50-60 or so and then repeat from there. Never seems to go back further in my bookmark history than that.

yellowbluesky commented 2 years ago

Thank you for taking the time to write about an issue.

As of the time the bookmark artwork feed feature was added, this issue is a limitation of the Pixiv API.

When I ask the Pixiv API for bookmark images (and all authenticated images in general), it responds with a JSON that contains only:

The metadata for 30 artworks
The URL to query for the next 30 artworks

The issue is that I don't know how many artworks the user has bookmarked in total, so I cannot simply just download a dozen JSON's and then choose from them. It also seems wasteful to download a dozen JSON's only to pick three artworks out of them, but this is arguable.

A workaround for now would be to never clear the image cache, Muzei tries its best to not display any old artworks. With this you should start to see older artworks being displayed.

However, the Pixiv API has undergone quite a lot of changes and had a lot of new functions added, so there may be something useful now. I will check and report back.

yellowbluesky commented 2 years ago

Progress update: I can determine the number of bookmarks a user has, which is good.

The bad news, that number is pretty useless, as there is not an easy way in the API to directly navigate to any bookmark. The only way to "move" the "window" of bookmark artworks is to adjust the max_bookmark_id query parameter when making the API request. However, this is a mystery value:

It appears to be a timestamp of some sort.
It appears to count up in centiseconds, with an epoch time sometime in late 2015 or early 2016.
This number corresponds to when the last artwork of the artwork window was bookmarked.

A workable way would be to perform a binary search on the max_bookmark_id query parameter to figure out when the first artworks were bookmarked. Now we can randomly select a max_bookmark_id query parameter and then randomly select a bookmark artwork.

This is a very un-elegant way to do it, so I'm still looking to see if there's an better way to do it.

yellowbluesky commented 2 years ago

Work on this feature is progress on the branch tracked here https://github.com/yellowbluesky/PixivforMuzei3/tree/ImprovedBookmarkFetching

yellowbluesky commented 2 years ago

This feature has been implemented. it is avaialble in version 3.4.5, downloadable here on Github or from the Google Play Store in a few hours.

misternem0 commented 2 years ago

Excellent, I have it downloaded and it's now active, I'll report back with results after some time seeing how it processes bookmarked images.

yellowbluesky commented 2 years ago

This is embarassing, I didn't do adequate testing and I released a bug in the bookmark feed option.

I have now pushed a new build onto Github and the Google Play Store rectifying this.

misternem0 commented 2 years ago

After a few days of use I can confirm that this issue has been resolved. I am seeing bookmarked images both old and new. Thank you very much!