Cannot get users' galleries on DeviantArt

HanatakeYurii commented 7 years ago

Hi,

I keep trying to figure out why when I try to get an user's gallery Nekopaw doesn't display anything, like there was no image in the gallery, which is false.

I tried with many different galleries and never get anything. Had DA changed something in the site structure that prevents Nekopaw to grab them?

Please talk if you have an idea or encounter same issue (or not, it should be me in that case).

UP: Can't this be related to infinite scrolling and/or css classes of the links? ... I'll dive in Nekopaw source code a bit...

UP2: Classes are fine, something else then

JarieSuisen commented 7 years ago

I can't even log into DA through Nekopaw, though fine in a browser.

HanatakeYurii commented 7 years ago

Now that you say it, i tried, i can't as well

I tried to make little python script to connect and it's working well, so I assume there were some changes in DA that broke NP login.

catgirlfighter commented 7 years ago

Login issue is known. If you mean https handshake issue, of course. openssl lib with Indy gives it, and I have no idea, how to fix it without changing Indy to something else (and it will be hell of a task).

Yes, it can be related to infinite scrolling, and change of template overall. I'll try to go through issues this week.

HanatakeYurii commented 7 years ago

Ow thanks.

Sorry, I didn't got what you meant by "https handshake", I'm not very used to crawler scripts. But I understand what Indy is for, and yes replacing it would simply be hell. too many script to change.

About infinite scrolling, I found some interesting thing in my attempts to make a little script getting user's library:

first if you're connected and you set your config to show gallery "per pages" instead of scrolling, I'm able to get nearly all library (using offset)
offset can be used even with default config, though links for pages (using offset) are hidden
it's possible to get the number of pictures from user's stats (number of deviants shown on user's profile includes other things than pictures likes stories and so)
it's better trying to get pictures in the gallery from the "All" page since it shows every pics even if in albums/folders

I assume you knew most of it, but I hope some can help you find out what to do. Also thanks for making Nekopaw that great

catgirlfighter / nekopaw

Cannot get users' galleries on DeviantArt #369