[BUG] Unable to download Twitter media at full-resolution (4K)

[tmpfx00]

Describe the bug SCrawler does not download media at full-resolution (4K).

To Reproduce Steps to reproduce the behavior:

1. Profile URL: https://twitter.com/nasahqphoto
2. Download selected (F5)
3. Check and compare resolution/size discrepancy in File Explorer. I.e. Extract one image from SCrawler output folder, and compare against the same image downloaded in chrome & IOS. In screenshots section below I compared FTFVFwyWYAEcIz4.

Expected behavior I'd expect SCrawler in the original raw resolution (maximum attainable).

Screenshots

Release information (please complete the following information):

• OS [Windows 10]
• Architecture [x64]
• Version [3.0.0.9]
• NET.Framework version

Additional context I've only seen this issue with images, I have not tested if videos are downloaded in full.

[AAndyProgram]

Yes, I know about this problem, but no one posted a link to the profile. Thanks for the profile link. The problem has been identified. The next version will fix it if everything is ok.

[tmpfx00]

Thanks for the reply, I appreciate your hard work.

Just a heads up, not all photos on that profile are in 4K. I had to use the IOS Twitter app to determine this by holding on the image and tapping "Load in 4K" in the pop up.

[AAndyProgram]

Very interesting API! Currently, image dimensions are obtained using this function. By default, almost every image on Twitter has size options: thumb, small, medium, large. SCrawler gets these sizes and determines the best one. When this happens, the following text is appended to the image URL: ?format=jpg&name=large. As a result, we have the following URL: https://pbs.twimg.com/media/SomeImage?format=jpg&name=large. https://github.com/AAndyProgram/SCrawler/blob/f491e03812166c8d08698172daefac3f3c5f3f26/SCrawler/API/Twitter/UserData.vb#L130-L149

But it looks like every photo with the additional query 4096x4096 (https://pbs.twimg.com/media/SomeImage?format=jpg&name=4096x4096) provides a full photo. I tested it on 10 photos and each one opens at its original size.

Example:

[AAndyProgram]

I fixed it, but a few days for tests

[tmpfx00]

That is very interesting, I'm amazed how quickly you fixed it! It's great to see active devs in the community.

I don't really understand programming and API's, but would the 4096x4096 query still work for non-4K images or would the request error out?

[AAndyProgram]

I'm amazed how quickly you fixed it!

I could have fixed it sooner if someone sent me the profile URL like you did.

It's great to see active devs in the community.

Thank you 😊

I don't really understand programming and API's, but would the 4096x4096 query still work for non-4K images or would the request error out?

I tested it and yes, it also works with non-4K images. But I didn't remove the function of determining the size. I expanded it and added a special checker. So if the image is not 4K, large (or any other size) will be appended to the image URL.

[AAndyProgram]

Fixed. Please update to the latest release.

[tmpfx00]

Works like a charm! Thank you so much!

[AAndyProgram]

Thank for your help too.

BTW, if you like SCrawler, please like the program on this site: https://alternativeto.net/software/scrawler/about/.