Can't download (some) earlier images (server response is HTTP 404 Not Found)

[shakraw]

Hi, I'm trying to build up a demo application in Python3.5.2+PyQT5 (on MacOS platform atm) using WebWhatsapp-Wrapper to be able to choose images from a chat, select them and download selected images all at once.

I was able to do all that, but I found a strange use case I don't know how to solve. So if someone can help me I would be glad of that. I can also try to make a PR for this if I can find the right pointers to solve it.

Let me explain what's going on: I'm able to show received images in a chat as thumbnails. I can also use the self.__driver.chat_load_earlier_messages(chat.id) and then reload the images list to show earlier images.

From these earlier images there are some not available (non every images, just some of them): you see these images in WhatsApp having a blurry thumbnail and "download" overlay button . If I try to download them using WAPI.downloadFile method the request hangs because the server side HTTP status code is 404 Not Found.

If you click this download button in Whatsapp the image is downloaded smoothly. The problem is that I cannot find a way to reproduce this with this wrapper: trying to call WAPI.downloadFile on these images I always get HTTP 404... and so I'm not able to download them.

Does anyone encountered this behaviour before? Do you have any suggestion on how to solve it or any pointers to give me? Thank you in advance.

[tulssinep]

Hey, can you check whether those earlier images are available in the "real" Whatsapp Web instead of from the phone?

[shakraw]

Hi @tulssinep, in Whatsapp Web I see these earlier images blurry with a download button overlay (the same as in Whatsapp Desktop and on smartphone). Then, if I click on the download button, the image become available (as the other recent images) after some processing and I am able to see it (i.e. I can download them).

[anirudhrata]

Hi, I just found out if you call resumeUpload method on the message object, the 404'd images will come back, and you can download them as usual. Hope this helps!

[anirudhrata]

Additional info:

When you click the download button or call resumeUpload, it appears that the media is again uploaded from the phone to Whatsapp (probably due to their recent policy changes around media retention) and the clientUrl of the message object is regenerated.

The domain where they are stored is now "mmg-fna.whatsapp.net" rather than "mmi*.whatsapp.net" which don't resolve to an IP currently.

If the media doesn't reside on your phone, there is no way you are getting the 404'd media (except for the thumbnail). You can see the status of your media at messageObj.mediaData.mediaStage. If it is RESOLVED then you can get the media, if it is NEED_POKE, you need to call resumeUpload and if it exists on your phone then WW will fetch it and mark as RESOLVED.

[cprcrack]

@anirudhrata I was just working on this very same issue and discovered about NEED_POKE as well when the media has not been preloaded in the WhatsApp Web client.

However in my case the msg.clientUrl is already available and doesn't change when calling msg.resumeUpload(), but still a call to resumeUpload() is needed for the URL to become valid (otherwise you get the 404).

Are you aware of any way to make the clientUrl file downloadable without calling resumeUpload()? I don't want the whole image to be loaded in the client.

I have experimented with msg._mediaObject.downloadPromise.control.poke() and msg._mediaObject.downloadPromise.control.waitForPoke() but they don't seem to have any effect. Also there's msg.resumeRemoteUpload(), msg.downloadMedia() and msg.forceDownloadMedia() but all have a similar effect to resumeUpload() regarding this matter.