Open sealabr opened 7 years ago
I too have same problem.
I have the same problem, looks like a problem where whatsapp changed their method..
Just solved part of the issue:
The
If you make the needed changes in iq_requestupload.py and iq_requestupload_result.py (replace "media" with "encr_media", you can get to the upload step.
I could not, however, get the upload to actually work. Server always gives 400 error.
Maybe the server checks if the image is actually encrypted?
Gotta check some of the forks which can actually decrypt the image to see if we can figure out a procedure for encryption.
I'm trying to get the correct POST format from the decompiled java source, with no success.
If anyone has any idea how to proceed, I'd be immensely grateful.
Glad to hear that someone is trying to do something, now I know that is needed to send encryped media, instead of plain text media, as lguardi commented on
Basically, you need to implement the inverse of the send procedure:
- Generate a random key for encryption
- Encrypt media and upload to the server
- send encrypted url to recipient and the encryption key in a protobuf message
I'm looking forward to understand more about encrypted protocol in yowsup, to actually do something about send encrypted media protocol, I will be glad to any help given.
@etinin , i follow you to change "media" to "encr_media", got the upload URL, but then got this post & response:
POST: https://mmg.whatsapp.net/u/f/FIcI1bQcJGbRXaGee0_cSllrJNoABVRrIxaEqA/AkObQlFqUS5DA21KPJHyYOnL47WKzVMSGN8Jf0SaiBE8 Content-Type: multipart/form-data; boundary=zzXXzzYYzzXXzzQQ Host: mmg.whatsapp.net User-Agent: WhatsApp/2.16.11 S40Version/14.26 Device/Nokia-302 Content-Length: 4880
HEADER & FOOTER: --zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="from"
6281315318608@s.whatsapp.net --zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="file"; filename="d6d02600f4f65c6733303c90a821800e.jpg" Content-Type: image/jpeg
RESPONSE: HTTP/1.1 400 Bad Request Content-Type: application/json Date: Sat, 15 Jul 2017 17:10:05 GMT Connection: close Content-Length: 0
It seems enc image is send in the protobuf message, like when extracted image from receiving image. I think @jlguardi , can help give us some more clues.
I'm currently short of the required time, but if someone could take a look at jiguardi's fork, this might be solved. As I understand it, his fork has the decryption method properly implemented.
https://github.com/jlguardi/yowsup/tree/master/yowsup
In a parallel note, maybe we could try uploading a pre-encrypted image file? As I understand, the server should probably accept it, since it doesn't really have the key. But, again, I haven't studied the proper code.
It seems you just have to send yourself an encrypted image and check the specifics of the AES encryption being employed.
I've tried intercepting the whatsapp app requests to solve this issue but it's harder than I believed. The APP both on iPhone and Android seems to ignore proxy settings entirely when it can connect straight to the internet. And even some requests that get through the proxy are closed by Whatsapp when the app sees the selfsigned SSL certificate (even though the cert is properly added to the system.
Anyone has knowledge any reliable procedure for intercepting requests? Med school is keeping me way too busy to keep working blindly.
I made some modifications, to send back received image with enc, But recipient only got this message in WA instead of the image: Waiting for this message. This may take a while. Learn More
This my debug log:
[message to="62851008xxxx@s.whatsapp.net" type="media" id="1500356450-2"] [enc type="msg" mediatype="image" v="2"] 3! �ƕ�u����[ -�J�,��q���z�/�6wE"�� �MQ�H�x�Y�6��� .....BLABLA HEX:330a2105845ae2d89594e ...BLABLA [/enc] [/message]
2017-07-18 12:40:51,414 [MainThread ] [DEBUG] rx: [ack t="1500356451" from="628510080xxxx@s.whatsapp.net" class="message" id="1500356450-2"] [/ack]
2017-07-18 12:40:52,778 [MainThread ] [DEBUG] rx: [receipt type="retry" from="628510080xxx@s.whatsapp.net" id="1500356450-2" t="1500356452"] [retry count="1" t="1500356451" id="1500356450-2" v="1"] [/retry] [registration] ('+ê HEX:28272bea [/registration] [/receipt]
2017-07-18 12:40:52,779 [MainThread ] [DEBUG] tx: [ack to="628510080xxxx@s.whatsapp.net" type="retry" id="1500356450-2" class="receipt"] [/ack]
@bahtiarp, you've got to properly encrypt the image and send the encryption key. How did you get to this point? Are you uploading the image or just resending something you received?
Could you please send the patch?
@etinin
This is how to encrypt:
def encryptImg(img, refkey): img = pad(img) derivative = HKDFv3().deriveSecrets(binascii.unhexlify(refkey), binascii.unhexlify("576861747341707020496d616765204b657973"), 112) parts = ByteUtil.split(derivative, 16, 32) iv = parts[0] cipherKey = parts[1] cipher = AES.new(key=cipherKey, mode=AES.MODE_CBC, IV=iv) imgEnc=cipher.encrypt(img) return imgEnc[:-6]
refkey = binascii.hexlify(os.urandom(112))
Where should this be? In mediauploader.py after "stream = f.read()" ? I got this error: global name pad is not defined. Should I also change the media-tag to encr_media ?
@dietzi try this
def pad(self, s):
return s + ((16-len(s) % 16) * '{')
Here is my diff: https://pastebin.com/mWvjhSUX Now I get "json data not found" at mediauploader.py line 160
try this:
after ssl_sock.write(bytearray(fBAOS.encode()))
dataux = ssl_sock.recv()
lines = dataux.decode().splitlines()
@dietzi
If you see the log, you got this when you upload the data:
RESPONSE: HTTP/1.1 400 Bad Request Content-Type: application/json Date: Sun, 16 Jul 2017 13:24:23 GMT Connection: close Content-Length: 0
(so you dont get the json data, as expected by the next processed).
I've checked from wa web version (chrome).
I see the header when uploading encr image is diffrent from yowsup mediauploader done.
yowsup mediauploader.py version for Header POST:
POST https://mmg.whatsapp.net/u/f/Gqp_7Z4ZN1PsLrg8b3xHGllraQYABVRvM1QKAg/AkObQlFqUS5DA21KPJHyYOnL47WKzVMSGN8Jf0SaiBE8 Content-Type: multipart/form-data; boundary=zzXXzzYYzzXXzzQQ Host: mmg.whatsapp.net User-Agent: WhatsApp/2.16.11 S40Version/14.26 Device/Nokia-302 Content-Length: 4892
--zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="to"
628510xxxxxx@s.whatsapp.net --zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="from"
628510xxxxxx@s.whatsapp.net --zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="file"; filename="AkObQlFqUS5DA21KPJHyYOnL47WKzVMSGN8Jf0SaiBE8.enc" Content-Type: image/jpeg
Response from Whatsapp: HTTP/1.1 400 Bad Request Content-Type: application/json Date: Sun, 16 Jul 2017 13:24:23 GMT Connection: close Content-Length: 0
Whatsapp WEB (chrome) version for Header POST:
------WebKitFormBoundary83mOQO9W0A1qhgX7 Content-Disposition: form-data; name="hash"
xxfuy7CpXoeRjSI321m74C8sXG/u3Bls8niwzzdGxuE= ------WebKitFormBoundary83mOQO9W0A1qhgX7 Content-Disposition: form-data; name="refs"
UL9-EipnOTCQuDjlaN-C5_pwW5k= p64bypqos9A0WxZrNLYkUOg2oZ4= ------WebKitFormBoundary83mOQO9W0A1qhgX7-- Content-Disposition: form-data; name="file"; filename="blob" Content-Type: application/octet-stream
Whatsapp WEB (chrome) RESPONSE: {"type":"encrypted","mimetype":"application/octet-stream","size":"7034","filehash":"xxfuy7CpXoeRjSI321m74C8sXG/u3Bls8niwzzdGxuE=","url":"https://mmg-fna.whatsapp.net/d/f/AsL21rfdOsW-bDtnaMAHOcSsmWgt2gjQAtCfwiELT_Mj.enc"}
i've checked that this: xxfuy7CpXoeRjSI321m74C8sXG/u3Bls8niwzzdGxuE= is the b64Hash of image file.
While this two (Content-Disposition: form-data; name="refs"): UL9-EipnOTCQuDjlaN-C5_pwW5k= p64bypqos9A0WxZrNLYkUOg2oZ4= I still dont know how to genereate.
But although we got the correct response from whatsapp when uploaded, we have totally have to change the next process in yowsup. because it's diffrent from old responses that yowsup processed.
@bahtiarp The refs is generated using the send number and recipient number with encripty:
https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/crypto-js.min.js
https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/sha256.min.js
https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/enc-utf8.min.js
https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/enc-base64.min.js
https://cdnjs.cloudflare.com/ajax/libs/crypto-js/3.1.9-1/lib-typedarrays.min.js
e = "55628226****@c.us"
for (var t = e.length, n = [], r = 0; t > r; r++)
n[r >>> 2] |= (255 & e.charCodeAt(r)) << 24 - r % 4 * 8;
a = new Uint8Array();
b = CryptoJS.lib.WordArray.create(a)
var t = CryptoJS.HmacSHA256(CryptoJS.enc.Utf8.parse(e), b);
t.sigBytes = 20
t.clamp()
var e = t.toString(CryptoJS.enc.Base64)
alert(e)
console.log(e)
If I add "hash" to the header with a random key then I get "unsupported media type". I think if we could get the hash-code then we will be able to send media. Does anybody know about to generate the hash?
@pabloruan0710 using your js script i still cannot get the correct refs;
from: 62851xxxxxx to: 62895330xxxxx
refs should be:
UL9-EipnOTCQuDjlaN-C5_pwW5k= qohsee679xXFE8N_aXqB40PLX1U=
can you try to get the same result with your js script?
@bahtiarp This numbers ref is random Cripty, not is same number, check post with sniffer in Whatsapp web.
I can Send, and received this response:
Content-Type: multipart/form-data; boundary=zzXXzzYYzzXXzzQQ
Host: mmg.whatsapp.net
User-Agent: WhatsApp/2.16.12 S40Version/14.26 Device/Nokia-302
Content-Length: 41905
HTTP/1.1 200 OK
Content-Type: application/json
Date: Tue, 08 Aug 2017 19:19:51 GMT
Connection: close
Content-Length: 221
{"type":"encrypted","mimetype":"application/octet-stream","size":"41517","filehash":"Ll9P92iz1ERZHgM1pSflrGT0UU6RUEzU4fTAE95JkA8=","url":"https://mmg-fna.whatsapp.net/d/f/AoEGRkSJG36LMUwEjvkGtUyjcNLUq8ftLcqu2Xf9EMZt.enc"}
However didn't received image in recipient number
@pabloruan0710 Have you sent ProtocolMedia message to the recipient? Or you have just uploaded the file.
@pabloruan0710 What did you do with refs? Did you encrypt sender and recipient respectively?
@pabloruan0710 great you've succeed to upload the image and get the response. (i still figuring out how to get the "refs" correct. unless you show the code :) )
The next step is like @jlguardi said, you have todo some modification in protocol media (toProtocolTreeNode), so the message will send in the following protobuf format:
[message to="62851008xxxx@s.whatsapp.net" type="media" id="1500356450-2"] [enc type="msg" mediatype="image" v="2"] 3! �ƕ�u����[ -�J�,��q���z�/�6wE"�� �MQ�H�x�Y�6��� ..... --> DATA [/enc] [/message]
where DATA consist (encrypted) of: random mediaKey, url (we got when uploaded)
sorry for my bad english.
@bahtiarp i can send the media message (without encrypt img) and response similar like @pabloruan0710 , the recipient never received the message. But if i send the encrypted image, always return 400 bad request. To generate the refs, i made some modifications of hBAOS :
digTo = hmac.new("".encode("utf-8"), self.jid.replace("@s.whatsapp.net","@c.us").encode("utf-8"), hashlib.sha256).digest()[:20]
refTo = base64.b64encode(digTo).decode()
digFrom = hmac.new("".encode("utf-8"), self.accountJid.replace("@s.whatsapp.net","@c.us").encode("utf-8"), hashlib.sha256).digest()[:20]
refFrom = base64.b64encode(digFrom).decode()
hBAOS = "--" + boundary + "\r\n"
hBAOS += "Content-Disposition: form-data; name=\"hash\"\r\n\r\n"
hBAOS += self.__class__.getFileHashForUpload(sourcePath) + "\r\n"
hBAOS += "--" + boundary + "\r\n"
hBAOS += "Content-Disposition: form-data; name=\"refs\"\r\n\r\n"
hBAOS += refFrom + "\r\n"
hBAOS += refTo + "\r\n"
hBAOS += "--" + boundary + "\r\n"
hBAOS += "Content-Disposition: form-data; name=\"file\"; filename=\"" + "blob" + "\"\r\n"
hBAOS += "Content-Type: " + "application/octet-stream" + "\r\n\r\n"
How to encrypt image properly? Any help would be appreciated.
@whyasta I think you're getting "400 bad request" because the length of encrypted file isn't the same from the raw file, so your "filesize" needs to be equal "len(stream)".
@whyasta
It's seems that that your Content-Length is based on file before encrypted.
This is how i fixed it:
try:
filename = os.path.basename(sourcePath)
filetype = MimeTools.getMIME(filename)
f = open(sourcePath, 'rb')
stream = f.read()
f.close()
refkey = binascii.hexlify(os.urandom(112))
stream=self.encryptImg(stream,refkey)
fenc = open(filename+".enc", 'wb')
fenc.write(stream)
fenc.seek(0, 2)
filesize=fenc.tell()
fenc.close()
sha1 = hashlib.sha256()
sha1.update(stream)
b64Hash = base64.b64encode(sha1.digest())
Now we can continue to fixing the next process.
@bahtiarp did you get to successfully upload the file? I'm stopped on an error saying "415 Unsupported Media Type"
@duzzifelipe Yes, i got this:
POST https://mmg.whatsapp.net/u/f/ctq7Cm2nDwGv6rv1jqGWdlmMe7gABVZn2tCVcQ/AmJ99kyxGcMaIS1BkYC8vyFDFmPPbXQ779hJKmhWMCZL Content-Type: multipart/form-data; boundary=zzXXzzYYzzXXzzQQ Host: mmg.whatsapp.net User-Agent: WhatsApp/2.16.11 S40Version/14.26 Device/Nokia-302 Content-Length: 7422
--zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="hash"
NCzF2z0pL8F+zAN41s/MFiJPOrL4yi0cNMmCsWSHc6w= --zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="refs"
bsD9smTr55H5ifQkU3xd+F1Webo= KuU/zMxPnXiAqbatZ0tN3aKXjqA= --zzXXzzYYzzXXzzQQ Content-Disposition: form-data; name="file"; filename="blob" Content-Type: application/octet-stream
RESPONSE: HTTP/1.1 200 OK Content-Type: application/json Date: Thu, 10 Aug 2017 15:28:56 GMT Connection: close Content-Length: 220
{"type":"encrypted","mimetype":"application/octet-stream","size":"7034","filehash":"NCzF2z0pL8F+zAN41s/MFiJPOrL4yi0cNMmCsWSHc6w=","url":"https://mmg-fna.whatsapp.net/d/f/AtOdjxzg-CAUEi-2LL_uepNfYNquO9rIItlkp-uKRngh.enc"}
As you can see: Content-Length: 7422 --> size of the encrypted file + headers length "size":"7034" --> original size of the encrypted file return by whatsapp.
@bahtiarp woow! I got it using your code to generate b64Hash. Thank you! Now I'll try to see how this message is sent by whatsapp, maybe using this plugin: https://github.com/jlguardi/wireshark-whatsapp
Can you help us, the Loqui team, to get this working under our messenger LoquiIM? It's based on yowsup, but uses javascript as language. You can join our telegram group. Thank you, that we can see some work on yowsup again!
Does anybody know what to do for sending the image to the receipent? The file is uploading correctly but not sent to the other number. There is no error or something else. What message must be send to push the image to the receipent? Here is my diff: https://pastebin.com/LhtH7GyX
See my diff: https://pastebin.com/ADJX5EnQ Now I'm getting an ack-message from the receipent but there is no message. It seems the message is sent but it never comes to the phone... See my log: https://pastebin.com/ettx3t0p
@dietzi
btw why is it twice to send same request for upload: (see line 3 and 9 on your log) and also you get 2 same rx for those request: line 15 and line 39.
I don't know. The log says the message is send but it never retrieves on the phone..... I think the error is at line 32
if you see my post 26 days ago, and follow the same format your message will arrive in dest phone. the encrypted message consist of :
I'm still looking for my idle time to learn and modify in axolot layer for sending encrypted image message, or waiting for others solved this first.
What did you changed in the code? Can you show a diff?
i made some modifications but i forgat all, but mainly i modify https://github.com/tgalal/yowsup/blob/master/yowsup/layers/axolotl/layer_send.py line 46, so that the message will send in that format.
Just go to console in your yowsup-folder and type "git diff"
@bahtiarp
You wrote in your post (https://github.com/tgalal/yowsup/issues/2149#issuecomment-322039950) "encrypted message consist of...". Have the message to be encrypted? Yowsup sends this message after upload (I changed the code so mediaKey (got from refkey) is included):
ID: 1502735729-2 To: 49176****@s.whatsapp.net Type: media Timestamp: 1502735729 Media Type: image Has Preview: True MimeType: image File Hash: 3736a38324a39507852765042775a37514a76564c6a57432f564761626156576e7544582b5074695457593d URL: https://mmg.whatsapp.net/d/f/ApbBa3ek7yXDgkMaPr3gyYZYAcKNDIWTq3zvSRTmQ1D4.enc IP: None File Size: 5136 File name: i.png Encoding: raw Width: 192 Height: 192 mediaKey: 99c1863d363939796dcf66b6161cc0ed56d12bc2a282a0c132c8c5178d46b3865e34cec0f8aa92e8e5afc233f32df93bfd80a0b010e165f6f2796c5033eab2a7c12236709b12647d2733dee9f294f03c411c89a7db9f25ab45445883ccc23554dda99c970bbc2a3d55e9819cdfb6c3aa
`
But on my phone I only get the message "Waiting for this message......". Yowsup shows "\
Base on @jlguardi hint: We need the inverse procedure when receiving image:
this is my log when receiving encrypted message with url to encrypted image: image-received-log this is my log when sent data: where [DATA] in body of mediaNode is encrypted of protobuf imageMesage.
this is how i compose imageMesage in protobuf before it encrypted and sent.
def toProtobufMessage(self):
mediaKey=binascii.unhexlify(mediaKey) --> mediaKey we genereted when uploading enc image
fileHash= filehash --> filehash we got as in responsed when uploading enc image
url=self.url..encode('latin-1') -> url we got in responsed when uploading enc image
image_message = ImageMessage()
image_message.url = url
image_message.width = self.width
image_message.height = self.height
image_message.mime_type = self.mimeType
image_message.file_sha256 = self.fileHash
image_message.file_length = self.size
image_message.caption = "judul"
image_message.jpeg_thumbnail = self.preview
image_message.media_key = mediaKey
return image_message
i still doubt in filehash (because when i observed when receiving image in binary format, but i sent still in plain hash).
So if i put [DATA] as random string, the message will reached the destination phone, but we no image only text: Waiting for this message. This may take a while. Learn More
But if i put [DATA] with the encrypted ImageMessage. the message is sent ang i got ACK but the message didn't reached the destination phone.
Currently i still didn't have idle time for further analyzing. May be if somebody can sniff from whatsapp web how is the propper node structure to send image message, will help to speedup solving this case.
i tried to send image from yowsup A to yowsup B, in B logs i found that the message arrived but, yowsup cannot parsed the enc message.
File "/home/dev/WA/py/yowsup-master/yowsup/layers/axolotl/layer_receive.py", line 137, in handleWhisperMessage whisperMessage = WhisperMessage(serialized=enc.getData()) File "/usr/lib/python2.7/site-packages/axolotl/protocol/whispermessage.py", line 38, in init whisperMessage.ParseFromString(message) google.protobuf.message.DecodeError: Error parsing message.
@bahtiarp maybe the way you send is leaving the "enc" node child empty and decoder cannot understand.
And did you get to send an image message? I'm almost there. The closest I got was this:
I know I forgot something, but I didn't find it yet.
if i fill "enc" child node empty, the message reached the dest, but no image warning message only. if i fill with encrypted imageMessage protobuf, then i got ack but the image didn't rendered on dest. (i know that the message is reached the dest, but still not proper so the dest cannot render).
btw, can you show your send log when the data shown as in your screen captured. i'm affraid that the enc attribute not attribute for image type.
my new progress, i can send image from yowsup A to yowsup B. and message and image decrypted properly, and i can view the image correctly. but if i send from yowsup to whatsapp phone the image still not rendered although , my yowsup get
The prev image now come but cannot renderd the full image.
when click error message: Download failed
"The download was unable to complete. Please try again later."
Maybe the problem now is with correctly upload encrypted media, because the preview is generated based on the file path (at least on my repo) and does not depends on the media upload.
This week I had no time to do it right and I even lost 2 numbers. But I'll try this weekend, at least get at the same step as you.
@bahtiarp how did you got to this point? Can you send a diff please? So we can work together to get the problem solved. I'm sniffing the packets by using a proxy on my home-server, so I can decrypt the SSL-messages.
I tried with sniffing but WhatsApp (on Android) expects a special certificate for SSL-connection. So my fakecert wont't work.....
@dietzi , this is the diff: diff
with that i now can:
now 2.10 am morning, need for sleep now 😴
Your link don't work. Good night ;)
Hello I'm trying to upload media by yowsup, using
RequestUploadIqProtocolEntity
but it's not possible because lambda function allways direct toonRequestUploadError
, and cant send file ...Request upload for file /tmp/jpeg.jpg for 5524XXXXXXXX@s.whatsapp.net failed
I also tried with yowsupCLI using image send
/image send 5524XXXXXX /tmp/3DCDA9A9436B01F9A3.jfif