Media Encryption

[yniv]

Hi all, I know that @jlguardi and more ppl are working on the media encryption problem. Can someone update on the progress? and if it's already working, then what is the branch to work with? (a pull request here can also be nice)

Thanks for all the effort ppl are making to make this work!

[jlguardi]

@yniv jlguardi/encrypted_media recieve pkmsg and msg media types (but documents and urls which are decrypted but not processed by media layer). I'm registering my improvements in my wiki.

Media send isn't implemented.

[bdbais]

File "/usr/local/lib/python2.7/dist-packages/yowsup2-2.4.48-py2.7.egg/yowsup/layers/protocol_media/layer.py", line 40, in recvMessageStanza if mediaNode.getAttributeValue("type") == "image": AttributeError: 'NoneType' object has no attribute 'getAttributeValue' Exception in thread YowPingThread-1 (most likely raised during interpreter shutdown): Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 552, in __bootstrap_inner File "/usr/local/lib/python2.7/dist-packages/yowsup2-2.4.48-py2.7.egg/yowsup/layers/protocol_iq/layer.py", line 89, in run <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'sleep'

is it the same problem? @jlguardi I tried your last update on encryption branch but result is the same.

[bdbais]

File "/usr/local/lib/python2.7/dist-packages/yowsup2-2.4.48-py2.7.egg/yowsup/layers/init.py", line 105, in receive recv(node) File "/usr/local/lib/python2.7/dist-packages/yowsup2-2.4.48-py2.7.egg/yowsup/layers/protocol_media/layer.py", line 40, in recvMessageStanza if mediaNode.getAttributeValue("type") == "image": AttributeError: 'NoneType' object has no attribute 'getAttributeValue'

tried to reset and clone / build /install again...

[Dimelay]

--- yowsup-master/yowsup/layers/axolotl/layer.py        2015-12-14 15:02:54.000000000 +0800
+++ yowsup/yowsup/layers/axolotl/layer.py       2016-03-28 11:22:41.403074893 +0800
@@ -24,9 +24,14 @@
 from axolotl.nosessionexception import NoSessionException
 from axolotl.untrustedidentityexception import UntrustedIdentityException
 from .protocolentities.receipt_outgoing_retry import RetryOutgoingReceiptProtocolEntity
+from yowsup.common import YowConstants
+from axolotl.kdf.hkdfv3 import HKDFv3
+from axolotl.util.byteutil import ByteUtil
 import binascii
 import sys
-
+from Crypto.Cipher import AES
+import urllib2
+import encrypted_media_pb2
 import logging
 logger = logging.getLogger(__name__)

@@ -103,7 +108,7 @@
                 self.store = None

     def send(self, node):
-        if node.tag == "message" and node["type"] == "text" and node["to"] not in self.skipEncJids:
+        if node.tag == "message" and node["type"] == "text" and node["to"] not in self.skipEncJids and not YowConstants.WHATSAPP_GROUP_SERVER in node["to"]:
             self.handlePlaintextNode(node)
             return
         self.toLower(node)
@@ -113,7 +118,7 @@
         :type protocolTreeNode: ProtocolTreeNode
         """
         if not self.processIqRegistry(protocolTreeNode):
-            if protocolTreeNode.tag == "message":
+            if protocolTreeNode.tag == "message" or protocolTreeNode.tag == "media":
                 self.onMessage(protocolTreeNode)
                 return
             elif protocolTreeNode.tag == "notification" and protocolTreeNode["type"] == "encrypt":
@@ -257,13 +262,8 @@
         preKeyWhisperMessage = PreKeyWhisperMessage(serialized=pkMessageProtocolEntity.getEncData())
         sessionCipher = self.getSessionCipher(pkMessageProtocolEntity.getFrom(False))
         plaintext = sessionCipher.decryptPkmsg(preKeyWhisperMessage)
-
         if pkMessageProtocolEntity.getVersion() == 2:
             plaintext = self.unpadV2Plaintext(plaintext)
-
-
-        bodyNode = ProtocolTreeNode("body", data = plaintext)
-        node.addChild(bodyNode)
         self.toUpper(node)

     def handleWhisperMessage(self, node):
@@ -275,18 +275,64 @@

         if encMessageProtocolEntity.getVersion() == 2:
             plaintext = self.unpadV2Plaintext(plaintext)
-
-        bodyNode = ProtocolTreeNode("body", data = plaintext)
-        node.addChild(bodyNode)
-        self.toUpper(node)
+        if encMessageProtocolEntity._type == "text":
+            bodyNode = ProtocolTreeNode("body", data = plaintext)
+            node.addChild(bodyNode)
+            self.toUpper(node)
+        elif encMessageProtocolEntity._type == "media":
+            if node.getChild("enc")["mediatype"] == "image":
+                media = encrypted_media_pb2.Media()
+                media.ParseFromString(plaintext)
+                media = encrypted_media_pb2.Media()
+                media.ParseFromString(plaintext)
+                encimg = urllib2.urlopen(media.url).read()
+                derivative = HKDFv3().deriveSecrets(media.refkey, binascii.unhexlify('576861747341707020496d616765204b657973'), 112)
+                parts = ByteUtil.split(derivative, 16, 32)
+                iv = parts[0]
+                cipherKey = parts[1]
+                e_img = encimg[:-10]
+                AES.key_size=128
+                cr_obj = AES.new(key=cipherKey,mode=AES.MODE_CBC,IV=iv)
+                myimg = cr_obj.decrypt(e_img)
+                pos = plaintext.upper().find(binascii.unhexlify('ffd8ffe0'.upper()))
+                preview = ""
+                if pos > 0:
+                    preview = plaintext[pos:]
+                else:
+                    preview = "lol"
+                inode = ProtocolTreeNode("media",)
+                inode.setAttribute("mimeType", media.mimetype)
+                inode.setAttribute("file", myimg)
+                inode.setAttribute("filehash", media.sha256)
+                inode.setAttribute("url", media.url)
+                inode.setAttribute("ip", "idk")
+                inode.setAttribute("size", media.length)
+                inode.setAttribute("filename", media.url.split("/")[-1:])
+                inode.setAttribute("encoding", "unknown")
+                inode.setAttribute("width", media.width)
+                inode.setAttribute("height", media.height)
+                inode.setAttribute("caption",media.caption)
+                inode.setAttribute("preview", preview)
+                inode.setAttribute("type", "image")
+                node.addChild(inode)
+                self.toUpper(node)
+
+    def decodeInt7bit(self, string):
+        idx = 0
+        while ord(string[idx]) >= 128:
+            idx += 1
+        consumedBytes = idx + 1
+        value = 0
+        while idx >= 0:
+            value <<= 7
+            value += ord(string[idx]) % 128
+            idx -= 1
+        return value, consumedBytes

     def unpadV2Plaintext(self, v2plaintext):
-        if len(v2plaintext) < 128:
-            return v2plaintext[2:-1]
-        else: # < 128 * 128
-            return v2plaintext[3: -1]
-
-    ####
+        end = -ord(v2plaintext[-1]) # length of the left padding
+        length,consumed = self.decodeInt7bit(v2plaintext[1:])
+        return v2plaintext[1+consumed:end]

     ### keys set and get
     def sendKeys(self, fresh = True, countPreKeys = _COUNT_PREKEYS):

--- yowsup-master/yowsup/layers/protocol_media/protocolentities/message_media_downloadable.py   2015-12-14 15:02:54.000000000 +0800
+++ yowsup/yowsup/layers/protocol_media/protocolentities/message_media_downloadable.py  2016-03-28 07:40:35.670568141 +0800
@@ -19,12 +19,12 @@
     </message>
     '''
     def __init__(self, mediaType,
-            mimeType, fileHash, url, ip, size, fileName, 
+            mimeType,  fileHash, url, ip, size, fileName,rawfile, 
             _id = None, _from = None, to = None, notify = None, timestamp = None, 
             participant = None, preview = None, offline = None, retry = None):

         super(DownloadableMediaMessageProtocolEntity, self).__init__(mediaType, _id, _from, to, notify, timestamp, participant, preview, offline, retry)
-        self.setDownloadableMediaProps(mimeType, fileHash, url, ip, size, fileName)
+        self.setDownloadableMediaProps(mimeType, fileHash, url, ip, size, fileName, rawfile)

     def __str__(self):
         out  = super(DownloadableMediaMessageProtocolEntity, self).__str__()
@@ -44,14 +44,21 @@

     def getMimeType(self):
         return self.mimeType
+    
+    def getFileName(self):
+        return self.fileName
+    
+    def getRawFile(self):
+        return self.rawfile

-    def setDownloadableMediaProps(self, mimeType, fileHash, url, ip, size, fileName):
+    def setDownloadableMediaProps(self, mimeType, fileHash, url, ip, size, fileName, rawfile):
         self.mimeType   = mimeType
         self.fileHash   = fileHash
         self.url        = url
         self.ip         = ip
         self.size       = int(size)
         self.fileName   = fileName
+        self.rawfile    = rawfile

     def toProtocolTreeNode(self):
         node = super(DownloadableMediaMessageProtocolEntity, self).toProtocolTreeNode()
@@ -63,7 +70,6 @@
             mediaNode.setAttribute("ip",        self.ip)
         mediaNode.setAttribute("size",      str(self.size))
         mediaNode.setAttribute("file",      self.fileName)
-
         return node

     @staticmethod
@@ -77,6 +83,7 @@
             mediaNode.getAttributeValue("url"),
             mediaNode.getAttributeValue("ip"),
             mediaNode.getAttributeValue("size"),
+            mediaNode.getAttributeValue("filename"),
             mediaNode.getAttributeValue("file")
             )
         return entity
@@ -87,7 +94,8 @@
         filehash = filehash or WATools.getFileHashForUpload(fpath)
         size = filesize or os.path.getsize(fpath)
         fileName = os.path.basename(fpath)
+        rawfile = None

-        return DownloadableMediaMessageProtocolEntity(mediaType, mimeType, filehash, url, ip, size, fileName, to = to, preview = preview)
+        return DownloadableMediaMessageProtocolEntity(mediaType, mimeType, filehash, url, ip, size, fileName, rawfile, to = to, preview = preview)

 yowsup-master/yowsup/demos/cli/layer.py 

     def getDownloadableMediaMessageBody(self, message):
-         return "[Media Type:{media_type}, Size:{media_size}, URL:{media_url}]".format(
+         path= '/tmp/%s.jpg' % (message.getFileName()[0])
+         f=open(path,'wb' )
+         f.write(message.getRawFile())
+         f.close()         
+         return "[Media Type:{media_type}, Size:{media_size}, Save to:{media_url}]".format(
             media_type = message.getMediaType(),
             media_size = message.getMediaSize(),
-            media_url = message.getMediaUrl()
+            media_url = path
             )

Use path. Get Image, decrypt, save to /tmp/Filename.jpg and send ack. Integrates to layers. Work.

[yniv]

@Dimelay can you post the code in a more understandable way? also, i don't use the cli so i think it won't help me.

@jlguardi your develop branch is the last working version?? also, were do you store the media after you download them? can the storage be on a different server?

[jlguardi]

‎@yniv No, I moved develop branch to master. In yowsup cli you have an example of media downloading to file.

If you need the content in a remote server you can download the encrypted image and upload without encryption again to wa server and then use the upload url.

De: yniv Enviado: lunes, 28 de marzo de 2016 10:05 Para: tgalal/yowsup Responder a: tgalal/yowsup CC: jlguardi Asunto: Re: [yowsup] Media Encryption (#1415)

@Dimelay can you post the code in a more understandable way? also, i don't use the cli so i think it won't help me.

@jlguardi your develop branch is the last working version?? also, were do you store the media after you download them? can is the storage to be on a different server?

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/tgalal/yowsup/issues/1415#issuecomment-202292627

[yniv]

@jlguardi thanks! i'm going to try merge your version with mine. do i need to change your code if I want to upload to a remote server? where exactly?

[jlguardi]

@yniv see yowsup-cli layer and make a mix between onMedia and image_send and replace onUploadMediaSuccess.

De: yniv Enviado: lunes, 28 de marzo de 2016 12:41 Para: tgalal/yowsup Responder a: tgalal/yowsup CC: jlguardi Asunto: Re: [yowsup] Media Encryption (#1415)

@jlguardi thanks! i'm going to try merge your version with mine. do i need to change your code if I want to upload to a remote server? where exactly?

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/tgalal/yowsup/issues/1415#issuecomment-202338220

[yniv]

@jlguardi thanks for your help. question: when i get the media in the yowsup-cli it is already decrypted right?

[jlguardi]

@yniv yes, getMediaContent() does the magic.

[IvanTurgenev]

so python 3.5.1 yowsup-cli v2.0.13 yowsup v2.4.48

yowsup-cli demos -c .yowsup -s xxxxxx "lol"

ImportError: No module named 'urllib2'

did git clone today

EDIT: OOO never mind it was from @jlguardi repo, installed the default repo but now gives me another error

File "/usr/lib/python3.5/site-packages/google/protobuf/internal/python_message.py", line 848 except struct.error, e: ^

EDIT: Installed pip install -Iv protobuf==3.0.0b2.post2

now gives me more errors will try later more

[jlguardi]

See ‎https://github.com/jlguardi/yowsup/issues/6#issuecomment-202446917

I'm waiting a PR.

De: IvanTurgenev Enviado: lunes, 28 de marzo de 2016 16:53 Para: tgalal/yowsup Responder a: tgalal/yowsup CC: jlguardi Asunto: Re: [yowsup] Media Encryption (#1415)

so python 3.5.1 yowsup-cli v2.0.13 yowsup v2.4.48

yowsup-cli demos -c .yowsup -s xxxxxx "lol"

ImportError: No module named 'urllib2'

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/tgalal/yowsup/issues/1415#issuecomment-202425671

[Dimelay]

yowsup-cli  demos  -c  .yowsup -s xxxxxx "lol"

ImportError: No module named 'urllib2'

Sorry, I forgot to add

yowsup-master/yowsup/demos/cli/layer.py 
import urllib2

[bdbais]

Image message come but images are saved crypted :(

[bdbais]

    def getMediaMessageBody(self, message):
        if message.getMediaType() in ("image", "audio", "video", "document"):
            return self.getDownloadableMediaMessageBody(message)
        else:
            return "[Media Type: %s] %s" % (message.getMediaType(), message)

    def getDownloadableMediaMessageBody(self, message):
        filename = "%s/%s%s"%(tempfile.gettempdir(),message.getId(),message.getExtension())
        with open(filename, 'wb') as f:
            f.write(message.getMediaContent())
        return "[Media Type:{media_type}, Size:{media_size}, URL:{media_url}, FILE:{fname}]".format(
            media_type=message.getMediaType(),
            media_size=message.getMediaSize(),
            media_url=message.getMediaUrl(),
            fname=filename
        )

I added this code to my EchoLayer class and I found jpeg decoded, thx:

[Media Type:image, Size:266270, URL:https://mmi460.whatsapp.net/d/AbCug9uqO0c8ceBnvPLs8lb9caI/AomgwxTRNa3DOMmXfnKtwYJt28q7nP9sDfGt3q0yCIJo.enc, FILE:/tmp/87FBD7E98DBFFA1A4846C5A2C8ABF1.jpeg]

[thundergreen]

Does this also work for

https://mmi262.whatsapp.net/d/C9t25Wc5tc7p8j66ynMZK1b9XvA/Aq_yauHSxDsyBzIlxFR33So6yevBnW49J488dDuiZWAQ.enc

? I don't get ackn when sending pics ..any workaround already ?

Thorsten Fröhlich Am 31.03.2016 20:57 schrieb "bdbais" notifications@github.com:

def getMediaMessageBody(self, message):
    if message.getMediaType() in ("image", "audio", "video", "document"):
        return self.getDownloadableMediaMessageBody(message)
    else:
        return "[Media Type: %s] %s" % (message.getMediaType(), message)

def getDownloadableMediaMessageBody(self, message):
    filename = "%s/%s%s"%(tempfile.gettempdir(),message.getId(),message.getExtension())
    with open(filename, 'wb') as f:
        f.write(message.getMediaContent())
    return "[Media Type:{media_type}, Size:{media_size}, URL:{media_url}, FILE:{fname}]".format(
        media_type=message.getMediaType(),
        media_size=message.getMediaSize(),
        media_url=message.getMediaUrl(),
        fname=filename
    )

I added this code to my EchoLayer class and I found jpeg decoded, thx.

— You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub https://github.com/tgalal/yowsup/issues/1415#issuecomment-204077665

[Natureshadow]

What's the state, and how can it be completed?

[albaizzz]

still not found solution for downloading media file from .enc.

any body get success download media from .enc ?

[mocart2]

still not found solution for downloading media file from .enc.

any body get success download media from .enc ?

+1