Add Imgur c2 transport and lsbjpg encoder

[Und3rf10w]

Adds a new transport mechanism, Imgur, and a new encoder to support it, lsbjpg.

See #11 for additional information.

Tasklist

• [x] Add logic to encoder_lsbjpg to order a list of received image when decoding - 20e8eae
• [ ] Fix image header numbering to support more images
• [x] Add logic to transmit access and refresh token from server to client - 1823013
• [x] Refactor the way large chunks of data are transmitted - a9c3325
• [x] Review whether commonutils.sendData() & commonutils.retrieveData() needs a refactor
• [x] Determine how to handle ImgurClientRateLimitError - 692cfe8 &

[Und3rf10w]

Imgur's authorization logic is a bit weird because for authentication actions, it will require that the user clicks a link to authenticate and receive an access and refresh tokens.

Obviously, this can't be done on a client we can't communicate with.

We can use our header as a way to store this info though.

if pix[0,0][0] == 1, then we know that this image contains the tokens we need. The client will be able to view this image containing the tokens without authenticating, while maintaining our opsec.

Once the client has received the tokens, it can then trigger an acknowledgement to the server by deleting the album with the tokens.

Only caveat is that the operator will have to configure the application to accept when starting the server.

All this logic could look something like this:

                                  +----------------------------+
                                  |                            |        +-----------------------------+
  Server's prepTransport()        |  Add the following header: |        |                             |
                                  |  pix[0,0] = (1, 1, 1)      +------->+  Insert the list of tokens  |
+-----------------------------+   |                            |        |  across the diagonal pixels |
|                             |   +---------+------------------+        |  ignoring first             |
| Server requests that        |             |                           |                             |
| user authorizes application |             |                           +-----------------------------+
|                             |             |
+--------+--------------------+             |
         |                                  |                                  Client's prepTransport()
         v                       +----------+----------+
+--------+---------------+       |                     |                 +------------------------------+
|                        |       | Server sends        |                 |                              |
| User inputs auth token +------>+ tokens as new album |                 | Client sees new single-photo +<-----------+
|                        |       |                     |                 | album                        |            |
+------------------------+       +----------+----------+                 |                              |            |
                                            |                            +------------------------------+            |
                +--------------------+      |                                        |                               |
                |                    |      |                                        |                               |
                |                    v      v                                        |                               |
          +-----+-------+       +----+------+----------+                             v                               |
          |             |       |                      |                 +-----------+-------------+       +---------+---------+
          | Album still |       | Server checks to see |                 |                         |       |                   |
          | exists      +<------+ if album was deleted |                 | Client checks value of  +------>+  pix[0,0][0] != 1 |
          |             |       |                      |                 | pix[0,0][0] == 1        |       |                   |
          +-------------+       +----------+-----------+                 |                         |       +-------------------+
                                           |                             +----------+--------------+
                                           |                                        |
                                           |                                        |
                                           v                                        v                                               +---------------+
        +---------------+        +---------+--------+                      +--------+---------+             +--------------+        |               |
        |               |        |                  |                      |                  |             |              |        | Continue main |
        | Continue main |        |  Album no longer |                      | pix[0,0][0] == 1 |             | Delete album +------->+ application   |
        | application   +<-------+  exists          |                      |                  |             |              |        | logic         |
        | logic         |        |                  |                      +--------+---------+             +------+-------+        |               |
        |               |        +------------------+                               |                              ^                +---------------+
        +---------------+                                                           v                              |
                                                                          +---------+--------------+               |
                                                                          |                        |         +-----+----------------------+
                                                                          | Iterate through pixels |         |                            |
                                                                          | diagonally, ignoring   +-------->+ Extract access and refresh |
                                                                          | first                  |         | tokens from photo          |
                                                                          |                        |         |                            |
                                                                          +------------------------+         +----------------------------+

[Und3rf10w]

Commit 6cf73a5 adds logic to handle the transmission of the token between the server and client.

getTokens() retrieves the token from the image based on it's header, then deletes the token image and album as a way to ack to the server that it has received the credentials

[Und3rf10w]

With commit 68d50fd, we've gotten completed most of the core functionality, but we need to refactor the way images are transmitted. Currently, we're building an array of images in memory, then plan to send them out to a single album.

This will not work for large amounts of data, because 255 1920x1080 pngs consume A LOT of memory.

[Und3rf10w]

Spoke too soon, commit 1823013 fixed tons of bugs in the token transaction process, but lays good foundation for sendData() and recvData()

[Und3rf10w]

Commit a9c3325 adds pretty much all needed functionality. I've tested all of bits individually, but need to test in full deployment.

[Und3rf10w]

Identified a number of bugs in runtime debugging:

• [ ] commonutils.SendData calls commonUtils.prepData(), which calls encoder.encode(), which will break the encoding process if we're encoding chunk by chunk. Fixed by changing functionality of commonutils.sendData() to just call transport.sendData()

• [x] transport will have to import the encoder with import utils.encoders.encoder_lsbjpg as encoder - cd28c61

• [x] Need to provide an album name for communication between the client and server so they have something to watch for when doing comms. - 7d90f3e

[Und3rf10w]

Commit 7d90f3e labels sending and receiving albums, and adds logic to ensure that the album we're looking at is the one we're intending based on the album title.

[Und3rf10w]

Adding to top level comment, but I'm running into an issue in sendData() where if the application triggers ImgurClientRateLimitError on request = client.make_request('POST', 'upload', image_upload_fields) the application crashes.

At this moment, I can't just have the application sleep for the hour because the receiving application (e.g. client) will not know that the sending application isn't done transmitting its data

[Und3rf10w]

Commit 692cfe8 adds logic for the sending application to sleep for 10 minutes, then retry transmission of the photo when the upload rate limit is hit (50 images in an hour).

This does not resolve the receiving side.

[Und3rf10w]

With commit 2ba1ef4, I have successfully verified that everything works. The only thing that I really had to change was the way the server was using utils.commonUtils.sendData() and utils.commonUtils.retrieveData().

The changes for running it are shown below for utils/commonUtils.py:

def sendData(data):
    # This will upload the data via the covert channel
    # returns a confirmation that the data has been sent

    if config.debug:
        print (color("RAW DATA TO BE SENT: ", status=False, yellow=True) + "%s") % (data)
    # Prepares the data to be sent via the covert channel
    # preped_data = prepData(data) # < --- Commented this line out

        transport.sendData(data) # < ---- Added this line in

def retrieveData():
    # This will retireve data via the covert channel
    # Returns unencoded data

    data = transport.retrieveData()

    if config.debug:
        print (color("RAW RETRIEVED DATA: ", status=False, yellow=True) + "%s") % (data)

    # Prepare the recieved data by running it through the decoder
    # preped_data = decodeData(data)  # < --- Commented this line out

    # return preped_data       # < --- Commented this line out
    return data        # < ---- Added this line in

If those functions are changed, everything will operate normally. This brings this pull request into a state good enough to merge into dev. I will open a new issue to refactor commonUtils.retrieveData() and commonUtils.sendData() before we can merge dev into master.

[Und3rf10w]

Issue for the bug added and documented in #13, merging into dev