kaitai-io / kaitai_struct

Kaitai Struct: declarative language to generate binary data parsers in C++ / C# / Go / Java / JavaScript / Lua / Nim / Perl / PHP / Python / Ruby
https://kaitai.io
4.01k stars 196 forks source link

Processing with real cryptographic ciphers #45

Open GreyCat opened 7 years ago

GreyCat commented 7 years ago

From time to time I get requests to implement "real" cryptographic ciphers in KS (like DES, AES, IDEA, RC*, etc). I'll try to outline what I have in mind on how to implement that.

  1. We start creating a KS standard for cipher naming and parameters. Currently, there are tons of names available in different implementations. We'd want to have a unified naming that will be used in .ksy file, that will be mapped into existing implementations.

  2. We'll slowly gather implementations for these algorithms in all target languages. We'll try to map stuff to standard libraries as much as possible, but, of course, not everything would be there. Thus we'll have to include external libraries - that's ok, but we'll strive as much as possible for that to be optional (i.e. if .ksy is not using algorithm X, there is no need to link/include library for X support in target project).

Example

- id: key
  size: 16
- id: iv
  size: 16
- id: buf
  size: 1024
  process: aes(128, key, iv, cbc)

It should be translated into something like that in Ruby:

@buf = KaitaiStream::unpack_aes(128, @_io.read(1024), key, iv, :CBC)

# ...

def unpack_aes(data, key_len, key, iv, mode)
  require 'openssl'
  decipher = OpenSSL::Cipher::AES.new(key_len, mode)  
  decipher.decrypt
  decipher.key = key
  decipher.iv = iv
  decipher.update(data) + decipher.final
end

If anyone's really interested in this, I suggest to start with gathering a list of ciphers with parameters and list of libraries in target languages that will support them.

arekbulski commented 6 years ago

The key/IV should be settable from application, its rarely in the same file as the encrypted data. The only exception to that would possibly be some ransomware?

GreyCat commented 6 years ago

It doesn't really matter where it comes from. As long as it's available, for example, via some sort of expression (and that, it turn, might use seq attributes, instance attributes, params, etc), it should work.

Getting into another file and reading it from there is technically not impossible too if we'll do #125.

arekbulski commented 6 years ago

Is there an expression that means "input at runtime"?

GreyCat commented 6 years ago

For example, you do a type like that:

params:
  - id: key
  - id: iv
seq:
  - id: encrypted_data
    process: aes(128, key, iv)

When you'll be using resulting class in your app, you'll have to pass key + iv to it — i.e. the constructor would look like that:

    def __init__(self, key, iv, _io, _parent=None, _root=None):
        self._io = _io
        self._parent = _parent
        self._root = _root if _root else self
        self.key = key
        self.iv = iv
        self._read()
arekbulski commented 6 years ago

Ahh, okie dokie. I wasnt aware of "params". I started reading the user manual but am far from finishing it. Now that I think of it, there isnt actually any equivalent in Construct.