Remove direct dependence on OpenSSL

[iceiix]

OpenSSL is an annoying dependency, because it has to be installed separately, and has to be a precise version. Ubuntu 18.04.1 for example comes with OpenSSL 1.1, which isn't compatible with 1.0 required by rust-openssl 0.7.8, and the Rust crate itself has significantly changed in .8 and .10: https://crates.io/crates/openssl - it is a large cumbersome dependency to lug around.

Should look into replacing this dependency with native Rust cryptography crates, if possible.

• [x] sha1 https://github.com/iceiix/steven/commit/163556fbf13c3df8fd6e8db6aa949129f648e05e
• [x] cfb8 https://github.com/iceiix/steven/pull/10
• [x] rsa.public_encrypt https://github.com/iceiix/steven/pull/12
• [x] rand_bytes https://github.com/iceiix/steven/pull/12
• [x] Remove OpenSSL from README.md, appveyor.yml, .travis.yml https://github.com/iceiix/steven/pull/12

[iceiix]

Replaced the use of OpenSSL for SHA1, but it is also used for pkey, rand_bytes, and symm. For the latter this module looks like a good choice: https://crates.io/crates/aes_frast API compatible with OpenSSL and supports AES CFB, but, 0.1.2 uses new Rust language features:

   Compiling aes_frast v0.1.2
error: expected identifier, found `*`
   --> /Users/admin/.cargo/registry/src/github.com-1ecc6299db9ec823/aes_frast-0.1.2/src/aes_core.rs:952:17
    |
952 |     use super::{*};
    |                 ^

error: expected one of `;`, `self`, or `}`, found `*`
   --> /Users/admin/.cargo/registry/src/github.com-1ecc6299db9ec823/aes_frast-0.1.2/src/aes_core.rs:952:17
    |
952 |     use super::{*};
    |                 ^ expected one of `;`, `self`, or `}` here

error: aborting due to 2 previous errors

not available in nightly-2017-08-31, may need to first https://github.com/iceiix/steven/issues/3 update to a newer Rust.

[iceiix]

aes_frast v0.1.2 compiles within steven after updating to nightly-2018-09-30 in https://github.com/iceiix/steven/issues/3, now to implement it in src/protocol/mod.rs. And src/server/mod.rs, what to replace pkey? https://crates.io/crates/rsa? (but its version 0.0.0) https://crates.io/crates/ring? rand_bytes maybe https://crates.io/crates/rand-bytes

[iceiix]

Made some progress porting to RustCrypto https://github.com/RustCrypto/block-ciphers/, saving here:

diff --git a/Cargo.toml b/Cargo.toml
index 1959136..75bc7f2 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -10,6 +10,8 @@ authors = [ "Thinkofdeath <thinkofdeath@spigotmc.org>" ]
 opt-level = 1

 [dependencies]
+aes = "0.2.0"
+block-modes = "0.1.0"
 sha1 = "0.6.0"
 sdl2 = "0.31.0"
 byteorder = "0.5.0"
diff --git a/src/main.rs b/src/main.rs
index ddffb7e..2e68807 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -23,6 +23,8 @@ extern crate byteorder;
 extern crate serde_json;
 extern crate openssl;
 extern crate sha1;
+extern crate aes;
+extern crate block_modes;
 extern crate hyper;
 extern crate flate2;
 extern crate rand;
diff --git a/src/protocol/mod.rs b/src/protocol/mod.rs
index 4ada321..b92e8e7 100644
--- a/src/protocol/mod.rs
+++ b/src/protocol/mod.rs
@@ -15,6 +15,11 @@
 #![allow(dead_code)]

 use openssl::crypto::symm;
+use aes::block_cipher_trait::generic_array::GenericArray;
+use aes::Aes128;
+use block_modes::{BlockMode, BlockModeIv, Cfb};
+use block_modes::block_padding::ZeroPadding;
+type Aes128Cfb = Cfb<Aes128, ZeroPadding>; // TODO: CFB doesn't use padding
 use serde_json;
 use hyper;

@@ -743,6 +748,7 @@ pub struct Conn {
     pub state: State,

     cipher: Option<symm::Crypter>,
+    cipher2: Option<Aes128Cfb>,

     compression_threshold: i32,
     compression_read: Option<ZlibDecoder<io::Cursor<Vec<u8>>>>,
@@ -770,6 +776,7 @@ impl Conn {
             direction: Direction::Serverbound,
             state: State::Handshaking,
             cipher: Option::None,
+            cipher2: Option::None,
             compression_threshold: -1,
             compression_read: Option::None,
             compression_write: Option::None,
@@ -860,6 +867,10 @@ impl Conn {
         let cipher = symm::Crypter::new(symm::Type::AES_128_CFB8);
         cipher.init(if decrypt { symm::Mode::Decrypt } else { symm::Mode::Encrypt }, key, key);
         self.cipher = Option::Some(cipher);
+
+        let cipher2 = Aes128Cfb::new_varkey(key, GenericArray::from_slice(key)).unwrap();
+        println!("enabling encryption with key={:?}", key);
+        self.cipher2 = Option::Some(cipher2);
     }

     pub fn set_compresssion(&mut self, threshold: i32) {
@@ -967,10 +978,21 @@ impl Read for Conn {
             Option::None => self.stream.read(buf),
             Option::Some(cipher) => {
                 let ret = try!(self.stream.read(buf));
+
+                println!("decrypting {:?}", &buf[..ret]);
+
                 let data = cipher.update(&buf[..ret]);
                 for i in 0..ret {
                     buf[i] = data[i];
                 }
+                println!("ossl buf = {:?}", buf);
+
+                /*
+                let result = self.cipher2.as_mut().unwrap().decrypt_nopad(&mut buf[..ret]);
+                println!("result = {:?}", result);
+                println!("buf = {:?}", buf);
+                */
+
                 Ok(ret)
             }
         }
@@ -982,7 +1004,15 @@ impl Write for Conn {
         match self.cipher.as_mut() {
             Option::None => self.stream.write(buf),
             Option::Some(cipher) => {
+                println!("encrypting buf = {:?}", buf);
                 let data = cipher.update(buf);
+                println!("data = {:?}", data);
+                /* TODO
+                let mut data: [u8; 32] = [0; 32];
+                data.clone_from_slice(&buf);
+                cipher.encrypt_nopad(&mut data).expect("failed to encrypt");
+                */
+
                 try!(self.stream.write_all(&data[..]));
                 Ok(buf.len())
             }
@@ -1003,6 +1033,7 @@ impl Clone for Conn {
             direction: self.direction,
             state: self.state,
             cipher: Option::None,
+            cipher2: Option::None,
             compression_threshold: self.compression_threshold,
             compression_read: Option::None,
             compression_write: Option::None,

but it fails with BlockModeError, may not be able to use block-modes crate in this way: https://github.com/RustCrypto/block-ciphers/issues/28

Long story short, CFB (and CTR) modes turn a block cipher into a stream cipher. https://github.com/RustCrypto/stream-ciphers implements CTR but we need CTR. Try a different crate? Not many hits for cfb on crates.io: https://crates.io/search?q=cfb there's aes_frast again, it gets it: https://github.com/KaneGreen/aes_frast/blob/master/src/aes_with_operation_mode.rs#L226

/// CFB (Cipher Feedback) Encryption
/// 
/// The feedback size is fixed to 128 bits, which is the same as block size.  
/// This mode doesn't require padding.

[iceiix]

Incomplete aes_frast attempt:

diff --git a/Cargo.toml b/Cargo.toml
index 1959136..50bfb6d 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -10,6 +10,7 @@ authors = [ "Thinkofdeath <thinkofdeath@spigotmc.org>" ]
 opt-level = 1

 [dependencies]
+aes_frast = "0.1.2"
 sha1 = "0.6.0"
 sdl2 = "0.31.0"
 byteorder = "0.5.0"
diff --git a/src/main.rs b/src/main.rs
index ddffb7e..4f5fdc5 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -23,6 +23,7 @@ extern crate byteorder;
 extern crate serde_json;
 extern crate openssl;
 extern crate sha1;
+extern crate aes_frast;
 extern crate hyper;
 extern crate flate2;
 extern crate rand;
diff --git a/src/protocol/mod.rs b/src/protocol/mod.rs
index 4ada321..ddad380 100644
--- a/src/protocol/mod.rs
+++ b/src/protocol/mod.rs
@@ -15,6 +15,7 @@
 #![allow(dead_code)]

 use openssl::crypto::symm;
+use aes_frast::{aes_core, aes_with_operation_mode};
 use serde_json;
 use hyper;

@@ -743,6 +744,8 @@ pub struct Conn {
     pub state: State,

     cipher: Option<symm::Crypter>,
+    w_keys: Vec<u32>,
+    iv: Vec<u8>,

     compression_threshold: i32,
     compression_read: Option<ZlibDecoder<io::Cursor<Vec<u8>>>>,
@@ -770,6 +773,8 @@ impl Conn {
             direction: Direction::Serverbound,
             state: State::Handshaking,
             cipher: Option::None,
+            w_keys: vec![0u32; 60],
+            iv: vec![0; 16],
             compression_threshold: -1,
             compression_read: Option::None,
             compression_write: Option::None,
@@ -860,6 +865,8 @@ impl Conn {
         let cipher = symm::Crypter::new(symm::Type::AES_128_CFB8);
         cipher.init(if decrypt { symm::Mode::Decrypt } else { symm::Mode::Encrypt }, key, key);
         self.cipher = Option::Some(cipher);
+
+        aes_core::setkey_enc_auto(&key, &mut self.w_keys);
     }

     pub fn set_compresssion(&mut self, threshold: i32) {
@@ -1003,6 +1010,8 @@ impl Clone for Conn {
             direction: self.direction,
             state: self.state,
             cipher: Option::None,
+            w_keys: vec![0u32; 60],
+            iv: vec![0; 16],
             compression_threshold: self.compression_threshold,
             compression_read: Option::None,
             compression_write: Option::None,

There is another dimension to CFB, the "segment size", we probably want 8-bit (1 byte) not 128-bit (8 byte), failed with aes_frast https://github.com/KaneGreen/aes_frast/issues/2 but watch https://github.com/RustCrypto/block-ciphers/issues/28

[newpavlov]

BTW if you are using RustCrypto crates, consider using sha-1 crate instead of sha1.

[iceiix]

Saving progress here on RustCrypto cfb-mode porting attempt:

diff --git a/Cargo.toml b/Cargo.toml
index 3f4e636..73c1541 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -27,6 +27,8 @@ cgmath = "0.7.0"
 lazy_static = "1.1.0"
 collision = {git = "https://github.com/TheUnnamedDude/collision-rs", rev = "f80825e"}
 openssl = "0.7.8"
+aes = "0.2.0"
+cfb-mode = "0.1.0"
 # clippy = "*"

 [dependencies.steven_gl]
diff --git a/src/main.rs b/src/main.rs
index b038860..7c3edf3 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -22,6 +22,8 @@ use std::time::{Instant, Duration};
 extern crate byteorder;
 extern crate serde_json;
 extern crate openssl;
+extern crate aes;
+extern crate cfb_mode;
 extern crate sha1;
 extern crate hyper;
 extern crate flate2;
diff --git a/src/protocol/mod.rs b/src/protocol/mod.rs
index 6625da4..7ab3f93 100644
--- a/src/protocol/mod.rs
+++ b/src/protocol/mod.rs
@@ -15,6 +15,8 @@
 #![allow(dead_code)]

 use openssl::crypto::symm;
+use aes::Aes128;
+use cfb_mode::Cfb;
 use serde_json;
 use hyper;

@@ -735,6 +737,8 @@ impl ::std::fmt::Display for Error {
     }
 }

+type Aes128Cfb = Cfb<Aes128>;
+
 pub struct Conn {
     stream: TcpStream,
     pub host: String,
@@ -743,6 +747,7 @@ pub struct Conn {
     pub state: State,

     cipher: Option<symm::Crypter>,
+    cipher2: Option<Aes128Cfb>,

     compression_threshold: i32,
     compression_read: Option<ZlibDecoder<io::Cursor<Vec<u8>>>>,
@@ -770,6 +775,7 @@ impl Conn {
             direction: Direction::Serverbound,
             state: State::Handshaking,
             cipher: Option::None,
+            cipher2: Option::None,
             compression_threshold: -1,
             compression_read: Option::None,
             compression_write: Option::None,
@@ -860,6 +866,10 @@ impl Conn {
         let cipher = symm::Crypter::new(symm::Type::AES_128_CFB8);
         cipher.init(if decrypt { symm::Mode::Decrypt } else { symm::Mode::Encrypt }, key, key);
         self.cipher = Option::Some(cipher);
+
+        let cipher2 = Aes128Cfb::new_var(key, key).unwrap();
+        println!("enabling encryption with key={:?}", key);
+        self.cipher2 = Option::Some(cipher2);
     }

     pub fn set_compresssion(&mut self, threshold: i32) {
@@ -967,10 +977,19 @@ impl Read for Conn {
             Option::None => self.stream.read(buf),
             Option::Some(cipher) => {
                 let ret = try!(self.stream.read(buf));
+
+                println!("decrypting {:?}", &buf[..ret]);
                 let data = cipher.update(&buf[..ret]);
+                println!("ossl = {:?}", &data);
+
+                self.cipher2.as_mut().unwrap().decrypt(&mut buf[..ret]);
+                println!("buf  = {:?}", &buf[..ret]);
+
+
                 for i in 0..ret {
                     buf[i] = data[i];
                 }
+
                 Ok(ret)
             }
         }
@@ -982,7 +1001,14 @@ impl Write for Conn {
         match self.cipher.as_mut() {
             Option::None => self.stream.write(buf),
             Option::Some(cipher) => {
+                println!("encrypting buf = {:?}", buf);
                 let data = cipher.update(buf);
+                println!("ossl data = {:?}", data);
+                /* TODO
+                self.cipher2.as_mut().unwrap().encrypt(&mut buf);
+                println!("buf = {:?}", buf);
+                */
+
                 try!(self.stream.write_all(&data[..]));
                 Ok(buf.len())
             }
@@ -1003,6 +1029,7 @@ impl Clone for Conn {
             direction: self.direction,
             state: self.state,
             cipher: Option::None,
+            cipher2: Option::None,
             compression_threshold: self.compression_threshold,
             compression_read: Option::None,
             compression_write: Option::None,

but cfb-mode currently only supports CFB128, whereas Steven requires CFB8... https://github.com/RustCrypto/stream-ciphers/issues/4#issuecomment-427002843

[iceiix]

reqwest, added in https://github.com/iceiix/steven/pull/7, can use OpenSSL but only on Linux, for other platforms it uses the native TLS libraries: From https://github.com/seanmonstar/reqwest/

Reqwest uses rust-native-tls, which will use the operating system TLS framework if available, meaning Windows and macOS. On Linux, it will use OpenSSL 1.1.

[iceiix]

For the other functionality, https://crates.io/crates/rand-bytes uses ring: https://briansmith.org/rustdoc/ring/rand/index.html - ring also has public key signature signing and verification https://briansmith.org/rustdoc/ring/signature/index.html but can it not encrypt to a public key? Basically (besides CFB) only need to replace this snippet of code in src/server/mod.rs connect():

        let rsa = Rsa::public_key_from_der(&packet.public_key.data).unwrap();
        let mut shared = [0; 16];
        rand_bytes(&mut shared).unwrap();

        let mut shared_e = vec![0; rsa.size() as usize];
        let mut token_e = vec![0; rsa.size() as usize];
        rsa.public_encrypt(&shared, &mut shared_e, Padding::PKCS1)?;
        rsa.public_encrypt(&packet.verify_token.data, &mut token_e, Padding::PKCS1)?;

        try!(profile.join_server(&packet.server_id, &shared, &packet.public_key.data));

[iceiix]

Sadly ring doesn't support CFB mode, it was actually removed! https://github.com/briansmith/ring/commit/49c0edec78f0c295df096fe6af795b97f4c4b205

Not easy to find these obscure crypto methods.. but it is mandatory for protocol compatibility.

[iceiix]

RustCrypto cfb8 now available, switched over to it in https://github.com/iceiix/steven/pull/10.

Now the only remaining use is RSA (and rand), https://github.com/iceiix/steven/issues/2#issuecomment-433673776 - what to use to replace rsa.public_encrypt?

[iceiix]

RSA public key encryption in Rust is a problem, it appears openssl-rust is the only existing option. What is involved?

• ASN.1 DER parsing: the public key structure looks like this, per openssl asn1parse -inform DER -in:

    0:d=0  hl=3 l= 159 cons: SEQUENCE
    3:d=1  hl=2 l=  13 cons: SEQUENCE
    5:d=2  hl=2 l=   9 prim: OBJECT            :rsaEncryption
   16:d=2  hl=2 l=   0 prim: NULL
   18:d=1  hl=3 l= 141 prim: BIT STRING

https://crates.io/crates/asn1 - last updated 3 years ago https://crates.io/crates/eagre-asn1 - supporting ObjectIdentifier is on the to do list, as it BitString https://crates.io/crates/asn1_der - DerObject can hold any object, can Vec OctetString hold BitString? https://crates.io/crates/simple_asn1 - "this library automates the process of understanding the DER encoded objects in an ASN.1 data stream. These tokens can then be parsed by your library, based on the ASN.1 description in your format". The example https://github.com/acw/simple_asn1/blob/master/test/key.bin is an RSA public key, just what we want to parse! (Except it is an OCTET STRING instead of a BIT STRING in my case, and their example has an INTEGER :00 mine lacks).

• Bignum arithmetic - https://crates.io/crates/num
• RSA encryption to public key. https://en.wikipedia.org/wiki/RSA_(cryptosystem)
• PKCS1 padding. The only hit in https://crates.io/search?q=pkcs1 is something with YubiHSM and it is in documentation saying it is not supported. PKCS1v15: https://tools.ietf.org/html/rfc2313

[iceiix]

Parsing with simple_asn1, extracting the bit string:

extern crate simple_asn1;
use simple_asn1::{from_der, ASN1Block};

fn find_bitstrings(asns: Vec<ASN1Block>, mut result: &mut Vec<Vec<u8>>) {
    for asn in asns.iter() {
        match asn {
            ASN1Block::BitString(_, _, _, bytes) => result.push(bytes.to_vec()),
            ASN1Block::Sequence(_, _,  blocks) => find_bitstrings(blocks.to_vec(), &mut result),
            _ => (),
        }
    }
}

fn main() {
    let asns: Vec<ASN1Block> = from_der(&packet_public_key_data).unwrap();

    let mut result: Vec<Vec<u8>> = vec![];
    find_bitstrings(asns, &mut result);
    println!("result[0] = {:?}", result[0]);
}

asns = [Sequence(Universal, 0, [Sequence(Universal, 3, [ObjectIdentifier(Universal, 5, OID([BigUint { data: [1] }, BigUint { data: [2] }, BigUint { data: [840] }, BigUint { data: [113549] }, BigUint { data: [1] }, BigUint { data: [1] }, BigUint { data: [1] }])), Null(Universal, 16)]), BitString(Universal, ...])])]

[iceiix]

https://tools.ietf.org/html/rfc8017#section-7.2

RSAES-PKCS1-v1_5 combines the RSAEP and RSADP primitives (Sections 5.1.1 and 5.1.2) with the EME-PKCS1-v1_5 encoding method (Step 2 in Section 7.2.1, and Step 3 in Section 7.2.2).

https://tools.ietf.org/html/rfc8017#section-5.1.1

5.1.1.  RSAEP

   RSAEP ((n, e), m)

   Input:

         (n, e) RSA public key

         m message representative, an integer between 0 and n - 1

   Output:  c ciphertext representative, an integer between 0 and n - 1

   Error:  "message representative out of range"

   Assumption:  RSA public key (n, e) is valid

   Steps:

      1.  If the message representative m is not between 0 and n - 1,
          output "message representative out of range" and stop.

      2.  Let c = m^e mod n.

      3.  Output c.

https://docs.rs/num-bigint/0.2.1/num_bigint/struct.BigUint.html#method.modpow

pub fn modpow(&self, exponent: &Self, modulus: &Self) -> Self | [src] Returns (self ^ exponent) % modulus. Panics if the modulus is zero.

[iceiix]

https://github.com/iceiix/steven/pull/12 removes the direct dependency, in Cargo.toml, but note that Cargo.lock still lists openssl, because native-tls depends it, for TLS, but it will use alternative TLS stacks if available on different platforms. Removing the immediate usage is the important part.

[newpavlov]

If server supports TLS v1.3 you can try rustls instead of the native-tls.

[iceiix]

Unfortunately reqwest uses native-tls so I think I'm stuck with it for now, but there's an open issue for rustls support in reqwest: https://github.com/seanmonstar/reqwest/issues/378 if implemented will be 100% OpenSSL-free 👍