Extend `smtp-server`'s API to enable implementation of LMTP servers as well

Alexis211 commented 2 years ago

This PR introduces the following changes:

Add support for LMTP's LHLO message in smtp-message
Add a PROTOCOL constant to the smtp_server::Config trait (it defaults to Smtp)
Add a funciton handle_mail_multi to trait smtp_server::Config that can be implemented insteaad of handle_mail for LMTP server implementors that must be able to send back multiple responses

This patch does not change the API of smtp_server::Config for users that implement SMTP servers, in other words it should break no existing code.

Alexis211 commented 2 years ago

Tested with this code, seems to work fine.

Alexis211 commented 2 years ago

I still have to do the following:

fix other parts of Kannader to adapt to API change for handle_mail (returning a stream, but also is_ehlo becomming is_extended)
finding a better way to define the return type of handle_mail

Alexis211 commented 2 years ago

Grepping the source code base shows that the following impl smtp_server::Config for X need to be fixed to conform to the new API:

smtp-server/fuzz/fuzz_targets/fuzz_interact.rs
tests/integration/tests/tests.rs

However I don't know how to run these tests to check if they are working.

Otherwise, this is ready for a new round of review (ping @Ekleog )

Ekleog commented 2 years ago

I'll try to review this code this week!

As for the two you listed, the first one is a fuzzer that can be run by running cd fuzz && ./fuzz_interact.sh, and the second one is a test that should be runnable by doing cargo test -p integration :)

Alexis211 commented 2 years ago

As for the two you listed, the first one is a fuzzer that can be run by running cd fuzz && ./fuzz_interact.sh, and the second one is a test that should be runnable by doing cargo test -p integration :)

I'm unable to run either of these commands :(

[lx@pc fuzz]$ ./fuzz_interact.sh
Usage: ./fuzz_interact.sh --jobs [number of jobs] [other arguments to cargo fuzz]
[lx@pc fuzz]$ ./fuzz_interact.sh --jobs 1
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `rustc - --crate-name ___ --print=file-names -Cpasses=sancov-module -Cllvm-args=-sanitizer-coverage-level=4 -Cllvm-args=-sanitizer-coverage-trace-compares -Cllvm-args=-sanitizer-coverage-inline-8bit-counters -Cllvm-args=-sanitizer-coverage-pc-table --cfg fuzzing -Clink-dead-code -Zsanitizer=address -Cllvm-args=-sanitizer-coverage-stack-depth -Cdebug-assertions -C codegen-units=1 -C link-arg=-fuse-ld=lld --target wasm32-wasi --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (exit status: 1)
  --- stdout
  ___.wasm
  lib___.rlib
  ___.wasm
  lib___.a
  /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu
  debug_assertions
  fuzzing
  panic="abort"
  proc_macro
  sanitize="address"
  target_abi=""
  target_arch="wasm32"
  target_endian="little"
  target_env=""
  target_family="wasm"
  target_has_atomic="16"
  target_has_atomic="32"
  target_has_atomic="64"
  target_has_atomic="8"
  target_has_atomic="ptr"
  target_has_atomic_equal_alignment="16"
  target_has_atomic_equal_alignment="32"
  target_has_atomic_equal_alignment="64"
  target_has_atomic_equal_alignment="8"
  target_has_atomic_equal_alignment="ptr"
  target_has_atomic_load_store="16"
  target_has_atomic_load_store="32"
  target_has_atomic_load_store="64"
  target_has_atomic_load_store="8"
  target_has_atomic_load_store="ptr"
  target_os="wasi"
  target_pointer_width="32"
  target_thread_local
  target_vendor="unknown"

  --- stderr
  error: address sanitizer is not supported for this target

  warning: dropping unsupported crate type `dylib` for target `wasm32-wasi`

  warning: dropping unsupported crate type `proc-macro` for target `wasm32-wasi`

  error: aborting due to previous error; 2 warnings emitted

Error: failed to build fuzz script: "cargo" "build" "--manifest-path" "/home/lx/Deuxfleurs/kannader/smtp-server/fuzz/Cargo.toml" "--target" "x86_64-unknown-linux-gnu" "--release" "--all-features" "--bin" "fuzz_interact"

[lx@pc kannader]$ cargo test -p integration
   Compiling zstd-safe v2.0.6+zstd.1.4.7
   Compiling zstd-sys v1.4.18+zstd.1.4.7
   Compiling wasmtime-runtime v0.22.0
   Compiling wasmtime-obj v0.22.0
error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:574:21
    |
574 |             .map_or(true, |n| n as usize > elem.len())
    |              ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |              |
    |              required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:575:44
    |
575 |             || dst.checked_add(len).map_or(true, |m| m > table.size())
    |                                     ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |                                     |
    |                                     required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:621:21
    |
621 |             .map_or(true, |n| n as usize > src_mem.current_length)
    |              ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |              |
    |              required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:624:25
    |
624 |                 .map_or(true, |m| m as usize > dst_mem.current_length)
    |                  ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |                  |
    |                  required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:659:21
    |
659 |             .map_or(true, |m| m as usize > memory.current_length)
    |              ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |              |
    |              required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:723:21
    |
723 |             .map_or(true, |n| n as usize > data.len())
    |              ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |              |
    |              required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:726:25
    |
726 |                 .map_or(true, |m| m as usize > memory.current_length)
    |                  ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |                  |
    |                  required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

   Compiling zstd v0.5.4+zstd.1.4.7
error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
    --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:1272:21
     |
1272 |             .map_or(true, |end| end > table.size() as usize)
     |              ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
     |              |
     |              required by a bound introduced by this call
     |
note: required by a bound in `Option::<T>::map_or`
    --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
     |
933  |         U: ~const Drop,
     |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
    --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/instance.rs:1363:21
     |
1363 |             .map_or(true, |end| end > memory.current_length)
     |              ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
     |              |
     |              required by a bound introduced by this call
     |
note: required by a bound in `Option::<T>::map_or`
    --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
     |
933  |         U: ~const Drop,
     |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `(): std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/table.rs:153:60
    |
153 |                 let slot = x.get_mut(index as usize).ok_or(())?;
    |                                                      ----- ^^ the trait `std::ops::Drop` is not implemented for `()`
    |                                                      |
    |                                                      required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::ok_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:997:12
    |
997 |         E: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::ok_or`

error[E0277]: the trait bound `(): std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/table.rs:157:60
    |
157 |                 let slot = x.get_mut(index as usize).ok_or(())?;
    |                                                      ----- ^^ the trait `std::ops::Drop` is not implemented for `()`
    |                                                      |
    |                                                      required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::ok_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:997:12
    |
997 |         E: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::ok_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/table.rs:181:21
    |
181 |             .map_or(true, |n| n > src_table.size())
    |              ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |              |
    |              required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

error[E0277]: the trait bound `bool: std::ops::Drop` is not satisfied
   --> /home/lx/.cargo/registry/src/github.com-1ecc6299db9ec823/wasmtime-runtime-0.22.0/src/table.rs:184:25
    |
184 |                 .map_or(true, |m| m > dst_table.size())
    |                  ------ ^^^^ the trait `std::ops::Drop` is not implemented for `bool`
    |                  |
    |                  required by a bound introduced by this call
    |
note: required by a bound in `Option::<T>::map_or`
   --> /home/lx/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/option.rs:933:12
    |
933 |         U: ~const Drop,
    |            ^^^^^^^^^^^ required by this bound in `Option::<T>::map_or`

For more information about this error, try `rustc --explain E0277`.
error: could not compile `wasmtime-runtime` due to 13 previous errors
warning: build failed, waiting for other jobs to finish...
error: build failed

Alexis211 commented 2 years ago

Ok, I managed to run the integration test in nix-shell, and it passes now. Still no luck with the fuzzer :(

Ekleog commented 2 years ago

No problem! Great that you've been able to run the integration tests, and I'll need to rework the fuzzers in order to use cargo-bolero instead of cargo-fuzz anyway, so let's forget about them :) (I'm having the same issue locally)

Alexis211 commented 2 years ago

Shoud be good this time :) Yes the simplified lifetimes look much better!

Alexis211 commented 2 years ago

Great news! Yes, a release on crates.io could be useful for us, if it's not too much work for you

Ekleog commented 1 year ago

Ok, so it's taken me a while, sorry about that! I wanted to first upgrade dependencies, to not push crates with too outdated dependencies, and just completed this work (wasmtime completely changed its ownership model in 1.0)

smtp-server is now released as 0.1 on crates.io :) Not all crates are yet (due to https://github.com/rust-lang/cargo/issues/11203 that I hit while publishing), but hopefully this is enough to help you a bit!

Alexis211 commented 1 year ago

Thanks @Ekleog for the great work!

Ekleog / kannader

Extend `smtp-server`'s API to enable implementation of LMTP servers as well #178