search

project-oak / oak

Meaningful control of data in distributed systems.
Apache License 2.0
1.32k stars 113 forks source link

Handle panics in Oak Functions loader #1992

Closed tiziano88 closed 3 months ago

tiziano88 commented 3 years ago

We should ensure that no matter what happens while executing the Wasm module, there is no way to make the server (in production mode) panic, or produce any other macroscopically observable effect.

Perhaps we should also set up fuzzing, though I am not sure whether fuzzing the Wasm bytecode would be a productive way of doing this. We could also fuzz user requests, given a fixed Wasm module.

cc @alastairreid

_Originally posted by @conradgrobler in https://github.com/project-oak/oak/pull/1991#discussion_r617271100_

alastairreid commented 3 years ago

One of my takeaways from the Rust-for-Linux RFC discussion on LKML is that trading liveness for safety is not ideal for a server/kernel. That is, panicking when something bad happens (eg array bounds check fails) prevents something bad happening but it may also cause loss of data, loss of logging (eg an attacker might quickly perform an attack and then crash the kernel before logging info can be saved), denial of service, and similar which are maybe just as bad as the original problem.

So, yes, really important to try to eliminate panics from the entire Rust codebase. A combination of fuzzing and our formal verification tools is the answer.

wrt Bytecode... The thing not to do is generate random files of bytes and call it good. You will find lots of ways to create malformed bytecode but you will only be getting coverage over the bytecode loader. You need multiple layers of fuzzing - each layer bypassing the issues that the previous layers have already checked.

  1. Random byte files - to check for errors in the bytecode loader
  2. Correctly structured wasm files - with type errors, overly large stack offsets as immediate fields, illegal opcodes, etc.
  3. Well formed wasm that probes memory boundaries, etc.
  4. Random sequences of commands. eg if the Oak API provides functions F(,), G(_) and H(), then you generate random sequences like [("F", [42, 87]), ("H", []), ("F", [234231,-5]), ("H", [])] and supply that to a Wasm program that loops over the list applying the named function to the list of arguments. (Variations on this approach exist - but that's the basic idea)

(Did a variation of (4) on a microcontroller OS before - found some issues.)