More flexible Rust<>Cairo types interface

[bilboquet]

From Ariel Elperin

• Support sending various uints (not just u8) to riscv call (potentially make it generic over the param type, where the param type has to implement some kind of Serde trait)
• User defined structs
• The endgame here could be something like a type definition by the dev, e.g. in a proto file, and a cli command that generates rust + cairo modules with those types. That way, you know that whatever is generated can be used as arguments/return values of the externalized rust functions and as arguments/return values of riscv_call

[bilboquet]

@damip please share your thoughts about this:

Let's focus on the use of a proto file to describe the interface between rust and cairo code.

On the rust side, we need

1. a proto to rust code generator which is used at compile time
2. a runtime support library written in rust to help serialize/deserialize data in protobuf format (that will be executed inside the riscv VM).
3. some basic code (the actual format of a risc_call is [u8]->[u8]).

Points 1 and 2 can be filled by https://docs.rs/micropb/0.1.0/micropb/ (and integration is almost complete). Point 3 is not a big problem and can even be done manually by the project developer.

On the cairo side, we have the same requirements. Should we start writing a proto for the cairo code generator (in rust) and the corresponding cairo runtime (in cairo)?

[damip]

OK For the rust side.

We need to check how proto works on the cairo side. Looks like it is already used natively for hints: https://reilabs.io/blog/introducing-hints-in-cairo-programming-language/

https://github.com/reilabs/cairo-hints

In particular, here is the cairo proto codegen they use, conveniently written as a rust crate: https://github.com/reilabs/cairo-hints/tree/main/cairo-proto-build but let's make sure it generates the right version of cairo

[bilboquet]

@damip Integration of cairo-proto-build went relatively fine. But I just found that they are based on a different language edition (i.e. 2023_10) and that was the cause of the pain I had lately:

Something related to the visibility rules changed between edition 2023_10 and 2024_07: given this cairo code:

// module mod1
struct A {
    a: u8,
}

// module main
mod mod1;
use mod1::A;

fn main2() -> bool {
    let x = A { a: 1 };
    x.a == 1
}

This code builds in edition 2023_10 but fails in edition 2024_07 because both A and A.a are not visible anymore. This would work (but pub has to be added manually for the moment):

// module mod1
pub struct A {
    pub a: u8,
}

[bilboquet]

Added an issue to cairo-hints project https://github.com/reilabs/cairo-hints/issues/35

[damip]

@bilboquet could we work on a fork and propose a PR upstream ? To avoid blocking our progress

[bilboquet]

I'm going to do that. It's probably not too hard.

EDIT1: @damip I've finally figured out how Cairo-Hints' black magic works, and it doesn't fit our needs. Cairo-Hints uses the cheatcode feature that cairo provide for tests. Basically a cheatcode in cairo, allows manipulating the internal states of the cairo vm. cairo-hints use this to:

• take some data serialized in the cairo program in the cairo format (so a list of felt)

  data = [
  Felt(
      FieldElement {
          value: UnsignedInteger {
              limbs: [
                  576460752302463888,
                  18446744073709551615,
                  18446744073709551615,
                  18446744073709495169,
              ],
          },
      },
  ),
  ]

• deserialize them into

  data = Object {
  "n": Number(1764),
  }

• make a json object with those data then call the rpc method

• on the server side, the received data are "payload [8, 228, 13]" which looks more like a protobuf encoded stuff (varint encoding of 1764). (so another conversion seems to happen in between)

For the response, the scheme is the same in the reverse order.

For the rust code to operate correctly on data, we need to

• (de)serialize to bytes: this must be done in cairo (and in rust)
• convert from felts to rust types (back and forth): provided we have the to_bytes/from_bytes, it may be possible to do this only on one side (TBC)

EDIT2: How to serialize to bytes example https://github.com/kkrt-labs/kakarot-ssj/blob/9203b9c29c5e049d556cafdb3554929f7c719143/crates/utils/src/traits/bytes.cairo#L176-L459

[bilboquet]

@damip sharing my mind in #9 on how to go forward about this issue.