esp-rs / esp-idf-svc

Type-Safe Rust Wrappers for various ESP-IDF services (WiFi, Network, Httpd, Logging, etc.)
https://docs.esp-rs.org/esp-idf-svc/
Apache License 2.0
303 stars 172 forks source link

ESP Modem for Rust (LTE / GSM / Sim Card) #468

Open DaneSlattery opened 1 month ago

DaneSlattery commented 1 month ago

The ESP-Modem Component is not supported.

It would enable SMS+GSM +LTE connections to work with the current Netif interface.

I have started calling into the the esp-modem component, and figured it would be good to wrap this up for use in esp-idf-svc.

The code in [netif.rs](https://github.com/esp-rs/esp-idf-svc/blob/bf47d2bcee6e1bc71530a23264eae63a7af84aae/src/netif.rs#L34C5-L36C9) suggests support for PPP mode, but currently there are no ways to create that interface.

I am happy to start development, but I think this may be a pretty big project and would appreciate some help fitting this into current components. The official esp-modem component is quite chunky: https://github.com/espressif/esp-protocols/tree/master/components/esp_modem.

ivmarkov commented 1 month ago

Unfortunately I haven't played with these modems yet, so below my very limited understanding:

Probably this is a start?

Also, are you only interested in modems for internet connectivity? I assume modems can also be used for simple SMS send/receive? Not sure how calling / receiving voice calls work over these, if supported at all... Anyway, these are probably corner cases...

Yes the project is large-ish, but maybe not that much after all...

ivmarkov commented 1 month ago

Oh. One more thing but maybe important - given that ALL communication with the modem is basically over a UART channel (sending and receiving bytes of data) - is there any pure-Rust crate that implements this? Even if this crate (obivously) does not talk to ESP Netif's PPP out of the box, it can be made to do so.

I'm mentioning this because if there exists a well-supported Rust crate that does the AT comm with the modem, it might be easier and cleaner to integrate with it instead, rather than going down the ESP IDF C route again.

ivmarkov commented 1 month ago

Interesting. And sounds rather simple.

ivmarkov commented 1 month ago

Also this is interesting. Given that @diondokter is active in the Embassy Matrix channel, why don't you ping him in there? Given his library, I'm sure he had dealt with a modem or two and their pesky AT-command interface. :)

@diondokter Sorry maybe even better to also ping you here (as I actually already did, from above):

The TL;DR is - we are contemplating wrapping in Rust an ESP IDF C library called "ESP Modem" that does the heavy lifting of configuring a modem via AT commands so that eventually it is switched to PPP at the end (this always happens, right?) and then once the UART channel is in "PPP mode" so to say, we we can further hook it up with the ESP IDF networking layer (this is not embassy-net but this is not really important) so that we can do IP networking.

However - this all might be unnecessary, and might just bring more unsafe C code and a lot of work for us, I fear.

... the one remaining challenge is how to configure the modem via AT commands. Given your crate I referenced above, you've surely dealt with this in the past. Do you have any link / example code how this is typically done? (Ideally, with your "at-commands" crate :) ?

If the example is with - say - embassy-net + embassy-net-ppp we can still understand that, and we'll translate it all to the ESP IDF context. It is the "AT part" which is interesting to us...

diondokter commented 1 month ago

Hi, it really depends on your modem! I used the at-commands crate in https://github.com/diondokter/nrf-modem which is a library that wraps the Nordic libmodem C library. I believe you can do the thing you propose here where you can setup the modem and then drop into a uart stream for TCP data or something lower level.

With nrf-modem however, there are actual APIs that you can use. So a socket is just a socket and that doesn't use the at commands. At-commands are used only for things where no API is available.

Example for building a command: https://github.com/diondokter/nrf-modem/blob/ec738c3762974bfd3559becdc785907cafff07e4/src/lib.rs#L305-L314 Example for parsing a command: https://github.com/diondokter/nrf-modem/blob/ec738c3762974bfd3559becdc785907cafff07e4/src/lib.rs#L178-L184

I'm not familiar with the ESP modem though, so the only tip I can give is to make your interface to the modem nice to use. Modems are already notoriously frustrating at times, so don't add to that with a cumbersome interface. I'm not quite sure what the embassy-net-ppp interface is like, but I'd try to make the modem device struct be able to split into two channels:

The AT channel you can then use to setup the modem and query it for information. The PPP channel can then be handed to the embassy-net-ppp library. The channels themselves then coordinate the state of the uart and the switching of modes of the modem.

Not sure what else I can add... But feel free to ask anything

ivmarkov commented 1 month ago

Not sure what else I can add... But feel free to ask anything

That was already plenty, thanks a lot!

Regarding PPP, this is just a way to tunnel IP traffic (IP packets) over a point-to-point connection, which in our case would be between the host (The ESP MCU) and the modem itself (which is whatever modem is connected over UART to the ESP).

I was imagining that's all there is to it (1- AT commands to setup the modem and then 2- a PPP channel for encapsulating the IP traffic/packets)... but then, looking at your Nordic modem wrapper, you have a lot of code in there, including stuff like TCP socket struct, UDP socket struct and whatnot.

... which brings the question... Why is any of that necessary actually? If indeed the IP traffic between the modem and the host (ESP or whatever it is, could be even Linux) is just flowing over a PPP tunnel, then you can use any IP/TCP/UDP library that can packetize/depacketize IP over PPP. As in:

... so in a way you won't need to model your own TCP or UDP socket. That would come from whatever IP/TCP/UDP stack library you choose to use over the PPP tunnel...? So where is the complexity in your case coming from?

diondokter commented 1 month ago

I was imagining that's all there is to it (1- AT commands to setup the modem and then 2- a PPP channel for encapsulating the IP traffic/packets)... but then, looking at your Nordic modem wrapper, you have a lot of code in there, including stuff like TCP socket struct, UDP socket struct and whatnot.

In your case that is indeed all there is to it.

... which brings the question... Why is any of that necessary actually? If indeed the IP traffic between the modem and the host (ESP or whatever it is, could be even Linux) is just flowing over a PPP tunnel, then you can use any IP/TCP/UDP library that can packetize/depacketize IP over PPP.

Yep, you're right

... so in a way you won't need to model your own TCP or UDP socket. That would come from whatever IP/TCP/UDP stack library you choose to use over the PPP tunnel...? So where is the complexity in your case coming from?

While it's possible to use PPP on the nRF91, it's not needed. It provides its own socket APIs with which you can do TCP, UDP and more. This TCP/IP stack then runs on the modem itself. This has two advantages:

If I ran the TCP/IP on the host core, then that takes away compute and it makes the firmware bigger. There still might be good reason to want to run your own stack, for example when the built-in stack is buggy or lacks some feature, but I've seen no issue in using the existing socket APIs.

For you though, you don't seem to have a choice since the ESP modem doesn't list anything about a built-in TCP/IP stack.

ivmarkov commented 1 month ago

@diondokter Crystal clear, thanks a ton!

@DaneSlattery I am somehow leaning towards seriously evaluating first if we can implement the modem in pure Rust. (I'm already regretting a bit that for the one-wire we did not instead expose the new RMT driver in Rust and then just write on top a pure-Rust one-wire impl that can simply operate on top of two APIs: send_bit and recv_bit. As that would've allowed us to easily re-target the pure Rust code to operate over - say - UART, as you can do send_bit / recv_bit over UART as well.)

Also in general, my feeling w.r.t. ESP IDF is that:

DaneSlattery commented 1 month ago

@ivmarkov I agree that the onewire rmt approach could be based on this API, and I would like to get to deprecating the IDF-v4 RMT driver in favour of IDF-v5, that would have simplified my approach. That said, working code is better than analysis paralysis 👍 , and I wanted this feature for my own projects. Nothing wrong with refactoring later. (I do wonder if this project has an overall direction and who is steering the wheel, would love to hear more about that working group).

As for a pure-rust implementation, I think given the information:

  1. The existing esp-idf-hal UART is a suitable base
  2. AT commands can be written in pure rust in a similar fashion to this: https://github.com/espressif/esp-protocols/blob/master/components/esp_modem/include/generate/esp_modem_command_declare.inc . I think even a small subset of these can be supported. I would use https://github.com/diondokter/at-commands as a base
  3. I would ignore cmux and vfs support for now (mostly because I can't test them, and they seem like an advanced edge case).
  4. PPP Mode should probably be based on the idf-lwip stack for now, although I agree that edge-net is a really solid library.
ivmarkov commented 1 month ago

@ivmarkov I agree that the onewire rmt approach could be based on this API, and I would like to get to deprecating the IDF-v4 RMT driver in favour of IDF-v5, that would have simplified my approach. That said, working code is better than analysis paralysis 👍 , and I wanted this feature for my own projects. Nothing wrong with refactoring later.

If I recall correctly, the approach where the new RMT driver needed to be exposed was simply deemed much more effort to implement by you. Hence why we ended up where we are. Which is OK, we still did a good progress! But we definitely did not have analysis-paralysis, I believe.

(I do wonder if this project has an overall direction and who is steering the wheel, would love to hear more about that working group).

There is no "steering committee" of sorts, because Espressif is taking a rather opportunistic / tactical approach towards its Rust-related portfolio. This is also valid for the bare-metal crates which do have paid developers working on those.

esp-idf-* is a community effort, and I guess I'm the most active one here, who historically contributed the largest portion of the code, but we also now have @Vollbrecht as an active committer and a few (now inactive) committers who contributed large portions as well (as in the esp-idf-sys "native" build and components that you are using - these originally came from @N3xed).

Do I miss a more "architectural" / "bigger-horizon" approach which can be discussed among more community members? I certainly do, but this requires a lot of prior exposure to the esp-idf-* crates (a huge time investment) and in the absence of Espressif steering it, it is what it is - i.e. the ones who sticked around for the longnest (and are vocal) tend to define the direction.

As in, there are bi-weekly meetings of the esp-rs community (this include both bare-metal and ESP-IDF crates) which are open for everyone that you might want to attend. But I fear these are more of a "sprint ceremony" "what was done last sprint" kind of a thing - primarily for the bare-metal team, rather than a forum where direction is being discussed. We can change that of course, but to change it, there must be more people that feel the change is useful and necessary, and I feel I'm in the minority there. :)

Anyway, I digress.

As for a pure-rust implementation, I think given the information:

  1. The existing esp-idf-hal UART is a suitable base
  2. AT commands can be written in pure rust in a similar fashion to this: https://github.com/espressif/esp-protocols/blob/master/components/esp_modem/include/generate/esp_modem_command_declare.inc . I think even a small subset of these can be supported. I would use https://github.com/diondokter/at-commands as a base
  3. I would ignore cmux and vfs support for now (mostly because I can't test them, and they seem like an advanced edge case).

Can you elaborate what cmux and vfs support in the ESP Modem C driver is about, if you've looked into it?

  1. PPP Mode should probably be based on the idf-lwip stack for now, although I agree that edge-net is a really solid library.

I did not mention edge-net as an alternative to the ifd-lwip stack as these are two different animals:

DaneSlattery commented 1 month ago

Thank you for the detail on the management structures. It is interesting to see. It seems like no_std is a key focus for espressif, and I wonder at what point it will supercede the rust-idf approach.

Here's what I have on cmux, which seems useful for keeping the UART in AT and PPP mode simultaneously. https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/applications/serial_lte_modem/doc/CMUX_AT_commands.html

And VFS: /* The VFS terminal is just a demonstration of using an abstract file descriptor

Here are some more relevant crates:

ivmarkov commented 1 month ago

Thank you for the detail on the management structures. It is interesting to see. It seems like no_std is a key focus for espressif, and I wonder at what point it will supercede the rust-idf approach.

It is. The ESP-IDF is viewed as a stop-gap solution, and obviously the solution if you already have a large C codebase that you can't or don't want to migrate - in one go or at all.

With that said, it's been like that for many years.

What I'm personally doing is trying to abstract from the underlying "OS" as much as possible - with e-hal, or by rolling your own traits etc. And by using async IO as much as possible as no other embedded platform besides esp-idf-* has threads (and STD). So that I can migrate between ESP-IDF and baremetal, or even to other MCUs.

Hence - the more code we have in Rust itself - the better for everyone!

Here's what I have on cmux, which seems useful for keeping the UART in AT and PPP mode simultaneously. https://docs.nordicsemi.com/bundle/ncs-latest/page/nrf/applications/serial_lte_modem/doc/CMUX_AT_commands.html

And VFS: / The VFS terminal is just a demonstration of using an abstract file descriptor which implements non-block reads, writes and selects to communicate with esp-modem. This configuration uses the same UART driver as the terminal created by create_uart_dte(), so doesn't give any practical benefit besides the FD use demonstration and a placeholder to use FD terminal for other devices /

Agreed that none of these is essential. Even CMUX - at least initially.

Here are some more relevant crates:

Yes this is the PPP impl for embassy. If your AT-commands code is in pure Rust (you can even abstract the UART by using just the async IO Read/Write traits), we can even write connectors for your code to embassy-PPP.

https://github.com/FactbirdHQ/atat/tree/master (usage here : https://github.com/atlas-aero/rt-esp-at-nal/blob/main/src/commands.rs) going to start here.

Sounds like a plan!

See, if everything goes the way we hope it goes, we might not even have to PR anything in esp-idf-svc. It might turn out that your code can live as an external crate, and just needs 3 lines of code or so for the PPP glue!

DaneSlattery commented 1 month ago

I have started writing out some AT commands over a UART device and getting replies. I will borrow a few commands from https://github.com/technocreatives/sim7000/tree/main but implement them using the atat traits.
I think it makes sense to impl atat::blocking::AtatClient and atat::asynch::AtatClient for the esp-idf-hal uart. I do think that whatever code we write here should also target something that impls embedded_io::Read + Write , that way it can be quite general.

What I am not sure of is how to glue the modem when it is in ppp mode to a netif. The docs say the general flow is:

Create a DCE instance (using esp_modem_new())

Call specific functions to issue AT commands (Modem commands)

Switch to the data mode (using esp_modem_set_mode())

Perform desired network operations (using standard networking API, unrelated to ESP-MODEM)

Optionally switch back to command mode (again esp_modem_set_mode())

Destroy the DCE handle (sing esp_modem_destroy())

I see this in the c-api: and this)

ivmarkov commented 1 month ago

I have started writing out some AT commands over a UART device and getting replies. I will borrow a few commands from https://github.com/technocreatives/sim7000/tree/main but implement them using the atat traits.

Great progress!

I think it makes sense to impl atat::blocking::AtatClient and atat::asynch::AtatClient for the esp-idf-hal uart.

I don't think this would be necessary, because the machinery around AtatClient (both blocking and async) just expects you pass in something which does implement embedded_io::Read + embedded_io::Write (for the blocking case) or embedded_io_async::Read + embedded_io_async::Write (for the async case). And the key is, that UartDriver already implements embedded_io::Read + embedded_io::Write, while AsyncUartDriver already implements embedded_io_async::Read + embedded_io_async::Write.

With that said, your mileage may vary, and we have to see how it is shaping up. Two reasons for that:

What I am not sure of is how to glue the modem when it is in ppp mode to a netif. The docs say the general flow is:

Create a DCE instance (using esp_modem_new()) Call specific functions to issue AT commands (Modem commands) Switch to the data mode (using esp_modem_set_mode())

It should boil down to the following:

Perform desired network operations (using standard networking API, unrelated to ESP-MODEM)

As per above, once you do the above ^^^ then you have a netif (network interface) which is attached to the ESP IDF LwIP stack and to/from which network traffic is routed. So you can use regular Rust STD API to open sockets and whatnot. That is, as long as you keep the netif instance and the driver attached to it alive.

Optionally switch back to command mode (again esp_modem_set_mode())

Let's wait with this.

Destroy the DCE handle (sing esp_modem_destroy())

I suggest not to use the ESP Modem component for anything but inspiration (copy-paste) of C code and then translation of that code to Rust unsafe calls into esp-idf-sys. Let's not think for now how to destroy our "netif" instance and the driver which is attached into it. We'll get there.

I see this in the c-api: and this)

Yes, exactly! This is from where you need to "copy-paste" code into Rust. :)

ivmarkov commented 1 month ago

By the way... (and no offense to the atat crate), but you might have a better luck and easier life by using the lower-level crate described here.

After all, all that you need is a utility, that converts an AT command to a sequence of bytes that you can then send via UartDriver::write (= embedded_io::Write) OR AsyncUartDriver::write (= embedded_io_async::Write) if you are using async; and the other way around - a utility that converts an sequence of bytes to an AT command response, where the sequence of bytes is coming from UartDriver::read (= embedded_io::Read) OR AsyncUartDriver::read (= embedded_io_async::Read) if you are using async.

All of these extra layers of abstractions in atat would either help you, or get in the way, especially considering the complicated "ingress" thing which you don't need and this is really a thing of the past: :D

So what is the point?

DaneSlattery commented 1 month ago

By the way... (and no offense to the atat crate), but you might have a better luck and easier life by using the lower-level crate described https://github.com/esp-rs/esp-idf-svc/issues/468#issuecomment-2282004218.

One thing I really like from atat are the atat::AtatCmd and atat::AtatResp traits and their derive macros. This allows for more "static" definitions of the commands, rather than building them on the fly.

for example:


/// 4.1 Manufacturer identification +CGMI
///
/// Text string identifying the manufacturer.
#[derive(Clone, AtatCmd)]
#[at_cmd("+CGMI", ManufacturerId)]
pub struct GetManufacturerId;

/// 4.1 Manufacturer identification
/// Text string identifying the manufacturer.
#[derive(Clone, Debug, AtatResp)]
pub struct ManufacturerId {
    pub id: String<64>,
}

impl<'d, T> EspModem<'d, T>
where
    T: embedded_svc::io::Read + embedded_svc::io::Write,
{
    pub fn new(serial: &'d mut T) -> Self {
        Self {
            serial,
            _d: PhantomData,
        }
    }

    pub fn send_cmd<CMD: AtatCmd>(&mut self, cmd: &CMD) -> Result<CMD::Response, atat::Error> {
        let mut buff = [0u8; 64];
        // flush the channel
        while self
            .serial
            .read(&mut buff)
            .map_err(|_err| atat::Error::Read)?
            > 0
        {}
        // write the command to the uart
        let len = cmd.write(&mut buff);
        self.serial
            .write(&buff[..len])
            .map_err(|_err| atat::Error::Write)?;

        // now read the uart to get the response

        let len = self
            .serial
            .read(&mut buff)
            .map_err(|_err| atat::Error::Read)?;

        cmd.parse(Ok(&buff[..len]))
    }
}

This gives us strong typing of the commands and responses, but the trait is a bit bloated:

pub trait AtatCmd {
    /// The type of the response. Must implement the `AtatResp` trait.
    type Response: AtatResp;

    /// The size of the buffer required to write the request.
    const MAX_LEN: usize;

    /// Whether or not this command can be aborted.
    const CAN_ABORT: bool = false;

    /// The max timeout in milliseconds.
    const MAX_TIMEOUT_MS: u32 = 1000;

    /// The max number of times to attempt a command with automatic retries if
    /// using `send_retry`.
    const ATTEMPTS: u8 = 1;

    /// Whether or not to reattempt a command on a parse error
    /// using `send_retry`.
    const REATTEMPT_ON_PARSE_ERR: bool = true;

    /// Force client to look for a response.
    /// Empty slice is then passed to parse by client.
    /// Implemented to enhance expandability of ATAT
    const EXPECTS_RESPONSE_CODE: bool = true;

    /// Write the command and return the number of written bytes.
    fn write(&self, buf: &mut [u8]) -> usize;

    /// Parse the response into a `Self::Response` or `Error` instance.
    fn parse(&self, resp: Result<&[u8], InternalError>) -> Result<Self::Response, Error>;
}

I feel like the at-commands library could perhaps get it's own traits.

diondokter commented 1 month ago

Kept lurking...

Probably not the place to discuss it, but the at-commands doesn't do anything with the transport of the commands. I'm not sure what it would need a trait for? Or are you saying it should model the transport layer?

I'd be open to make it use the embedded-io read and write traits instead of operating on a buffer directly. That could also help with hooking it up to a uart stream.

ivmarkov commented 1 month ago

Beauty is in the eye of the beholder, I guess, because I like this builder pattern much more. You can just call it from your modem code, and it will just serialize the commands directly in the buff you supply. Incrementally. No intermediate (potentially large-ish) objects, no nothing.

But it is you who are implementing it, so it is your choice of course.

ivmarkov commented 1 month ago

Kept lurking...

Probably not the place to discuss it, but the at-commands doesn't do anything with the transport of the commands. I'm not sure what it would need a trait for? Or are you saying it should model the transport layer?

I'd be open to make it use the embedded-io read and write traits instead of operating on a buffer directly. That could also help with hooking it up to a uart stream.

I - personally - like the at-commands crate as-is. I.e. it does not deal with IO. Hooking it with embedded_io(_async) is so trivial, that it is not worth it to be part of the crate. After all, it is just 2-3 lines of code.

Now, if communicating with the AT modem requires a complex re-try logic, then it might make sense, as this can't be modeled without introducing embedded_io(_async) into the picture, but I don't know whether that's true or not. atat does seem to model some sort of retry logic with each command...

DaneSlattery commented 1 month ago

I have worked it for the day and come up with an idea for the at-commands @diondokter. The builders are actually quite ergonomic.

The traits I was talking about was not about the transport layer, more along these lines. I like the fact that commands and their responses are linked in atat, so perhaps a higher level API can wrap them:


pub trait Cmd {
    /// The type of the response. Must implement the `Resp` trait.
    type Response: Resp;
    /// Write the command and return the number of written bytes.
    fn write(&self, buf: &mut [u8]) -> usize;

    /// Parse the response into a `Self::Response` or `Error` instance.
    fn parse(&self, resp: &[u8]) -> Result<Self::Response, Error>;
}
pub trait Resp {}

pub struct GetSignalQuality;

pub struct SignalQuality{
    ber: i32,
    rssi: i32,
};

impl Resp for SignalQuality {}

impl Cmd for GetSignalQuality {
    type Response = SignalQuality;
    fn parse(&self, resp: &[u8]) -> Result<Self::Response, Error> {
          let (rssi, ber) = CommandParser::parse(&buff[..len])
            .expect_identifier(b"\r\n+CSQ: ")
            .expect_int_parameter()
            .expect_int_parameter()
            .expect_identifier(b"\r\n\r\nOK\r\n")
            .finish()
            .unwrap();
        Ok(SignalQuality{rssi,ber})
    }

    fn write(&self, buf: &mut [u8]) -> usize {
        CommandBuilder::create_execute(&mut buf, true)
            .named("+CSQ")
            .finish()
            .unwrap();
    }
}

The part I seem to be after is more const functions, especially for builders. I don't think I should build a new signal quality command every time if the named value never changes. The command I build then is also not so easily shared with another device that might have the same command name but a different terminator (serde could help with this).

It might also be interesting to have the ability to configure delimiters for parsers and builders. For example, every response on the simcom7600 series replies with \r\n<REPLY>\r\n\r\nOK\r\n, so I expect that identifier with every call, but I must write every parser to expect that.

It might also be interesting to support a serde style serializer/deserializer with the above traits to account for more complex commands/responses.

That said, none of this is really hindering me. I will create a PR for this so that we may see some code

DaneSlattery commented 4 weeks ago

It should boil down to the following:

Once you send ATD*99# and you get an "ok" reply from the modem, you should assume that what is now flowing over UART read/write immediately after that is PPP traffic, not AT-commands text traffic So after the above ^^^ had happened, you need to: Instantiate a new EspNetif of type PPP with EspNetif::new_with_conf(&NetifConfiguration::ppp_client()) (for now, to keep it simple). Then the non-trivial stuff happens, as we need to follow the ESP modem C code and we'll do some unsafe calls into esp_idf_sys APIs as we don't have safe wrappers for these (yet). The "non-trivial stuff" is essentially attaching to your just-created PPP netif a netif "driver", which seems to be nothing else but a way to instruct your PPP netif what to call when it needs to send/transmit data. (for receiving, we'll just manually call esp_netif_receive() to "ingest" data in the ppp-netif) The key seems to be to call esp_netif_attach on your ppp-netif, and supply to this call also a prepared C struct which describes your "driver" - the C struct for the driver is this. Without going into further level of details, as I might mess it up without actually implementing it, you need to translate to Rust three things: The code in this constructor The code in this "post-attach" function, which the esp-idf PPP netif call will calback for you, as you pass a reference to this function when you call esp_netif_attach. This will take care of sending data from the netif into the UART. For receiving data from UART and ingesting it into the netif, you need to spin a loop {} which calls UartDriver::read and then calls esp_netif_receive (I think)

So I am completely stuck here with post_attach and the ppp_netif_driver translation, I don't know enough about FFI for this to make a netif glue. I did manage to get the modem into data mode successfully though, but hooking up the uart send and receive is out of reach for me right now. I will continue hacking away at it, but welcome input from anyone with more experience on FFI and rust-c interop.

ivmarkov commented 4 weeks ago

So I am completely stuck here with post_attach and the ppp_netif_driver translation, I don't know enough about FFI for this to make a netif glue. I did manage to get the modem into data mode successfully though, but hooking up the uart send and receive is out of reach for me right now. I will continue hacking away at it, but welcome input from anyone with more experience on FFI and rust-c interop.

I had to shoot in the dark a bit as I weren't sure what exactly is the problematic aspect. Hopefully I was right and this feedback would be helpful. If not (or if you have additional questions), I would gladly address those.

ivmarkov commented 4 weeks ago

Also see this (NOTE: not typechecked!) which is what I now believe we are ultimately trying to achieve over here! :)

ivmarkov commented 4 weeks ago

@DaneSlattery I decided it would be fair to at least typecheck my code, so you can (try to) use my EspNetifDriver idea from... esp-idf-svc master.

(We have to prohibit direct pushes to master; instead of git checkout -b netif-driver I did git checkout -p netif-driver and... here you go, my untested code ended up on master directly... oh well.)

To use the EspNetifDriver, you need to put the following in the Cargo.toml of your binary crate:

[patch.crates-io]
esp-idf-hal = { git = "https://github.com/esp-rs/esp-idf-hal" }
esp-idf-sys = { git = "https://github.com/esp-rs/esp-idf-sys" }

...and then in your binary crate you need to replace the ref to esp-idf-svc from crates.io with a ref from esp-idf-svc GIT

... or just also add

esp-idf-svc = { git = "https://github.com/esp-rs/esp-idf-svc" }

... to your [patch.crates-io] section, but since I assume my code is not bug-free, you might want to fork esp-idf-svc master and then bugfix it, and as such you should use your own GIT fork of esp-idf-svc master.

====

Basically, you need to call EspNetifDriver::new_ppp(tx), where tx is a callback so that the driver can call you back when it had produced a PPP packet that you need to send to UART. You of course need to supply the driver with a valid EspNetif which is created with PPP in mind (look at NetifStack::Ppp::default_configuration()).

To push RX packet into the driver, you need to call EspNetifDriver::rx with stuff you read from UART. Now, whether rx would expect complete PPP packets, or can work with fractional packets (I assume the latter, or else when reading from UART you need to understand where a packet ends/stops...?) I don't know...

ivmarkov commented 4 weeks ago

Unfortunately, I cannot test this code for real, as I'm away from home, so I'm not having access even to my measly GPRS modem which is lurking around (actually not even sure this cheapo thing supports PPP, as I never tried it but...).

ivmarkov commented 4 weeks ago

@DaneSlattery If you are already using my changes, please do cargo update in the root of your binary crate. I think I found a bug, where the PPP netif does not have an important flag raised upon creation, that I just added.

DaneSlattery commented 4 weeks ago

Hi @ivmarkov

Thank you for the assistance on this! The stuff you did boggles the mind, and I will continue to look around at the various comments. Just wanted you to know I have attempted this now, and I'm currently debugging this error.

Please note that my binary is actually the lte_example in my PR. I have pushed some updates there, and did a bit of life-time modification on the EspNetifDriver.

Update, ran this without free(buffer) and it at least started. Now I need to subscribe to network events I think.

I (1498) esp_idf_svc::modem: got response [65, 84, 90, 48, 13, 13, 10, 79, 75, 13, 10]
I (2498) esp_idf_svc::modem: got response [65, 84, 69, 48, 13, 13, 10, 79, 75, 13, 10]
I (3498) esp_idf_svc::modem: got response [13, 10, 43, 67, 71, 82, 69, 71, 58, 32, 48, 44, 48, 13, 10, 13, 10, 79, 75, 13, 10]
I (3498) esp_idf_svc::modem: CGREG: n: 0stat: 0, lac: None, ci: None 
I (4508) esp_idf_svc::modem: got response [13, 10, 79, 75, 13, 10]
I (5508) esp_idf_svc::modem: got response [13, 10, 67, 79, 78, 78, 69, 67, 84, 32, 49, 49, 53, 50, 48, 48, 13, 10]
I (5508) esp_idf_svc::modem: connect Some("115200")
Guru Meditation Error: Core  0 panic'ed (LoadProhibited). Exception was unhandled.

Core  0 register dump:
PC      : 0x40380194  PS      : 0x00060133  A0      : 0x8038000a  A1      : 0x3fca5af0  
0x40380194 - tlsf_free
    at ??:??
A2      : 0x3fc971c4  A3      : 0x3fca7c60  A4      : 0x0000002d  A5      : 0x00060123  
A6      : 0x002d0018  A7      : 0x3fca7c58  A8      : 0x00010083  A9      : 0x00010082  
A10     : 0x3fca7c58  A11     : 0xfffffffc  A12     : 0x3fca7c5c  A13     : 0x00000002  
A14     : 0x3fca4e84  A15     : 0x00000000  SAR     : 0x0000001f  EXCCAUSE: 0x0000001c  
EXCVADDR: 0x002d001c  LBEG    : 0x40056f5c  LEND    : 0x40056f72  LCOUNT  : 0x00000000  

Backtrace: 0x40380191:0x3fca5af0 0x40380007:0x3fca5b10 0x40377246:0x3fca5b30 0x40381425:0x3fca5b50 0x42002f92:0x3fca5b70 0x42071137:0x3fca5b90 0x4205dce5:0x3fca5bb0 0x4205516c:0x3fca5bd0 0x420552dc:0x3fca5bf0 0x42070c51:0x3fca5c20 0x4205b733:0x3fca5c40 0x4205b78f:0x3fca5c70 0x4205a80f:0x3fca5c90 0x42054de9:0x3fca5cb0 0x42055055:0x3fca5cd0 0x42054bbf:0x3fca5cf0 0x42054c5d:0x3fca5d10 0x4205ddba:0x3fca5d30 0x4205c6f1:0x3fca5d50 0x4205c19b:0x3fca5db0 0x42047241:0x3fca5dd0
0x40380191 - tlsf_free
    at ??:??
0x40380007 - multi_heap_aligned_free
    at ??:??
0x40377246 - heap_caps_free
    at ??:??
0x40381425 - cfree
    at ??:??
0x42002f92 - esp_idf_svc::netif::driver::EspNetifDriverInner<T>::raw_tx
    at ??:??
0x42071137 - esp_netif_transmit
    at ??:??
0x4205dce5 - pppos_low_level_output
    at ??:??
0x4205516c - pppos_output_last
    at ??:??
0x420552dc - pppos_write
    at ??:??
0x42070c51 - ppp_write
    at ??:??
0x4205b733 - fsm_sconfreq
    at ??:??
0x4205b78f - fsm_lowerup
    at ??:??
0x4205a80f - lcp_lowerup
    at ??:??
0x42054de9 - ppp_start
    at ??:??
0x42055055 - pppos_connect
    at ??:??
0x42054bbf - ppp_do_connect
    at ??:??
0x42054c5d - ppp_connect
    at ??:??
0x4205ddba - esp_netif_start_ppp
    at ??:??
0x4205c6f1 - esp_netif_start_api
    at ??:??
0x4205c19b - esp_netif_api_cb
    at ??:??
0x42047241 - tcpip_thread
    at ??:??

ELF file SHA256: 000000000

Rebooting...
���ESP-ROM:esp32s3-20210327
Build:Mar 27 2021
rst:0xc (RTC_SW_CPU_RST),boot:0x8 (SPI_FAST_FLASH_BOOT)
Saved PC:0x40376cc0
0x40376cc0 - esp_restart_noos
    at ??:??
SPIWP:0xee
mode:DIO, clock div:2
load:0x3fce3818,len:0x16f8
load:0x403c9700,len:0x4
load:0x403c9704,len:0xc00
load:0x403cc700,len:0x2eb0
entry 0x403c9908
ivmarkov commented 3 weeks ago

Please note that my binary is actually the lte_example in my PR. I have pushed some updates there, and did a bit of life-time modification on the EspNetifDriver.

Yes, I commented on it.

Update, ran this without free(buffer) and it at least started.

Let's keep free commented out for now, until we figure out (by examining the C ESP Modem) if we need to call it or whoever is calling us calls it for us once we return control to it.

Now I need to subscribe to network events I think.

Only if you want to watch what is going on. I.e., is the PPP netif getting a DHCP address and so on. Just subscribe on the system event loop with our eventloop module.

One inconvenience is that we don't have the IP_EVENT_PPP_GOT_IP / IP_EVENT_PPP_LOST_IP mapped. Maybe you can map them to Rust structures following the lead of what had been done for the (very similar) IP_EVENT_STA_GOT_IP / IP_EVENT_STA_GOT_IP and IP_EVENT_ETH_GOT_IP and IP_EVENT_ETH_GOT_IP?

ivmarkov commented 3 weeks ago

You can also spawn another thread which is examining the EspNetif::is_connected / ::get_ip_info in a loop + sleeping for a while if you don't want to deal with mapping ESP_EVENT_PPP_* to Rust immediately...

ivmarkov commented 3 weeks ago

Only if you want to watch what is going on. I.e., is the PPP netif getting a DHCP address and so on. Just subscribe on the system event loop with our eventloop module.

Now that I think of it, I'm not sure what protocol is used over PPP so that the ESP will get an IP address (that is, assuming you don't just assign a fixed IP). It can't use DHCP, as DHCP lives on the border between Ethernet and IP, and does assume the medium below IP is Ethernet.

So what is it? Maybe something within PPP itself?

DaneSlattery commented 3 weeks ago

Only if you want to watch what is going on. I.e., is the PPP netif getting a DHCP address and so on. Just subscribe on the system event loop with our eventloop module.

One inconvenience is that we don't have the IP_EVENT_PPP_GOT_IP / IP_EVENT_PPP_LOST_IP mapped. Maybe you can map them to Rust structures following the lead of what had been done for the (very similar) IP_EVENT_STA_GOT_IP / IP_EVENT_STA_GOT_IP and IP_EVENT_ETH_GOT_IP and IP_EVENT_ETH_GOT_IP?

I decided to just follow the c code while I try get this to work:

 esp!(unsafe {
            esp_event_handler_register(
                IP_EVENT,
                ESP_EVENT_ANY_ID as _,
                Some(Self::raw_on_ip_event),
                netif.handle() as *mut core::ffi::c_void,
            )
        })?;

    fn on_ip_event(event_id: u32, event_data: *mut ::core::ffi::c_void) {
        use log::info;
        info!("Got event id: {}", event_id);

        if event_id == ip_event_t_IP_EVENT_PPP_GOT_IP {
            let dns_info = esp_netif_dns_info_t::default();
            let event_data = { (event_data as *const ip_event_got_ip_t) };
            info!("modem connected to ppp server, info: {:?}", event_data);
        } else if event_id == ip_event_t_IP_EVENT_PPP_LOST_IP {
            info!("Modem disconnected from ppp server");
        }
    }

    unsafe extern "C" fn raw_on_ip_event(
        event_handler_arg: *mut ::core::ffi::c_void,
        event_base: esp_event_base_t,
        event_id: i32,
        event_data: *mut ::core::ffi::c_void,
    ) {
        Self::on_ip_event(event_id as _, event_data)
    }

I think it would be good to map the events.

Vollbrecht commented 3 weeks ago

Now that I think of it, I'm not sure what protocol is used over PPP so that the ESP will get an IP address (that is, assuming you don't just assign a fixed IP). It can't use DHCP, as DHCP lives on the border between Ethernet and IP, and does assume the medium below IP is Ethernet.

So what is it? Maybe something within PPP itself?

PPP itself is only Layer2, so only works with MAC addresses. E.g no routing.

PPP brings NCP's( Network control protocols) for every higher level supported protocol to the table. They are encapsulations to configure this higher level protocols. For the mention IP case its the IPCP ( Internet protocol control protocol)

DaneSlattery commented 3 weeks ago

I just got myself an IP! I (62618) esp_idf_svc::modem: ip_info = esp_netif_ip_info_t { ip: esp_ip4_addr { addr: <valid ip addr> }, netmask: esp_ip4_addr { addr: <valid netmask> }, gw: esp_ip4_addr { addr: 1077952522 } }

I need to:

  1. Use subscribe instead of esp_event_handler_register for the EspNetfiDriver to the system event loop.
  2. Wrap the event structures.
  3. Use subscribe instead of esp_event_handler_register for the IP_EVENT_PPP_GOT_IP events .
  4. Think about how to move between data mode and command mode
  5. Create static and non-static constructors.
  6. Think about an async variant :cry:
  7. Run the rx loop in a separate thread.
ivmarkov commented 3 weeks ago

I just got myself an IP! I (62618) esp_idf_svc::modem: ip_info = esp_netif_ip_info_t { ip: esp_ip4_addr { addr: <valid ip addr> }, netmask: esp_ip4_addr { addr: <valid netmask> }, gw: esp_ip4_addr { addr: 1077952522 } }

Were you able - in the meantime - to also open a TCP/UDP socket on top? Does it work?

I need to:

  1. Use subscribe instead of esp_event_handler_register for the EspNetfiDriver to the system event loop.

Not strictly necessary as that's an internal API inside EspNetifDriver. But if you are up for it - do it. In any case, what we need an EspIdfDriver::is_connected and probably some sort of Status, similar to how EspWifi and EspEth operate.

  1. Wrap the event structures.
  2. Use subscribe instead of esp_event_handler_register for the IP_EVENT_PPP_GOT_IP events .

As per above - up to you for the inside of EspNetifDriver.

  1. Think about how to move between data mode and command mode

I think PPP somehow has special frames which designate "end of data mode". Depending on where we go with the EspModem thing (see my latest comments to your PR), it might be as easy as monitoring for these frames (the ESP IDF PPP implementation should have events for that we should subscribe to - maybe lost-ip is good enough?) and then just exiting the run function? This functionality might live inside EspNetifDriver actually (or a short accompanying function/struct, if the functionality turns out to be PPP-specific).

If you decide to go in the direction where EspModem=SimModem is not concerned with the PPP (data) mode, then you don't even need to exit the "SimModem::run" function as that function would only be responsible for setting up the data mode and will exit when the data mode is ready to be established (but won't run the data mode).

  1. Create static and non-static constructors.

Yes.

  1. Think about an async variant 😢

That's actually easier than what you probably think it would take. But let's solve everything else first, then I'll try to provide a code snippet as to how that would work.

  1. Run the rx loop in a separate thread.

Not necessary in my opinion. The current semantics of EspModem::set_data_mode (should really be `EspModem::run) is that is should block until the modem hangs up the data connection (or an error appears before that), and then the user might wan to re-try running "run". It is completely up to the user to spawn a separate thread where "run" should be executed.

DaneSlattery commented 3 weeks ago

Were you able - in the meantime - to also open a TCP/UDP socket on top? Does it work? I haven't yet, I still need to get the run to happen in a separate thread so I can put the rx loop aside. Then I'll try to do an mqtt client or something

Not strictly necessary as that's an internal API inside EspNetifDriver. But if you are up for it - do it. In any case, what we need an EspIdfDriver::is_connected and probably some sort of Status, similar to how EspWifi and EspEth operate. As per above - up to you for the inside of EspNetifDriver.

I did it already. I think if we don't use the event loop, we must use esp_event_handler_register, at least that's what I get reading through the c-code.

In netif.rs, IpEvent listens for ip_event_t_IP_EVENT_PPP_GOT_IP and ip_event_t_IP_EVENT_PPP_LOST_IP events, so I've subscribed to those to tell if the netif is connected. So we don't need on_ip_event in modem.rs. Nothing about SLIP there by the way.

However, we still need to subscribe to on_ppp_event using the event loop to catch ppp specifics.

I think PPP somehow has special frames which designate "end of data mode". Depending on where we go with the EspModem thing (see my latest comments to your PR), it might be as easy as monitoring for these frames (the ESP IDF PPP implementation should have events for that we should subscribe to - maybe lost-ip is good enough?) and then just exiting the run function? This functionality might live inside EspNetifDriver actually (or a short accompanying function/struct, if the functionality turns out to be PPP-specific).

If you decide to go in the direction where EspModem=SimModem is not concerned with the PPP (data) mode, then you don't even need to exit the "SimModem::run" function as that function would only be responsible for setting up the data mode and will exit when the data mode is ready to be established (but won't run the data mode).

I think lost-ip is good enough, or any ppp_error_event

ivmarkov commented 3 weeks ago

Were you able - in the meantime - to also open a TCP/UDP socket on top? Does it work? I haven't yet, I still need to get the run to happen in a separate thread so I can put the rx loop aside. Then I'll try to do an mqtt client or something

Sure, but my point is, don't do this in the modem code, driver etc. Just do it in your example.

Not strictly necessary as that's an internal API inside EspNetifDriver. But if you are up for it - do it. In any case, what we need an EspIdfDriver::is_connected and probably some sort of Status, similar to how EspWifi and EspEth operate. As per above - up to you for the inside of EspNetifDriver.

I did it already. I think if we don't use the event loop, we must use esp_event_handler_register, at least that's what I get reading through the c-code.

esp_event_handler_register and the Rust type-safe wrappers for the event loop are one and the same thing. esp_event_handler_register is just the raw, unsafe API we use in our event wrappers under the hood.

However, we still need to subscribe to on_ppp_event using the event loop to catch ppp specifics.

It would be good if you expose a type-safe wrapper for the PPP events, similar to IpEvent.

I think PPP somehow has special frames which designate "end of data mode". Depending on where we go with the EspModem thing (see my latest comments to your PR), it might be as easy as monitoring for these frames (the ESP IDF PPP implementation should have events for that we should subscribe to - maybe lost-ip is good enough?) and then just exiting the run function? This functionality might live inside EspNetifDriver actually (or a short accompanying function/struct, if the functionality turns out to be PPP-specific). If you decide to go in the direction where EspModem=SimModem is not concerned with the PPP (data) mode, then you don't even need to exit the "SimModem::run" function as that function would only be responsible for setting up the data mode and will exit when the data mode is ready to be established (but won't run the data mode).

I think lost-ip is good enough, or any ppp_error_event

Might be the case yes.

DaneSlattery commented 3 weeks ago
.[0;32mI (561212) esp_idf_svc::modem:  ip_info = esp_netif_ip_info_t { ip: esp_ip4_addr { addr:super secret }, netmask: esp_ip4_addr { addr: 4294967295 }, gw: esp_ip4_addr { addr: super secret} } .[0m
.[0;32mI (561222) esp-netif_lwip-ppp: Connected.[0m
.[0;32mI (561232) esp_idf_svc::modem: modem connected to ppp server, info: ip_event_got_ip_t { esp_netif: 0x3fca67d4, ip_info: esp_netif_ip_info_t { ip: esp_ip4_addr { addr: super secret}, netmask: esp_ip4_addr { addr: 4294967295 }, gw: esp_ip4_addr { addr: super secret} }, ip_changed: true }.[0m
.[0;32mI (561262) esp_idf_svc::modem:  dns_info = 134744072 .[0m
.[0;31mE (561262) esp_netif_handlers: esp_netif_action_connected 47 connected action: dhcpc failed ret=0x102.[0m
.[0;32mI (561272) esp_idf_svc::modem: Got event PPP: ErrorNone.[0m
.[0;32mI (561282) esp_idf_svc::modem: Got event id ppp changed: 0.[0m
.[0;32mI (571362) lte_modem: -> GET http://ifconfig.net/.[0m
.[0;32mI (574932) lte_modem: <- 200.[0m
.[0;32mI (574932) lte_modem: Read 14 bytes.[0m
.[0;32mI (574932) lte_modem: Response body (truncated to 1024 bytes): "super secret\n".[0m

Amazing, it works. It takes a really long time (5 minutes now) to connect to the network, and I don't know if it's using 4G,3G or 2G, but I managed to make a network request with the example in lte_example in the PR.

@diondokter , do you know if there is a way to tell what network a modem is connected to?

diondokter commented 3 weeks ago

@DaneSlattery There might be an AT command to find that out

ivmarkov commented 3 weeks ago

@DaneSlattery I assume the slowdown is before your log snippet, as the action in your log snippet seems to take only 13 seconds or so.

Even if it is 2g, this (5 minutes negotiate time) just does not seem normal (assuming default baud rate of 115200, which you do set, right?)