search

rust-lang / socket2

Advanced configuration options for sockets.
https://docs.rs/socket2
Apache License 2.0
689 stars 230 forks source link

Support ESP-IDF Rust Target (for ESP32-C3 especially) #379

Closed stevefan1999-personal closed 1 year ago

stevefan1999-personal commented 1 year ago

Right now it fails to build:

   Compiling socket2 v0.4.7
error[E0432]: unresolved import `libc::SOCK_RAW`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:64:16
   |
64 | pub(crate) use libc::SOCK_RAW;
   |                ^^^^^^^^^^^^^^ no `SOCK_RAW` in the root

error[E0432]: unresolved import `libc::SOCK_SEQPACKET`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:66:16
   |
66 | pub(crate) use libc::SOCK_SEQPACKET;
   |                ^^^^^^^^^^^^^^^^^^^^ no `SOCK_SEQPACKET` in the root

error[E0432]: unresolved import `libc::IP_HDRINCL`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:79:16
   |
79 | pub(crate) use libc::IP_HDRINCL;
   |                ^^^^^^^^^^^^^^^^ no `IP_HDRINCL` in the root

error[E0432]: unresolved import `libc::IP_RECVTOS`
  --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:89:16
   |
89 | pub(crate) use libc::IP_RECVTOS;
   |                ^^^^^^^^^^^^^^^^ no `IP_RECVTOS` in the root

error[E0432]: unresolved imports `libc::ip_mreq_source`, `libc::IP_ADD_SOURCE_MEMBERSHIP`, `libc::IP_DROP_SOURCE_MEMBERSHIP`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:117:5
    |
117 |     ip_mreq_source as IpMreqSource, IP_ADD_SOURCE_MEMBERSHIP, IP_DROP_SOURCE_MEMBERSHIP,
    |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^  ^^^^^^^^^^^^^^^^^^^^^^^^^ no `IP_DROP_SOURCE_MEMBERSHIP` in the root
    |     |                               |
    |     |                               no `IP_ADD_SOURCE_MEMBERSHIP` in the root
    |     no `ip_mreq_source` in the root
    |
help: a similar name exists in the module
    |
117 |     ip_mreq_source as IpMreqSource, IP_ADD_MEMBERSHIP, IP_DROP_SOURCE_MEMBERSHIP,
    |                                     ~~~~~~~~~~~~~~~~~
help: a similar name exists in the module
    |
117 |     ip_mreq_source as IpMreqSource, IP_ADD_SOURCE_MEMBERSHIP, IP_DROP_MEMBERSHIP,
    |                                                               ~~~~~~~~~~~~~~~~~~

error[E0433]: failed to resolve: use of undeclared type `IovLen`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:790:38
    |
790 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                      ^^^^^^ use of undeclared type `IovLen`

error[E0433]: failed to resolve: use of undeclared type `IovLen`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:850:38
    |
850 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                      ^^^^^^ use of undeclared type `IovLen`

error[E0531]: cannot find unit struct, unit variant or constant `SOCK_RAW` in crate `libc`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:362:11
    |
362 |     libc::SOCK_RAW,
    |           ^^^^^^^^ not found in `libc`

error[E0531]: cannot find unit struct, unit variant or constant `SOCK_RDM` in crate `libc`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:364:11
    |
364 |     libc::SOCK_RDM,
    |           ^^^^^^^^ not found in `libc`

error[E0531]: cannot find unit struct, unit variant or constant `SOCK_SEQPACKET` in crate `libc`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:365:11
    |
365 |     libc::SOCK_SEQPACKET,
    |           ^^^^^^^^^^^^^^ not found in `libc`

error[E0412]: cannot find type `IovLen` in this scope
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:790:63
    |
790 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                                               ^^^^^^ not found in this scope

error[E0412]: cannot find type `IovLen` in this scope
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:850:63
    |
850 |     msg.msg_iovlen = min(bufs.len(), IovLen::MAX as usize) as IovLen;
    |                                                               ^^^^^^ not found in this scope

error[E0412]: cannot find type `ip_mreqn` in crate `libc`
    --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:1035:12
     |
1035 |   ) -> libc::ip_mreqn {
     |              ^^^^^^^^ help: a struct with a similar name exists: `ip_mreq`
     |
    ::: C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\unix\newlib\mod.rs:51:1
     |
51   | / s! {
52   | |     // The order of the `ai_addr` field in this struct is crucial
53   | |     // for converting between the Rust and C types.
54   | |     pub struct addrinfo {
...    |
228  | |     }
229  | | }
     | |_- similarly named struct `ip_mreq` defined here

error[E0422]: cannot find struct, variant or union type `ip_mreqn` in crate `libc`
    --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:1037:75
     |
1037 |           crate::socket::InterfaceIndexOrAddress::Index(interface) => libc::ip_mreqn {
     |                                                                             ^^^^^^^^ help: a struct with a similar name exists: `ip_mreq`
     |
    ::: C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\unix\newlib\mod.rs:51:1
     |
51   | / s! {
52   | |     // The order of the `ai_addr` field in this struct is crucial
53   | |     // for converting between the Rust and C types.
54   | |     pub struct addrinfo {
...    |
228  | |     }
229  | | }
     | |_- similarly named struct `ip_mreq` defined here

error[E0422]: cannot find struct, variant or union type `ip_mreqn` in crate `libc`
    --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sys\unix.rs:1042:77
     |
1042 |           crate::socket::InterfaceIndexOrAddress::Address(interface) => libc::ip_mreqn {
     |                                                                               ^^^^^^^^ help: a struct with a similar name exists: `ip_mreq`
     |
    ::: C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\libc-0.2.135\src\unix\newlib\mod.rs:51:1
     |
51   | / s! {
52   | |     // The order of the `ai_addr` field in this struct is crucial
53   | |     // for converting between the Rust and C types.
54   | |     pub struct addrinfo {
...    |
228  | |     }
229  | | }
     | |_- similarly named struct `ip_mreq` defined here

error[E0063]: missing field `sin_len` in initializer of `sockaddr_in`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sockaddr.rs:225:27
    |
225 |         let sockaddr_in = sockaddr_in {
    |                           ^^^^^^^^^^^ missing `sin_len`

error[E0063]: missing field `sin6_len` in initializer of `sockaddr_in6`
   --> C:\Users\steve\.cargo\registry\src\github.com-1ecc6299db9ec823\socket2-0.4.7\src\sockaddr.rs:260:28
    |
260 |         let sockaddr_in6 = sockaddr_in6 {
    |                            ^^^^^^^^^^^^ missing `sin6_len`

Some errors have detailed explanations: E0063, E0412, E0422, E0432, E0433, E0531.
For more information about an error, try `rustc --explain E0063`.
error: could not compile `socket2` due to 17 previous errors
Thomasdezeeuw commented 1 year ago

Does ESP32 have an OS we can target?

stevefan1999-personal commented 1 year ago

@Thomasdezeeuw Uh, yes and no. ESP32 is a family of single board computers which is of course bare metal and no_std, but the ESP-IDF development kit, does feature FreeRTOS on with couple more such as a Virtual filesystem component - ESP32 - — ESP-IDF Programming Guide latest documentation (espressif.com), and some people hacked together to introduce standard library support around the ESP-IDF ecosystem, see STD support for the ESP-IDF framework by ivmarkov · Pull Request #87666 · rust-lang/rust (github.com). Notice ESP-IDF is best coupled with ESP32-C3 right now, which is easily obtainable from every corner of the world at any part of the world for around $7.

While the arcanish ESP-IDF target does mean that some level of OS support (in the sense of a RTOS, however), meaning things like std::io, std::fs does work to some degree, but not everything is coming out of the box as a wholesale OS, as for example, I don't think ESP-IDF enabled IPv6 with LwIP out of the box, nor does std::os::env works. Therefore, the situation it's quite complicated to say the least.

To me specifically I'm experimenting smol-rs/smol: A small and fast async runtime for Rust (github.com) on ESP32-C3 and I was successful to some degree, yet here this specific socket2 library is blocking the compilation of async-io, rendering some of the libraries such as async-net which has a dependency of async-io and thus transitive dependency on socket2, so smol is impossible to use as a whole right now.

Thomasdezeeuw commented 1 year ago

@Thomasdezeeuw Uh, yes and no. ESP32 is a family of single board computers which is of course bare metal and no_std, but the ESP-IDF development kit, does feature FreeRTOS on with couple more such as a Virtual filesystem component - ESP32 - — ESP-IDF Programming Guide latest documentation (espressif.com), and some people hacked together to introduce standard library support around the ESP-IDF ecosystem, see STD support for the ESP-IDF framework by ivmarkov · Pull Request #87666 · rust-lang/rust (github.com). Notice ESP-IDF is best coupled with ESP32-C3 right now, which is easily obtainable from every corner of the world at any part of the world for around $7.

Socket2 is wrapper around the OS's network stack, i.e. it just wraps socket(2) and related system calls. It doesn't implemented a networking stack itself.

While the arcanish ESP-IDF target does mean that some level of OS support (in the sense of a RTOS, however), meaning things like std::io, std::fs does work to some degree, but not everything is coming out of the box as a wholesale OS, as for example, I don't think ESP-IDF enabled IPv6 with LwIP out of the box, nor does std::os::env works. Therefore, the situation it's quite complicated to say the least.

We don't need too much from std lib, some types from std::net, but those are OS independent.

To me specifically I'm experimenting smol-rs/smol: A small and fast async runtime for Rust (github.com) on ESP32-C3 and I was successful to some degree, yet here this specific socket2 library is blocking the compilation of async-io, rendering some of the libraries such as async-net which has a dependency of async-io and thus transitive dependency on socket2, so smol is impossible to use as a whole right now.

I'm not too familiar with smol or async-net/io my self so I can't say much about it. But perhaps it would be easier to simply not compile the networking side on ESP32 targets? Unless you actually need networking.

If you do need networking then we need good libc support and wrappers for various system calls (usually provided by libc). I would start by removing compilation of everything that isn't available at the moment and start slowing adding things that are available.

Thomasdezeeuw commented 1 year ago

Since this target doesn't (seem to) have an OS we won't support it in Socket2.

nevi-me commented 1 year ago

This seems relevant: https://github.com/rust-lang/socket2/commit/afeb7231b461652c49c97cac9996547feef66c23. It's a patch to make socket2 work on espidf. I came across it while also struggling with the error, and looking around to see if there's any solution.

Apologies for tagging you, @ivmarkov is this something upstreamable? Or is it better to keep it in the esp-rs-compat org?

For more context, I'm having trouble compiling https://github.com/project-chip/matter-rs for an esp32-c3 device.

Thomasdezeeuw commented 1 year ago

This seems relevant: afeb723. It's a patch to make socket2 work on espidf. I came across it while also struggling with the error, and looking around to see if there's any solution.

Apologies for tagging you, @ivmarkov is this something upstreamable? Or is it better to keep it in the esp-rs-compat org?

From my point of view it's Ok to upstream, ideally with a CI (at least with cargo check), but without it it's also fine. The downside of having no CI though is that any new release of Socket2 might break compilation on the target.

For more context, I'm having trouble compiling https://github.com/project-chip/matter-rs for an esp32-c3 device.

ivmarkov commented 1 year ago

I was always planning to upstream that. The one thing holding me off from doing it during the last couple of months was that in the meantime, the mainline of socket2 got updated to V0.5, while I still need the 0.4.X branch patched as well, because async-io still uses V0.4.X.

@Thomasdezeeuw - would you accept two PRs then? One for the v0.4.x branch (the one that I would use), and then another - for the master branch? You know - so that we are prepared once async-io upgrades its socket2 dependency?

As for running cargo check with target_os = "espidf" - I would be more than happy to touch socket2's CI for that (the riscv32imc-esp-espidf is an upstreamed Tier 3 target in Rust STD since > 1 year now so that would work BUT would require Cargo nightly and its -Zbuild-std) - yet - I can live without cargo check either. I mean, Rust STD is not even built for riscv32imc-esp-espidf as it is in Tier 3. Neither is libc, for that target.

Thomasdezeeuw commented 1 year ago

I was always planning to upstream that. The one thing holding me off from doing it during the last couple of months was that in the meantime, the mainline of socket2 got updated to V0.5, while I still need the 0.4.X branch patched as well, because async-io still uses V0.4.X.

@Thomasdezeeuw - would you accept two PRs then? One for the v0.4.x branch (the one that I would use), and then another - for the master branch? You know - so that we are prepared once async-io upgrades its socket2 dependency?

Yes, but please send it to target v0.5.x first and then we can backport that. Btw I'm going to try and release v0.5 this week(end), so at that point async-io can also upgrade (if it wants to).

As for running cargo check with target_os = "espidf" - I would be more than happy to touch socket2's CI for that (the riscv32imc-esp-espidf is an upstreamed Tier 3 target in Rust STD since > 1 year now so that would work BUT would require Cargo nightly and its -Zbuild-std) - yet - I can live without cargo check either. I mean, Rust STD is not even built for riscv32imc-esp-espidf as it is in Tier 3. Neither is libc, for that target.

Running with nightly is fine. You probably want something we use to have for Redox: https://github.com/rust-lang/socket2/commit/14be1bc7082b33592ffd62de175e795a02588193 (the part that's deleted in that commit).

jasta commented 1 year ago

Hi @Thomasdezeeuw, me again :) I've got a patch ready for the main branch and also the backported patch again v0.4.9 (which is needed because tokio latest appears to still target socket2 0.4.9). I'll post a PR soon once I confirm the master one actually works.