Netdev

[scottfeldman]

(I'm closing the original PR https://github.com/tinygo-org/drivers/pull/523 and submitting this one as a replacement).

This PR adds a network device driver model called netdev. There is a companion PR for TinyGo to update the "net" package here https://github.com/tinygo-org/tinygo/pull/3614. This PR is for the netdev interface definitions, netdev drivers (wifinina, rtl8720dn, espat) and network examples.

See netdev documentation here: README-net.md.

Testing results are here.

[scottfeldman]

Build fails because this PR is dependent on https://github.com/tinygo-org/tinygo/pull/3614.

[scottfeldman]

Build fails because this PR is dependent on tinygo-org/tinygo#3614.

[deadprogram]

I just tested this branch with the code from https://github.com/tinygo-org/tinygo/pull/3704 on my pyportal

These examples all worked perfectly, I did not have a chance to test any others yet.

http-get

$ tinygo flash -target pyportal -ldflags="-X main.ssid=AAAA -X main.pass=BBBB" -monitor ./examples/net/http-get/                                                        
Connected to /dev/ttyACM0. Press Ctrl-C to exit.                                                                                                                             

Tinygo ESP32 Wifi network device driver (WiFiNINA)                                                                                                                           

Driver version           : 0.0.1                                                                                                                                             
ESP32 firmware version   : 1.7.4                                                                                                                                             
MAC address              : aa:bb:cc:dd:ee:ff                                                                                                                                 

Connecting to Wifi SSID 'AAAA'...CONNECTED                                                                                                                                   

DHCP-assigned IP         : 192.168.1.223                                                                                                                                     
DHCP-assigned subnet     : 255.255.255.0                                                                                                                                     
DHCP-assigned gateway    : 192.168.1.1                                                                                                                                       

Getting https://httpbin.org/get?name=John+Doe&occupation=gardener                                                                                                            

HTTP/1.1 200 OK                                                                                                                                                              
Server: gunicorn/19.9.0                                                                                                                                                      
Access-Control-Allow-Origin: *                                                                                                                                               
Access-Control-Allow-Credentials: true                                                                                                                                       
Date: Thu, 04 May 2023 20:31:59 GMT                                                                                                                                          
Content-Type: application/json                                                                                                                                               
Content-Length: 332                                                                                                                                                          
Connection: keep-alive                                                                                                                                                       

{                                                                                                                                                                            
  "args": {                                                                                                                                                                  
    "name": "John Doe",                                                                                                                                                      
    "occupation": "gardener"                                                                                                                                                 
  },                                                                                                                                                                         
  "headers": {                                                                                                                                                               
    "Host": "httpbin.org",                                                                                                                                                   
    "User-Agent": "Go-http-client/1.1",

ntpclient

$ tinygo flash -target pyportal -ldflags="-X main.ssid=AAAA -X main.pass=BBBB" -monitor ./examples/net/ntpclient/                                                       
Connected to /dev/ttyACM0. Press Ctrl-C to exit.                                                                                                                             

Tinygo ESP32 Wifi network device driver (WiFiNINA)                                                                                                                           

Driver version           : 0.0.1                                                                                                                                             
ESP32 firmware version   : 1.7.4                                                                                                                                             
MAC address              : aa:bb:cc:dd:ee:ff                                                                                                                                   

Connecting to Wifi SSID 'AAAA'...CONNECTED                                                                                                                                   

DHCP-assigned IP         : 192.168.1.223                                                                                                                                     
DHCP-assigned subnet     : 255.255.255.0                                                                                                                                     
DHCP-assigned gateway    : 192.168.1.1                                                                                                                                       

Requesting NTP time...                                                                                                                                                       
NTP time: 2023-05-04 20:30:16 +0000 UTC                                                                                                                                      

Disconnected from Wifi SSID 'AAAA'

mqttclient

$ tinygo flash -target pyportal -ldflags="-X main.ssid=AAAA -X main.pass=BBBB" -monitor ./examples/net/mqttclient/                                                      
Connected to /dev/ttyACM0. Press Ctrl-C to exit.                                                                                                                             

Tinygo ESP32 Wifi network device driver (WiFiNINA)                                                                                                                           

Driver version           : 0.0.1                                                                                                                                             
ESP32 firmware version   : 1.7.4                                                                                                                                             
MAC address              : aa:bb:cc:dd:ee:ff                                                                                                                                 

Connecting to Wifi SSID 'AAAA'...FAILED (timed out)                                                                                                                          
Connecting to Wifi SSID 'AAAA'...CONNECTED                                                                                                                                   

DHCP-assigned IP         : 192.168.1.223                                                                                                                                     
DHCP-assigned subnet     : 255.255.255.0                                                                                                                                     
DHCP-assigned gateway    : 192.168.1.1                                                                                                                                       

ClientId: tinygo-client-RPUHOVIRML
Connecting to MQTT broker at tcp://test.mosquitto.org:1883
Connected
Subscribed to topic cpu/freq
Message 120.00Mhz received on topic cpu/freq
Message 120.00Mhz received on topic cpu/freq
Message 120.00Mhz received on topic cpu/freq
Message 120.00Mhz received on topic cpu/freq
Message 120.00Mhz received on topic cpu/freq

webserver

$ tinygo flash -target pyportal -ldflags="-X main.ssid=AAAA -X main.pass=BBBB" -stack-size=4KB -monitor ./examples/net/webserver/

Connected to /dev/ttyACM0. Press Ctrl-C to exit.

Tinygo ESP32 Wifi network device driver (WiFiNINA)

Driver version           : 0.0.1
ESP32 firmware version   : 1.7.4
MAC address              : aa:bb:cc:dd:ee:ff

Connecting to Wifi SSID 'AAAA'...FAILED (timed out)
Connecting to Wifi SSID 'AAAA'...CONNECTED

DHCP-assigned IP         : 192.168.1.223
DHCP-assigned subnet     : 255.255.255.0
DHCP-assigned gateway    : 192.168.1.1

[deadprogram]

cc @sago35 @bgould @conejoninja @soypat

[deadprogram]

This branch here https://github.com/tinygo-org/drivers/tree/scottfeldman-netdev3 resolvs the merge conflicts in your PR @scottfeldman against the latest dev branch.

I would like to see this PR merged along with https://github.com/tinygo-org/tinygo/pull/3704 for the next TinyGo release.

[soypat]

@scottfeldman Heads up: new version v0.5.1 of natiu-mqtt is out with huge bugfixes, new features and minor API changes.

[scottfeldman]

@soypat Thanks for the heads-up. I pulled and tested v0.5.1 natiu-mqqt against the examples/net/mqttclient/natiu. No issues, looks good.

[soypat]

Scott, could we hold off on adding the netlink changes until we'e used the interface with the cyw43439? It would also help lessen the cognitive load of this PR, which has become quite hard to parse mentally.

Im actually not sure if the netlink interface is final too...

[scottfeldman]

The only changes in netlink in this PR is the addition of two methods to the interface for Send/Recv Eth pkts. And I have high confidence that those are the right additions. I'd like to see this PR reviewed and merged to drivers:dev sooner than later. It's been here since Feb 15th. Holding it off until cyw43439 is done is the tail wagging the dog.

[scottfeldman]

I'll back out the netlink changes and push to this PR, tomorrow.

[scottfeldman]

This tinygo-drivers PR must be coordinated with tinygo and tinygo-net PRs:

tinygo-org/tinygo:dev   <-   tinygo-org/tinygo:net-submodule-netdev3
tinygo-org/net:dev      <-   scottfeldman/tinygo-net:netdev3           (PR#2)
tinygo-org/drivers:dev  <-   scottfeldman/tinygo-drivers:netdev3       (this PR)

[deadprogram]

@scottfeldman can you please test this branch against the latest updates in the https://github.com/tinygo-org/tinygo/pull/3704 to make sure things are still working?

[deadprogram]

tinygo-org/net:dev <- scottfeldman/tinygo-net:netdev3 (PR#2)

This is now done. :sparkle:

[scottfeldman]

@deadprogram, thanks for house keeping.

Ok, everything looks good except for one minor issue. I tested this PR against the latest from:

tinygo-org/tinygo:net-submodule-netdev3
tinygo-org/net:dev

example/net/* tests working on wifinina & rtl8720dn devices.

The minor issue is net:dev is missing go.mod, go.sum files. You'll hit this when running make on tinygo:net-submodule-netdev3. (After submodule --init).

[deadprogram]

@scottfeldman the net repo does not need its own go.mod file, as it is only meant to be used as a submodule from the main tinygo repo. So I am not exactly sure what problem you are referring to?

[scottfeldman]

@deadprogram oh, ok. I'm using go workspaces, so maybe I need to remove it from my go.work file? In any case, no problem leaving go.mod out of the net repo.

[deadprogram]

@scottfeldman please see #613