Find available devices

[roaldarbol]

I often use list_ports from serial.tools to find available devices. However, you always need a few more steps. As it's such a common need I was thinking it would be useful to add into Belay as a helper function. Could be:

from serial.tools import list_ports

def list_devices():
    ports = []
    for port in list(list_ports.comports()):
        ports.append(port.device)
    return(ports)

I'll do a PR, so you can comment on that too. 😅 EDIT: This is my first proper pull request on a collaborative project, so hope I've done it right.

[BrianPugh]

PR Merged! Thank you!

[raveslave]

related: are there any defined standards in the micropython world to identify a repl from any other cdc/acm device?

the device-class misc is used by most random usb gear incl. internal webcams in laptops etc vid & pid will of course be vendor & board specific, i.e. sparkfun, chipmaker-x etc looking at the manufacturer string seems to be most common to me.. still not universal

last resort would be to connect, test talking to it, disconnect, repeat... thoughts?

 bDeviceClass           :    0xef Miscellaneous
 ..
 iManufacturer          :    0x1 MicroPython
 iProduct               :    0x2 Board in FS mode

[BrianPugh]

we could maybe try searching for a "micropython" string in those attributes; I'm totally open to suggestions! Worst case, attempting-to-connect could work, would just be potentially slow and messy.

[raveslave]

will do some thinking. to me libusb is the best way to really find out whats connected regarldess of msd, cdc, etc. problem is that the "dev/tty" path isn't known (afaik) as its created by the OS post enumeration in various ways.

usually you know what board(s) you're using, so I think it would be nice with something like device = belay.Device(vid=0x1b4f, pid=0x0026)

not sure about the most pythonic way to handle if several devices of the same kind (vid, pid) are connected. best practice should be to provide the 'serial_number' argument as the order of com-ports are random depending on connection-order, hubs, etc.

a session-token/finger-print could be used to handle reconnections to the same device (assuming the conected device has a unique pid+vid+product+manufacturer+serial combination)

---

pyserial

ports = serial.tools.list_ports.comports()
for port in ports:
    vars(port)

{'description': 'Board in FS mode',
 'device': '/dev/cu.usbmodem147201',
 'hwid': 'USB VID:PID=1B4F:0026 SER=e46030f803311824 LOCATION=20-7.2',
 'interface': None,
 'location': '20-7.2',
 'manufacturer': 'MicroPython',
 'name': 'cu.usbmodem147201',
 'pid': 38,
 'product': 'Board in FS mode',
 'serial_number': 'e46030f803311824',
 'vid': 6991}

libusb / pyusb

u = usb.core.find(idVendor=0x1b4f, idProduct=0x0026)
usb.util.get_string(u, u.iManufacturer)
'MicroPython'

print(u)
DEVICE ID 1b4f:0026 on Bus 020 Address 010 =================
 bLength                :   0x12 (18 bytes)
 bDescriptorType        :    0x1 Device
 bcdUSB                 :  0x200 USB 2.0
 bDeviceClass           :   0xef Miscellaneous
 bDeviceSubClass        :    0x2
 bDeviceProtocol        :    0x1
 bMaxPacketSize0        :   0x40 (64 bytes)
 idVendor               : 0x1b4f
 idProduct              : 0x0026
 bcdDevice              :  0x100 Device 1.0
 iManufacturer          :    0x1 MicroPython
 iProduct               :    0x2 Board in FS mode
 iSerialNumber          :    0x3 e46030f803311824
 bNumConfigurations     :    0x1

[BrianPugh]

we could make list_devices instead return some sort of descriptor, and we could make a Device classmethod to construct a device from it; I'll also think about it.

[roaldarbol]

Sounds super interesting - I'd be excited to use such a function elsewhere too!

[raveslave]

belay is so awesome, all about convenience, so if we can find a way to make it super-easy to connect and also easily manage several usb devices that would be awe!

I think I would like to have it something like this:

belay.Device() 
without args returns a micropython board if there is one

belay.Device([vid | pid | serial | etc]) 
picks a specific vendor or filter on serial=123xyz for your own custom boards

if there are more than one device found
show a list of available ones
 * interactive terminal (arrow up/down or type -> enter)
 * pick one in the list, then store fingerprint in env-variable 
    ...or write a .mpdevice "json" file to local dir

I noticed many micropy boards out there are missing or re-using same serial number across pcb's

a wild idea would be to offer a belay.provision() feature that writes a unique 255byte identity to the user-flash this way we don't have to worry about serial-no is missing in the usb descriptor and can easily pin a specific board to a session even after rebooting things.

[raveslave]

once we have some utils in place like above, an idea would be to have a basic data-set to define the boards you normally use for a project. I tend to have a few different boards connected at the same time, sometimes two of the same type.

the user could do something like this and store a file in your working-repo, env or similar..

@dataclass
class USBDevice:
    vid: str
    pid: str
    serial: str
    name: str
    # (pseudo code, attributes and types tbd)

    def as_belay(self):
        return belay.Device('tty-goes-here')

# then something like this do provide a list of the boards you use
usbdevices = [
    USBDevice(vid=0x55, pid=0x01),
    USBDevice(vid=0x23, serial='0123456', desc='my own rp2040 pcb'),
    USBDevice(manufacturer='MicroPython', desc='any default stock uf2')
]

this would mainly be to create a basic lookup dataset for known devices/boards and describe them ...needs some more thinking how to make use of it when connecting to an explicit one, basic use-case would be to just have it as a filter option where you could say, give me all boards from this manufacturer or vid etc.

[BrianPugh]

I think I can get behind something like this. My primary concern is if we need to maintain an ever growing list of devices; it'd be better to externalize it. But it might not be too big of a deal.

[raveslave]

agree, my idea was that the list of devices is on the user application side, i.e. not part of this repo rather provide the tools and structure to register some favorite devices and get them auto-connected based on usb properties rather than a dev/tty path

[BrianPugh]

So I think there's certainly some good ideas here, and I think this is totally doable without too much work. However, there are some thoughts/questions/comments in no particular order:

1. Things inside a python script should not be interactive by default.
2. We can use the belay cli for interactive/configuration elements. Maybe a command like belay select --interactive
3. We could store data in the .belay/ folder, but I have some hesitations since the entire folder is currently intended to be under source control. A .belay_user.yaml in the root of the project is an alternative.
4. Writing/reading a specific fingerprint on the board's filesystem is super easy with Belay, but this should really just be a sanity check instead of the defining factor (since a connection to that board would have already been established at that point).
5. Multiple boards sharing a serial number could definitely be problematic; not sure if there's and real way around it besides (4).

So really we just need to agree on what the behavior should be, then I'm totally fine implementing this 😄 .

[raveslave]

1. True! I believe most straight forward would be to return the device if a direct-match or only one, and throw an exception if there are multiple matches (a bit like a non existent dev/tty.* would behave)
2. Great idea. 'mpremote' has a list and auto feature, so def leave interactive prompts to the belay cli
3. To get a first iteration in place, how about we just skip this and let the users manage it in the application. I'd be super-happy to just have a more parametric driven way of connecting to a device.
   • actually, PyFtdi has a pretty decent url-scheme to handle the same thing, see: https://eblot.github.io/pyftdi/urlscheme.html
4. actually something I think should be solved in micropython itself.
   • i.e. machine.unique_id
   • rp2040 example unique_board_id
   • after thinking a bit more... for belay I think a connection/session should use a composite of the parameters already available, including bus:address
5. solved by adding bus:address to the mix (just like PyFtdi)

[BrianPugh]

@raveslave closing discussion here, we can continue ideas/feedback in #122