Support COM port discovery via USB VID/PID

[MCUdude]

Apparently, @mariusgreuel's Avrdude fork for windows supports COM port discovery via USB VID/PID. This is actually very neat if you're using a UART-based programmer that has a tendency to bump the COM port number. I have a similar issue on my mac as well; The serial port (/dev/cu.*) gets a different name depending on which USB port I connect it to.

Would it be possible, within a reasonable amount of time, to make this work on non-windows systems as well?

[mariusgreuel]

The crucial part is figuring out the link between the USB device and the serial device. I never looked into non-Windows implementations, but on Windows there are two parts of metadata one can use: 1) You can enumerate through the device tree, and find the parent of the serial device, which will be the USB device. That is tricky if the USB device is a composite device. 2) Windows actually maintains a bunch of properties for every device, and Portname does what I needed.

[dl8dtl]

For a completely different example, on FreeBSD, they can only be examined by sysctl. Here's the example of the tty port of a curiosity nano board:

# sysctl dev.umodem
dev.umodem.0.ttyports: 1
dev.umodem.0.ttyname: U3
dev.umodem.0.%parent: uhub12
dev.umodem.0.%pnpinfo: vendor=0x03eb product=0x2175 devclass=0xef devsubclass=0x02 devproto=0x01 sernum="MCHP3372031800002359" release=0x0100 mode=host intclass=0x02 intsubclass=0x02 intprotocol=0x01 ttyname=U3 ttyports=1
dev.umodem.0.%location: bus=4 hubaddr=4 port=4 devaddr=5 interface=1 ugen=ugen4.5
dev.umodem.0.%driver: umodem
dev.umodem.0.%desc: Microchip Technology Incorporated nEDBG CMSIS-DAP, class 239/2, rev 2.00/1.00, addr 5
dev.umodem.%parent: 

These attach to the umodem driver (aka. CDC device), and the respective device here is /dev/cuaU3. Alas, one would have to walk across all possible drivers in addition to umodem (FTDI, PL230x, CH3xx).

So question is to find out how to get this information on MacOS.

Downside: it adds a lot of per-OS code to AVRDUDE.

[MCUdude]

Thank you for the details! I'll do a bit of research this evening to see if I can figure it out on MacOS.

Downside: it adds a lot of per-OS code to AVRDUDE.

Good point. But on the other side, this (and probably the "Arduino Leonardo style") is IMO functionality that should be considered even though OSes handles this differently. The end-user won't notice anything, and it's very convenient!

I'll agree that adding functionality to one "OS build" that wouldn't work for the others is a bad idea, but if we can figure out how to do it for Windows, MacOS, FreeBSD and other UNIX compatible OSes, I think we should consider it for the sake of the convenience this functionality adds.

[dl8dtl]

I guess finding a way for Linux will be possible as well. It would make sense to somehow abstract an API for that, and put that into its own implementation file, a bit like that whereami.[ch] stuff that has recently been added to find out about the location of the executable.

[MCUdude]

... a bit like that whereami.[ch] stuff that has recently been added to find out about the location of the executable.

Is whereami only needed for Windows builds?

I agree that it would make sense to "separate" OS-specific code into a separate API in order to not "pollute" existing code. The good this about this is that it makes it easier to add more OS-specific code later on. We should, however, be restrictive and not just throw in all the bells whistles just because we can. At the moment I can't think of other "nice to have" features that rely on OS-specific code other than port discovery via VID/PID and "1200bps touch" used on various Arduino boards.

EDIT: Another wild thought. Instead of always having to "manually" find the VID/PID how about a keyword in avrdude.conf that could hold the VID, PID, and other information about a serial device as well?

serialadapter
  id        = "ch340";
  desc      = "CH340* USB to serial adapter";
  usbvid    = 0x1A86;
  usbpid    = 0x7523;
;

serialadapter
  id        = "cp2102";
  desc      = "CP2102* USB to serial adapter";
  usbvid    = 0x10C4;
  usbpid    = 0xEA60;
;

serialadapter
  id        = "cp2104";
  desc      = "CP2104* USB to serial adapter";
  usbvid    = 0x10C4;
  usbpid    = 0xEA60;
;

$ avrdude -C avrdude.conf -p avr128da48 -c serialupdi -P ch340 -v -t

[MCUdude]

@dl8dtl It looks like sysctl is included in MacOS, but I couldn't figure out how to get it to work. However, Google reveals that there is at least two commands that does what we want. Currently, I have a CH340N USB to serial adapter and a PicKit4 connected to my mac.

First command:

$ ioreg -p IOUSB 
+-o Root  <class IORegistryEntry, id 0x100000100, retain 15>
  +-o AppleUSBXHCI Root Hub Simulation@14000000  <class AppleUSBRootHubDevice, id 0x100000340, registered, matched, ac$
    +-o Bluetooth USB Host Controller@14300000  <class AppleUSBDevice, id 0x10000046b, registered, matched, active, bu$
    +-o USB2.0-Serial@14100000  <class AppleUSBDevice, id 0x10000069e, registered, matched, active, busy 0 (2 ms), ret$
    +-o MPLAB PICkit 4 CMSIS-DAP@14200000  <class AppleUSBDevice, id 0x1000006b0, registered, matched, active, busy 0 $

Second command:

$ system_profiler SPUSBDataType
2022-03-22 20:58:58.311 system_profiler[2351:23194] SPUSBDevice: IOCreatePlugInInterfaceForService failed 0xe00002be
USB:

    USB 3.0 Bus:

      Host Controller Driver: AppleUSBXHCIWPT
      PCI Device ID: 0x9cb1 
      PCI Revision ID: 0x0003 
      PCI Vendor ID: 0x8086 

        Bluetooth USB Host Controller:

          Product ID: 0x8290
          Vendor ID: 0x05ac (Apple Inc.)
          Version: 1.68
          Manufacturer: Broadcom Corp.
          Location ID: 0x14300000

        MPLAB PICkit 4 CMSIS-DAP:

          Product ID: 0x2177
          Vendor ID: 0x03eb  (Atmel Corporation)
          Version: 1.00
          Serial Number: BUR204071896
          Speed: Up to 480 Mb/sec
          Manufacturer: Microchip Technology Incorporated
          Location ID: 0x14200000 / 6
          Current Available (mA): 500
          Current Required (mA): 500
          Extra Operating Current (mA): 0

        USB2.0-Serial:

          Product ID: 0x7523
          Vendor ID: 0x1a86
          Version: 2.63
          Speed: Up to 12 Mb/sec
          Location ID: 0x14100000 / 5
          Current Available (mA): 500
          Current Required (mA): 98
          Extra Operating Current (mA): 0

[dl8dtl]

Now, form that as an algorithm in C ;-)

[MCUdude]

Now, form that as an algorithm in C ;-)

Challenge accepted! Here's a proof of concept. Lots of code borrowed from here. Note that it will only recognize "modems", so from my understanding, serial devices only. I've tried to plug in various other USB equipment, but my USB to serial devices are the only ones that show up.

Output:

$ ./usb_vid_pid_test 
Found modem. Port: /dev/cu.usbserial-1410 USB VID: 0x1A86 PID: 0x7523
Found modem. Port: /dev/cu.SLAB_USBtoUART USB VID: 0x10C4 PID: 0xEA60

Build command:

gcc -framework CoreServices -framework IOKit -o usb_vid_pid_test usb_vid_pid_test.c

usb_vid_pid_test.c

```c #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include // Function prototypes static kern_return_t findModems(io_iterator_t *matchingServices); static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize); // Returns an iterator across all known modems. Caller is responsible for // releasing the iterator when iteration is complete. static kern_return_t findModems(io_iterator_t *matchingServices) { kern_return_t kernResult; CFMutableDictionaryRef classesToMatch; // Serial devices are instances of class IOSerialBSDClient. // Create a matching dictionary to find those instances. classesToMatch = IOServiceMatching(kIOSerialBSDServiceValue); if (classesToMatch == NULL) { printf("IOServiceMatching returned a NULL dictionary.\n"); } else { // Look for devices that claim to be modems. CFDictionarySetValue(classesToMatch, CFSTR(kIOSerialBSDTypeKey), CFSTR(kIOSerialBSDAllTypes)); } // Get an iterator across all matching devices. kernResult = IOServiceGetMatchingServices(kIOMasterPortDefault, classesToMatch, matchingServices); if (KERN_SUCCESS != kernResult) printf("IOServiceGetMatchingServices returned %d\n", kernResult); } return kernResult; } static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize) { io_object_t modemService; kern_return_t kernResult = KERN_FAILURE; bool modemFound = false; int vid; int pid; // Initialize the returned path *bsdPath = '\0'; // Iterate across all modems found. In this example, we bail after finding the first modem. while ((modemService = IOIteratorNext(serialPortIterator))) { // Variable declaration int pid, vid; CFTypeRef bsdPathAsCFString, cf_vendor, cf_product; cf_vendor = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane, CFSTR("idVendor"), kCFAllocatorDefault, kIORegistryIterateRecursively | kIORegistryIterateParents); cf_product = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane, CFSTR("idProduct"), kCFAllocatorDefault, kIORegistryIterateRecursively | kIORegistryIterateParents); bsdPathAsCFString = IORegistryEntryCreateCFProperty(modemService, CFSTR(kIOCalloutDeviceKey), kCFAllocatorDefault, 0); // Decode & print port, VID & PID if (cf_vendor && cf_product && bsdPathAsCFString && CFNumberGetValue(cf_vendor , kCFNumberIntType, &vid) && CFNumberGetValue(cf_product, kCFNumberIntType, &pid) && CFStringGetCString(bsdPathAsCFString, bsdPath, maxPathSize, kCFStringEncodingUTF8)) { printf("Found modem. Port: %s USB VID: 0x%04X PID: 0x%04X\n", bsdPath, vid, pid); modemFound = true; kernResult = KERN_SUCCESS; } // Release CFTypeRef if (cf_vendor) CFRelease(cf_vendor); if (cf_product) CFRelease(cf_product); if (bsdPathAsCFString) CFRelease(bsdPathAsCFString); } return kernResult; } int main(int argc, const char * argv[]) { kern_return_t kernResult; io_iterator_t serialPortIterator; char bsdPath[MAXPATHLEN]; kernResult = findModems(&serialPortIterator); if (KERN_SUCCESS != kernResult) { printf("No modems were found.\n"); } kernResult = getModemPath(serialPortIterator, bsdPath, sizeof(bsdPath)); if (KERN_SUCCESS != kernResult) { printf("Could not get path for modem.\n"); } IOObjectRelease(serialPortIterator); return EX_OK; } ```

[MCUdude]

Here's another example where the USB VID and PID is passed as arguments, and the program will tell if the device is present or not:

Output:

.$ /search_usb_vid_pid 0x1a86 0x7523
USB device found. Port: /dev/cu.usbserial-1420 USB VID: 0x1A86 PID: 0x7523

$ ./search_usb_vid_pid 0x10c4 0xea60
USB device found. Port: /dev/cu.SLAB_USBtoUART USB VID: 0x10C4 PID: 0xEA60

# Looking for an FT231X that's not currenty plugged in:
$ ./search_usb_vid_pid 0x0403 0x6015
USB device not found.

# After connecting the FT231X:
$ ./search_usb_vid_pid 0x0403 0x6015
USB device found. Port: /dev/cu.usbserial-DN02SHCS USB VID: 0x0403 PID: 0x6015

Build command:

gcc -framework CoreServices -framework IOKit -o search_usb_vid_pid search_usb_vid_pid.c 

Source:

```c #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include // Function prototypes static kern_return_t findModems(io_iterator_t *matchingServices); static bool getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize, int usb_vid, int usb_pid); // Returns an iterator across all known modems. Caller is responsible for // releasing the iterator when iteration is complete. static kern_return_t findModems(io_iterator_t *matchingServices) { kern_return_t kernResult; CFMutableDictionaryRef classesToMatch; // Serial devices are instances of class IOSerialBSDClient. // Create a matching dictionary to find those instances. classesToMatch = IOServiceMatching(kIOSerialBSDServiceValue); if (classesToMatch == NULL) { printf("IOServiceMatching returned a NULL dictionary.\n"); } else { // Look for devices that claim to be modems. CFDictionarySetValue(classesToMatch, CFSTR(kIOSerialBSDTypeKey), CFSTR(kIOSerialBSDAllTypes)); } // Get an iterator across all matching devices. kernResult = IOServiceGetMatchingServices(kIOMasterPortDefault, classesToMatch, matchingServices); if (KERN_SUCCESS != kernResult) { printf("IOServiceGetMatchingServices returned %d\n", kernResult); } return kernResult; } static bool getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize, int usb_vid, int usb_pid) { io_object_t modemService; int vid; int pid; // Initialize the returned path *bsdPath = '\0'; // Iterate across all modems found. In this example, we bail after finding the first modem. while ((modemService = IOIteratorNext(serialPortIterator))) { // Variable declaration int pid, vid; CFTypeRef bsdPathAsCFString, cf_vendor, cf_product; cf_vendor = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane, CFSTR("idVendor"), kCFAllocatorDefault, kIORegistryIterateRecursively | kIORegistryIterateParents); cf_product = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane, CFSTR("idProduct"), kCFAllocatorDefault, kIORegistryIterateRecursively | kIORegistryIterateParents); bsdPathAsCFString = IORegistryEntryCreateCFProperty(modemService, CFSTR(kIOCalloutDeviceKey), kCFAllocatorDefault, 0); // Decode & print port, VID & PID if (cf_vendor && cf_product && bsdPathAsCFString && CFNumberGetValue(cf_vendor , kCFNumberIntType, &vid) && CFNumberGetValue(cf_product, kCFNumberIntType, &pid) && CFStringGetCString(bsdPathAsCFString, bsdPath, maxPathSize, kCFStringEncodingUTF8)){ if(usb_vid == vid && usb_pid == pid) { printf("USB device found. Port: %s USB VID: 0x%04X PID: 0x%04X\n", bsdPath, vid, pid); return true; } } // Release CFTypeRef if (cf_vendor) CFRelease(cf_vendor); if (cf_product) CFRelease(cf_product); if (bsdPathAsCFString) CFRelease(bsdPathAsCFString); } return false; } int main(int argc, const char * argv[]) { kern_return_t kernResult; io_iterator_t serialPortIterator; char bsdPath[MAXPATHLEN]; char * end_ptr; int usb_vid; int usb_pid; char* token; usb_vid = strtol(argv[1], &end_ptr, 0); if (*end_ptr || (end_ptr == argv[1])) { printf("Could not parse argument %s\n", argv[1]); return 0; } usb_pid = strtol(argv[2], &end_ptr, 0); if (*end_ptr || (end_ptr == argv[1])) { printf("Could not parse argument %s\n", argv[2]); return 0; } kernResult = findModems(&serialPortIterator); if(KERN_SUCCESS != kernResult || !getModemPath(serialPortIterator, bsdPath, sizeof(bsdPath), usb_vid, usb_pid)) { printf("USB device not found.\n"); } IOObjectRelease(serialPortIterator); return EX_OK; } ```

[dl8dtl]

Cool

I wouldn't consider it for v7.0 though, there are enough other things still on the plate, and I'd rather have a release anytime soon than deferring it further by having to implement and debug a completely new feature.

[MCUdude]

I wouldn't consider it for v7.0 though, there are enough other things still on the plate, and I'd rather have a release anytime soon than deferring it further by having to implement and debug a completely new feature.

Very well, let's continue the discussion after 7.0 is released. Is there anything I or we can do to help out preparing for a release?

[dl8dtl]

Is there anything I or we can do to help out preparing for a release?

Well, you already did a great job by walking through the old issues. As I already wrote you, I'd like to have a look at that WiFi-based programming tool, and try making the timeout tweaks a little less hacky and more generic, but otherwise I'm fine with the current state.

[mcuee]

I remember python-serial has already got this done. https://pyserial.readthedocs.io/en/latest/tools.html

serial.tools.list_ports.comports(include_links=False)
Platform:   Posix (/dev files)
Platform:   Linux (/dev files, sysfs)
Platform:   OSX (iokit)
Platform:   Windows (setupapi, registry)

Example run log under Windows.

(py39x64venv) PS C:\work\python\pyserial\serial\tools> python -m serial.tools.list_ports -v
COM14
    desc: Arduino Leonardo (COM14)
    hwid: USB VID:PID=2341:8036 SER=5&586B51A&0&1 LOCATION=1-1:x.0
1 ports found

Unfortunately it does not work under FreeBSD, just as what the document says.

(mypy38venv) mcuee@freebsdx64vm:~/build $ python -m serial.tools.list_ports -v
/dev/cuaU0          
    desc: n/a
    hwid: n/a
1 ports found

[MCUdude]

Is this something that should be considered for the 7.1 release?

We could perhaps support the following syntaxes:

-P usb:0x1A86: 0x7523 or -P usb:1A86:7523 like @mariusgreuel's fork supports -P ch340, where the serial adapter chip is specified in avrdude.conf. or ft232:[serial_number] if two identical USB to serial adapters are present.

I'm having a hard time figuring out how to deal with config_gram.y, but we would probably need to extract id, desc, usbvid, usbpid, and place them in a struct. Then path and serno can be found using OS specific code. I'm able to extract the path and serial number on macOS using Apple's framework, IOKit, but have not tried on other operating systems

typedef struct serialport_t {
  LISTID     id;
  const char *desc;
  int        usbvid;
  LISTID     usbpid;
  char       path[MAXLEN];
  char       serno[MAXLEN];
} SERIALPORT;

Output from MacOS test program:

$ ./usb_vid_pid_test 
Found modem. Port: /dev/cu.usbserial-1410 USB VID: 0x1A86 PID: 0x7523, S/N: 
Found modem. Port: /dev/cu.usbmodem14201 USB VID: 0x2341 PID: 0x0058, S/N: 426D8DB85151363448202020FF16154A

[mcuee]

typedef struct serialport_t {
  LISTID     id;
  const char *desc;
  int        usbvid;
  LISTID     usbpid;
  char       path[MAXLEN];
  char       serno[MAXLEN];
} SERIALPORT;

Looks like a good proposal. The path will be OS specific.

It is kind of also similar to pyserial output.

(py310venv) C:\work\python>  python -m serial.tools.list_ports -v
COM7
    desc: USB-SERIAL CH340 (COM7)
    hwid: USB VID:PID=1A86:7523 SER= LOCATION=1-2.3.2.2
COM8
    desc: USB Serial Device (COM8)
    hwid: USB VID:PID=2341:0058 SER=0CB8B0715153314639202020FF0C0B23 LOCATION=1-2
2 ports found

[stefanrueger]

I would use const char * over char [MAXLEN]. I assume these strings won't be written to later on. I am using the libavrdude function cache_string(str) for the assignment in config_gram.y, which has the advantage that a string that's used hundreds of times will only be stored once, there is no need to strdup() these strings when doing a pgm_dup() and one does not have to free them on pgm_free(). Right now, the PROGRAMMER structure has the following USB related entries:

  int usbvid;
  LISTID usbpid;
  const char *usbdev;
  const char *usbsn;
  const char *usbvendor;
  const char *usbproduct;

I know little about USB (both hardware and software); is serno something you read from the physical programmer at runtime ot is that known as a thing associated to the type of programmer/manufacturer?

[mcuee]

 int usbvid;
  LISTID usbpid;
  const char *usbdev;
  const char *usbsn;
  const char *usbvendor;
  const char *usbproduct;

Great, just needs to add two entries.

 const char *desc;
 const char *path; (?)

The path may not be const since it may change for some use cases.

USB serial number needs to read from run-time and some device may not have serial number. And by USB spec the device either has no serial number or unique serial number (but some devices may violate this spec).

ABCs of USB: https://www.usbmadesimple.co.uk/

[dl8dtl]

serno is one of the standard USB string descriptors, like the vendor and product name. It is optional (device is not required to implement it).

Edit: mcuee was faster ;-)

[stefanrueger]

Thanks @mcuee and @dl8dtl. Some programmers already read in the serial number at run time. So, serno cannot be put into avrdude.conf.

It's likely that const char * is suitable for path. It's OK to assign different strings to a const char * during runtime, but it's not OK to write to the strings so assigned. If the string needs writing to at runtime (eg, want to replace all / with nul) then either an array is appropriate or one needs to strdup() a copy of the const char * variable first. The size for a path is some 4096 (in Linux) and that's sometimes a waste of space. For example, we used to have a 4096 byte char array for the path of the configuration file for each of the 300+ parts and 100+ programmers, so I changed that to const char *, so the full path of avrdude.conf is only stored once using cache_string(), and this turned out to be useful for many other string-like components for the parts and programmers.

[MCUdude]

It is perhaps a good idea to move the OS specific things into separate .c and .h files. Perhaps io.c/h or iousb.c/h?

For reference, here's an updated version of the MacOS test program that finds the path and serial number for all connected serial devices. It is a result of some copy-pasting, but it works as a proof of concept. Perhaps we can figure out a way for Linux and Windows as well?

MacOS USB VID/PID test program

```c #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include #include // Function prototypes static kern_return_t findModems(io_iterator_t *matchingServices); static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize); CFStringRef find_serial(int idVendor, int idProduct) { CFMutableDictionaryRef matchingDictionary = IOServiceMatching(kIOUSBDeviceClassName); CFNumberRef numberRef; numberRef = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &idVendor); CFDictionaryAddValue(matchingDictionary, CFSTR(kUSBVendorID), numberRef); CFRelease(numberRef); numberRef = 0; numberRef = CFNumberCreate(kCFAllocatorDefault, kCFNumberSInt32Type, &idProduct); CFDictionaryAddValue(matchingDictionary, CFSTR(kUSBProductID), numberRef); CFRelease(numberRef); numberRef = 0; io_iterator_t iter = NULL; if (IOServiceGetMatchingServices(kIOMasterPortDefault, matchingDictionary, &iter) == KERN_SUCCESS) { io_service_t usbDeviceRef; if ((usbDeviceRef = IOIteratorNext(iter))) { CFMutableDictionaryRef dict = NULL; if (IORegistryEntryCreateCFProperties(usbDeviceRef, &dict, kCFAllocatorDefault, kNilOptions) == KERN_SUCCESS) { CFTypeRef obj = CFDictionaryGetValue(dict, CFSTR(kIOHIDSerialNumberKey)); if (!obj) { obj = CFDictionaryGetValue(dict, CFSTR(kUSBSerialNumberString)); } if (obj) { return CFStringCreateCopy(kCFAllocatorDefault, (CFStringRef)obj); } } } } return NULL; } // Returns an iterator across all known modems. Caller is responsible for // releasing the iterator when iteration is complete. static kern_return_t findModems(io_iterator_t *matchingServices) { kern_return_t kernResult; CFMutableDictionaryRef classesToMatch; // Serial devices are instances of class IOSerialBSDClient. // Create a matching dictionary to find those instances. classesToMatch = IOServiceMatching(kIOSerialBSDServiceValue); if (classesToMatch == NULL) { printf("IOServiceMatching returned a NULL dictionary.\n"); } else { // Look for devices that claim to be modems. CFDictionarySetValue(classesToMatch, CFSTR(kIOSerialBSDTypeKey), CFSTR(kIOSerialBSDAllTypes)); } // Get an iterator across all matching devices. kernResult = IOServiceGetMatchingServices(kIOMasterPortDefault, classesToMatch, matchingServices); if (KERN_SUCCESS != kernResult) printf("IOServiceGetMatchingServices returned %d\n", kernResult); return kernResult; } static kern_return_t getModemPath(io_iterator_t serialPortIterator, char *bsdPath, CFIndex maxPathSize) { io_object_t modemService; kern_return_t kernResult = KERN_FAILURE; bool modemFound = false; int vid; int pid; // Initialize the returned path *bsdPath = '\0'; // Iterate across all modems found. In this example, we bail after finding the first modem. while ((modemService = IOIteratorNext(serialPortIterator))) { // Variable declaration int pid, vid; CFTypeRef bsdPathAsCFString, cf_vendor, cf_product; cf_vendor = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane, CFSTR("idVendor"), kCFAllocatorDefault, kIORegistryIterateRecursively | kIORegistryIterateParents); cf_product = IORegistryEntrySearchCFProperty(modemService, kIOServicePlane, CFSTR("idProduct"), kCFAllocatorDefault, kIORegistryIterateRecursively | kIORegistryIterateParents); bsdPathAsCFString = IORegistryEntryCreateCFProperty(modemService, CFSTR(kIOCalloutDeviceKey), kCFAllocatorDefault, 0); // Decode & print port, VID & PID if (cf_vendor && cf_product && bsdPathAsCFString && CFNumberGetValue(cf_vendor , kCFNumberIntType, &vid) && CFNumberGetValue(cf_product, kCFNumberIntType, &pid) && CFStringGetCString(bsdPathAsCFString, bsdPath, maxPathSize, kCFStringEncodingUTF8)) { CFStringRef obj = find_serial(vid, pid); char serial[256] = {0x00}; if (obj) CFStringGetCString(obj, serial, 256, CFStringGetSystemEncoding()); printf("Found modem. Port: %s USB VID: 0x%04X PID: 0x%04X, S/N: %s\n", bsdPath, vid, pid, serial); modemFound = true; kernResult = KERN_SUCCESS; } // Release CFTypeRef if (cf_vendor) CFRelease(cf_vendor); if (cf_product) CFRelease(cf_product); if (bsdPathAsCFString) CFRelease(bsdPathAsCFString); } return kernResult; } int main(int argc, const char * argv[]) { kern_return_t kernResult; io_iterator_t serialPortIterator; char bsdPath[MAXPATHLEN]; kernResult = findModems(&serialPortIterator); if (KERN_SUCCESS != kernResult) { printf("No modems were found.\n"); } kernResult = getModemPath(serialPortIterator, bsdPath, sizeof(bsdPath)); if (KERN_SUCCESS != kernResult) { printf("Could not get path for modem.\n"); } IOObjectRelease(serialPortIterator); return EX_OK; } ```

Output:

``` $ ./usb_vid_pid_test Found modem. Port: /dev/cu.usbserial-1410 USB VID: 0x1A86 PID: 0x7523, S/N: Found modem. Port: /dev/cu.usbmodem14201 USB VID: 0x2341 PID: 0x0058, S/N: 426D8DB85151363448202020FF16154A ```

[mcuee]

It's likely that const char is suitable for path. It's OK to assign different strings to a const char during runtime, but it's not OK to write to the strings so assigned. If the string needs writing to at runtime (eg, want to replace all / with nul) then either an array is appropriate or one needs to strdup() a copy of the const char * variable first.

Thanks. I am more talking about the following situation.

• https://github.com/avrdudes/avrdude/issues/788

You can see that the COM port assignment, USB PID and Path can change during run time (basically two different devices already). Maybe we have to treat them as two devices. In that case, I think const char* is okay.

[mcuee]

@MCUdude

Just wondering if you can give libserialport a try. Sigrok libserialport supports Windows (MSVC and mingw), Linux, macOS and FreeBSD.

• https://github.com/avrdudes/avrdude/issues/788#issuecomment-1196426797

[MCUdude]

@mcuee sorry, I totally forgot to reply!

Just wondering if you can give libserialport a try.

Wow, libserialport is a really nice tool! I think libserialport can make serial port discovery much easier! The provided examples are also very straightforward and easy to follow. Just what we need.

Here's the output from the list_ports example:

$ ./list_ports
Getting port list.
Found port: /dev/cu.Bluetooth-Incoming-Port
Found port: /dev/cu.usbserial-1410
Found 2 ports.
Freeing port list.

And here is the output from the port_info example:

$ ./port_info /dev/cu.usbserial-1410 
Looking for port /dev/cu.usbserial-1410.
Port name: /dev/cu.usbserial-1410
Description: USB2.0-Serial
Type: USB
Manufacturer: (null)
Product: USB2.0-Serial
Serial: (null)
VID: 1A86 PID: 7523
Bus: 0 Address: 0
Freeing port.

As you can see, libserialport gives us the /dev path and the USB VID/PID. Exactly what we need!

[mcuee]

@MCUdude and @stefanrueger

As @dl8dtl mentioned, this may add too many platform specific codes to avrdude, do we really want to persue this or we can leave it outside of avrdude?

[stefanrueger]

Seems like there is almost a solution that can be abstracted away. We need a champion, though. Sounds like this might be @MCUdude? If not I'd be fine with dropping this.

[MCUdude]

We use Avrdude for batch programming at work. I decided to stick with the USBasp programmer (USBISP hardware running modified USBasp firmware), since the COM port number on the Windows computer we use tends to change occasionally, even though there's only one USB to serial adapter connected.

If we instead could specify the USB vid/pid, or even better, the chip name itself, it would be much easier to deal with USB to serial devices when using a script to execute an Avrdude command. (-p ch340, -p 1A86:7523 -pch340:serno or -p 1A86:7523:serno)

Libserialport seems like the perfect match. It supports all major operating systems and has simple examples that do what we want.

I can create a test program that takes a chip name or a USB VID/PID and outputs the serial port. However, I'm not all that good at integrating with avrdude.conf and how integrating the libserialport source code. But I think this would be a very useful addition to Avrdude!

[mcuee]

But I think this would be a very useful addition to Avrdude!

In this case, I will keep this issue open.

@dl8dtl and @mariusgreuel

Any objection to add a new dependancy like libserialport?

[MCUdude]

Any objection to add a new dependancy like libserialport?

Since the libserialport project isn't really that large, we could bundle the entire source code, and have everything stored in a dedicated folder under src/. Another alternative is to make libserialport an optional dependency if users want serial port discovery provided by this library.

I've created a proof of concept program that either takes an id or is:serno, and outputs relevant port information if present.

It would be interesting to hear what @stefanrueger thinks of this, and how this functionality could be "properly" added to avrdude, including adding common USB to serial chips to avrdude.conf.

$ # In this example I've connected one FT232RL and one CH340N to my mac

$ ./port_finder ft232rl
Found port:
id: ft232rl
desc:   FT232R USB UART
port:   /dev/cu.usbserial-AH00M3HQ
serno:  AH00M3HQ

$ ./port_finder ft232rl:invalid_sn

$ ./port_finder ft232rl:AH00M3HQ
Found port:
id: ft232rl
desc:   FT232R USB UART
port:   /dev/cu.usbserial-AH00M3HQ
serno:  AH00M3HQ

$ ./port_finder ch340
Found port:
id: ch340
desc:   USB Serial
port:   /dev/cu.usbserial-1420
serno:  

port_finder.c source code + makefile (should work on any OS that libserialport supports)

```c #include #include #include #define PORTS 3 typedef struct { int vid; int pid; char id[128]; char desc[128]; char port[128]; char serno[128]; } serial_port; serial_port ser[PORTS] = { { .vid = 0x1a86, .pid = 0x7523, .id = "ch340", .desc = "", .port = "", .serno = "" }, { .vid = 0x2341, .pid = 0x0043, .id = "uno", .desc = "", .port = "", .serno = "" }, { .vid = 0x0403, .pid = 0x6001, .id = "ft232rl", .desc = "", .port = "", .serno = "" }, }; int main(int argc, char **argv) { if (argc < 2) { printf("Missing programmer ID, e.g ch340\n"); return -1; } /* A pointer to a null-terminated array of pointers to * struct sp_port, which will contain the ports found.*/ struct sp_port **port_list; /* Call sp_list_ports() to get the ports. The port_list * pointer will be updated to refer to the array created. */ enum sp_return result = sp_list_ports(&port_list); if (result != SP_OK) { printf("sp_list_ports() failed!\n"); return -1; } for (int i = 0; port_list[i]; i++) { struct sp_port *port = port_list[i]; int usb_vid, usb_pid; sp_get_port_usb_vid_pid(port, &usb_vid, &usb_pid); for(int j = 0; j < PORTS; j++) { // Get user specified serial number char* sn = NULL; if(!!strstr(argv[1], ":")) { sn = strchr(argv[1], ':'); sn++; // Remove leading ':' } // argv[1] starts with a known id int id_match = strncmp(argv[1], ser[j].id, strlen(ser[j].id)) == 0; // It's a match! if (usb_vid == ser[j].vid && usb_pid == ser[j].pid && id_match) { strcpy(ser[j].desc, sp_get_port_description(port)); strcpy(ser[j].port, sp_get_port_name(port)); if (sp_get_port_usb_serial(port) != NULL) strcpy(ser[j].serno, sp_get_port_usb_serial(port)); // Continue if user specified serial number doesn't match if (sn && strcmp(ser[j].serno, sn) != 0) continue; printf("Found port:\n"); printf("id:\t%s\n", ser[j].id); printf("desc:\t%s\n", ser[j].desc); printf("port:\t%s\n", ser[j].port); printf("serno:\t%s\n", ser[j].serno); return 0; } } } sp_free_port_list(port_list); // Free the array created by sp_list_ports() /* Note that this will also free all the sp_port structures * it points to. If you want to keep one of them (e.g. to * use that port in the rest of your program), take a copy * of it first using sp_copy_port(). */ return -1; } ``` Makefile: ``` make # A simple Makefile to build the examples in this directory. # # This example file is released to the public domain. CC = gcc PKG_CONFIG = pkg-config CFLAGS = -g -Wall $(shell $(PKG_CONFIG) --cflags libserialport) LIBS = $(shell $(PKG_CONFIG) --libs libserialport) SOURCES = $(wildcard *.c) BINARIES = $(SOURCES:.c=) %: %.c $(CC) $(CFLAGS) $< $(LIBS) -o $@ all: $(BINARIES) clean: rm $(BINARIES) ```

[mcuee]

http://sigrok.org/wiki/Libserialport

Offiically supported operating systems

1. Linux
2. Mac OS X
3. FreeBSD
4. Windows
5. Android

I believe OpenBSD and NetBSD will also work.

OpenBSD port (seems to be at the latets version and no patches required). http://ports.su/comms/sigrok/libserialport

[mcuee]

@MCUdude Any progress in this one?

@stefanrueger Any comments on the proposal from @MCUdude?

[MCUdude]

I haven't continued working on this because I haven't heard what @dl8dtl and @stefanrueger think of this, and it would be a waste of time to submit a PR that's never going to make it into the main branch. If they think this will "bloat" the codebase, and not be a useful feature, we can close this issue.

I think I'm capable of doing most of the work, but I'd need help with the following

• avrdude.conf grammar. There should be a few pre-defined serial chips that users can use without modifying avrdude.conf, avrdude -c urclock -P ft232rl -p atmega328pb
• libserialport integration. Perhaps the simplest would be to copy the entire library into the avrdude source code, and place it in a subfolder, libserialport. The other alternative is by dynamically linking it at compile time, where the user needs to install this before building Avrdude. I'm not sure what goes into adding a library to Avrdude's make/cmake system.

[mcuee]

For the second question, the usual solution is to add libserialport as a git submodule. But I am not familiar with git and CMake. Maybe @mariusgreuel can advice.

[stefanrueger]

Making it easier to specify the serial port for a board with an usb to serial adapter seems like a cool idea, provided it's coded in a way that the functionality id #ifdef'd and therefore can be dropped for an OS that does not have the needed library.

Have we thought about scenarios whether things can go wring? What happens, say, if someone has their 3d-printer plugged into the host, plugs a test board with a different usb-serial-adapter into the host as well. Can they seamlessly upload blink.hex onto the 3d-printer by mistakenly choosing the wrong serialadapter?

What is the problem that we are trying to solve? That /dev/ttyUSBx numbers can get reallocated each time the board is plugged in? To discriminate between two of the boards of the same type based on the serial number of the serial-USB-adapter, so I can always address a board with the same port string no matter how the OS has decided to name it?

avrdude.conf grammar

Could be done in a similar way as programmers are. That is a bit involved if one wants the developer options to print the structures neatly (as in -c/s or -p/s). Maybe a simpler idea is to allow stub programmers (ie, those with prog_modes being zeo and without having a type) and treat these as serial adapters; lexer.l could simply map two synonymous strings "programmer" and "serialadapter" to K_PROGRAMMER and developer_opts.c could print serialadapter instead of programmer if the programmer is a stub. This means -P can take a programmer/serialadapter name as argument. The main grammar change would be in lexer.c to replace the programmer line with

(programmer|serialadapter) { yylval=NULL; ccap(); current_strct = COMP_PROGRAMMER; return K_PROGRAMMER; }

[MCUdude]

Have we thought about scenarios whether things can go wring? What happens, say, if someone has their 3d-printer plugged into the host, plugs a test board with a different usb-serial-adapter into the host as well. Can they seamlessly upload blink.hex onto the 3d-printer by mistakenly choosing the wrong serialadapter?

I think it would be best to require a serial number if two chips with the same VID/PID are detected -P ft232rl:sernum.

What is the problem that we are trying to solve? That /dev/ttyUSBx numbers can get reallocated each time the board is plugged in? To discriminate between two of the boards of the same type based on the serial number of the serial-USB-adapter, so I can always address a board with the same port string no matter how the OS has decided to name it?

Correct. Having an alternative to the ever-changing COM port number/path would be a great enhancement IMO. Especially when you can enter a serial number and guarantee that it will only accept that exact chip, for instance, a 3D printer. And if Avrdude.conf/avrduderc would support optional serial numbers (and perhaps a default baud rate if not specified?), one could do -P my_3d_printer.

serialadapter
  id            = "my_3d_printer";
  desc          = "CP2104* USB to serial adapter";
  usbvid        = 0x10C4;
  usbpid        = 0xEA60;
  default_baud  = 250000; # Optional
  serial_number = 1234567890; # Optional
;

$ avrdude -curclock -patmega2560 -Pmy_3d_printer -Uflash:w:marlin.hex:i

[mcuee]

@dl8dtl

Any comment on the ideas by Hans?

[MCUdude]

I've managed to add libserialport to CMake, so now it should include libserialport if installed, and HAVE_LIBSERIALPORT should be defined. I tried adding it to the autoconf/make system as well, but I couldn't figure out how. @dl8dtl might have a clue.

Anyways, here's the development branch: https://github.com/MCUdude/avrdude/tree/ser-auto

I'll see if I can wrap my head around the grammar part (I wish I had learned about this in school rather than the Microsoft ecosystem)

[dl8dtl]

I guess it's the ser-auto branch, isn't it? I might look into the auto* tools stuff.

[dl8dtl]

Regarding the grammar stuff, I don't know how massive the changes might be, for simple keywords, you can basically copy and paste some existing code portion. After all, the YACC file structure ist just a BNF, where certain actions are inserted in {} brackets.

[MCUdude]

I guess it's the ser-auto branch, isn't it?

Yes, that's correct. I accidentally posted the URL to the wrong branch. I've updated the previous post.

Could be done in a similar way as programmers are. That is a bit involved if one wants the developer options to print the structures neatly (as in -c/s or -p/s). Maybe a simpler idea is to allow stub programmers (ie, those with prog_modes being zeo and without having a type) and treat these as serial adapters; lexer.l could simply map two synonymous strings "programmer" and "serialadapter" to K_PROGRAMMER and developer_opts.c could print serialadapter instead of programmer if the programmer is a stub. This means -P can take a programmer/serialadapter name as argument. The main grammar change would be in lexer.c to replace the programmer line with

(programmer|serialadapter) { yylval=NULL; ccap(); current_strct = COMP_PROGRAMMER; return K_PROGRAMMER; }

I tried to do this, but I'm just getting a parsing error:

avrdude error: programmer type serialadapter not found [/Users/hans/Downloads/avrdude/build_darwin/src/avrdude.conf:2531]
avrdude error: unable to process system wide configuration file /Users/hans/Downloads/avrdude/build_darwin/src/avrdude.conf

It looks like I need to define a programmer type, but this isn't really a programmer at all, it's just a way to get a COM port number/dev path from a serial adapter name

serialadapter
  id            = "ch340";
  desc          = "CH340 USB to serial adapter";
  type          = "serialadapter";
  usbvid        = 0x1a86;
  usbpid        = 0x7523;
  ;default_baud  = 250000; # Optional
  ;serial_number = 1234567890; # Optional
;

[dl8dtl]

I guess it's the ser-auto branch, isn't it? I might look into the auto* tools stuff.

Here's the suggested patch for the auto* tools. autotools.patch.txt

[dl8dtl]

I'm not so sure about the grammar changes. Parsing -P ft232rl:sernum does not involve grammar in the first place. But I guess what you actually want here is to allow for ft232rl to be declared in the config file. As I see it, that is going to be just an entry of its own, without any relationship to a programmer, isn't it? (Maybe I'm misunderstanding something here.) If so, establishing a separate grammatical entity for serialadapter is the way to go. It can probably be structured in a similar way to programmers (because for them, VID/PID is already handled), but I would decouple that from programmers, as a separate object kind.

[MCUdude]

Here's the suggested patch for the auto* tools.

Thanks! That worked, and I've updated the dev branch

[dl8dtl]

I could try drafting some code changes for the config grammar.

[MCUdude]

But I guess what you actually want here is to allow for ft232rl to be declared in the config file

Correct. Users may add their personal Arduino UNO, and perhaps even add a serial number to the config file (or avrduderc) so the can do -P my_unique_uno, and it will only allow that particular USB to serial chip, without having to type the serial number.

As I see it, that is going to be just an entry of its own, without any relationship to a programmer, isn't it?

Yes, that's my understanding as well. It's just a way to bring additional information to libserialport about the serial port/USB chip the user wants to use, and use it (libserialport) to figure out the actual COM port number/dev path.

I could try drafting some code changes for the config grammar.

That would be excellent, thank you!

[MCUdude]

As a recap, I think something like this would be very neat for unique "boards" that require a set of parameters to work:

serialadapter
  id            = "my_3d_printer";
  desc          = "CP2104* USB to serial adapter";
  usbvid        = 0x10C4;
  usbpid        = 0xEA60;
  default_baud  = 250000; # Optional
  serial_number = 1234567890; # Optional
;

[dl8dtl]

First draft. Obviously, you have to design the entire backend around "serialadapter", I just made assumptions an object class like that one does exist. I suggest reusing "serial" instead of "serial_number", to avoid yet another keyword. The "desc" handling has been turned into a kind of generic token these days. I guess Stefan might be better about filling in that part of the game. ;-) grammar.patch.txt

[dl8dtl]

PROGRAMMER * oops, that needs to become SERIALADAPTER * … Already a bit late at night. Please, eyeball-review everything, and see whether you can make any sense out of all that. If not, ask me.

[MCUdude]

Thank you, Jörg! I applied the patch and fixed the error you pointed out. Even though it doesn't build at the moment (due to the missing serialadapter backend, I still pushed the code if someone (@stefanrueger for instance?) wants to have a look at the current work.

There are still a lot of things that need to be figured out. The entire serialadapter backend needs to be written, and ideally, be left out if Avrdude was built without libserialport. And when the backend "infrastructure" is up and running, it has to be "filled" with the information from avrdude.conf.

But at least now we have a foundation; the libserialport library gets linked in and the grammar is present.

[dl8dtl]

Somewhere in the diff, I saw another "PROGRAMMER" that should have been a "SERIALADAPTER". Filling in the information is already there, through serialadapter_new(), and the grammar parser subsequently filling in the individual fields. Just turn the related functions into dummies complaining if HAVE_LIBSERIALPORT is not defined. I hope @stefanrueger can fill in the missing "desc" part through TKN_COMPONENT.

[MCUdude]

You mean this (K_PROGRAMMER -> K_SERIALADAPTER)?

+serialadapter_decl :
+  K_PROGRAMMER
+    { current_serialadapter = serialadapter_new();
+      current_serialadapter->config_file = cache_string(cfg_infile);
+      current_serialadapter->lineno = cfg_lineno;
+    }

+  |
+  /* needs to be handled through TKN_COMPONENT */
+  /* K_DESC TKN_EQUAL TKN_STRING
+  | */
+  K_DEFAULT_BAUD TKN_EQUAL numexpr {
+    {
+      current_serialadapter->default_baud = $3->value.number;
+      free_token($3);
+    }
+  }
+  |
+  K_SERIAL TKN_EQUAL TKN_STRING {
+    {
+      current_serialadapter->defaultbaud = cache_string($3->value.string);
+      free_token($3);
+    }
+  }
+;

I'll assume current_serialadapter->defaultbaud are supposed to be current_serialadapter->serial instead?

Another thing. wouldn't it be better to use the existing usbsn to hold the serial number for the USB to serial adapter? serial are for numerical expressions, while usbsn are supposed to hold strings. However, when looking closer, there are no traces of usbsn in the config_gram.y file.