Serial Port support for Raspberry Pi with Windows IoT

[shaggygi]

This issue was created to continue the discussion on further support of SerialPort related APIs for other platforms/devices. You can see related issues below:

Serial Port support on Windows Serial Port support on Unix/OSX

I actually don't know what the ultimate expectation is for this topic as I hope others would chime in with better scenarios. This will hopefully help contributors understand requirements when improving the APIs to support said features.

For example, what does "support on RPi" really mean? If it runs under Windows IoT, does that mean support for UWP? If support comes for Unix/OSx, does that mean the APIs will work running on Alpine/Ubuntu? And how about Docker support?

There were also discussions on using the on-board UART or using USB (e.g. FTDI, etc.) dongles.

Community, please add thoughts to get SerialPort APIs working in other areas in addition to Windows.

Thanks in advance.

FYI... @willdean @JeremyKuhne @karelz @danmosemsft

[maitredede]

Hi, I think that the SerialPort on Raspberry Pi is :

• Use of the on-board UART : usable as any serial port, if not used by default serial console
• Use of USB dongle serial ports : should be usable when linux support is added, I don't think there will be specific work for RPi here.

I don't know if Docker on Raspberry Pi is a common use case, but I think that it may just work when linux is supported.

[karelz]

@Petermarcu what are supported OS versions of Linux and Windows on Raspberry Pi?

[Petermarcu]

I would expect Ubuntu mate and Raspbian to be the main ones. The first already works and the second will soon. Win10iot works as well.

[cStorm]

To what extent is Win IoT supposed to work?

I am running a 2.0.0-preview2 build from a few days ago on v10.0.15026, but SerialPort.GetPortNames() returns no results when a USB serial dongle is connected. (It is a Z-Wave controller that works well for the same build on a Windows 10 desktop.)

I do see that in WinIoT the device does not appear in HKLM\HARDWARE\DEVICEMAP\SERIALCOMM (as it does in on regular Windows), but I do not know whether that is expected behavior in WinIoT or whether I'm completely missing something about the implementation.

If it matters, I can also say that the same device is discoverable on WinIoT using the Windows.Devices namespace from UWP.

[danmoseley]

@cStorm as you probably saw, GetPortNames() relies on that regkey but it notes that QueryDosDevice is an alternative. I also see GetDefaultCommConfig might be an alternative also but this suggests that it just reads the key.

If you're interested, you could try changing to QueryDosDevice ripping off the code from the teststo see whether they're picked up. Perhaps the "right" way is to combine the answers from both.

I don't know why IoT doesn't have the registry key.

[cStorm]

So I got a proof of concept working, and the short of it is that QueryDosDevice does pick up the ports (with the naming scheme seen in UWP). After relaxing SerialStream's requirement for a "COM*" port name, it seemed to communicate normally as well.

I'd like to share more of what I've found and eventually make a pull request, but before really getting into it, I'd like to confirm whether here or a new issue would be a better for this.

[shaggygi]

@cStorm What platform does your code work on? I'd say just create a pull request and add link to this issue. There is probably more work to do and this issue could be a method to group the progress on the different flavors (win iot, linux, etc). Thx for your effort in helping this feature.

[cStorm]

@shaggygi The change is for Win IoT running on a Raspberry Pi, and I'm using a Windows 10 desktop for building / basic regression testing.

On the topic of cross-platform support, there were actually a few things I wanted an opinion on. First is concerning the best value to return from SerialPort.GetPortNames(). To illustrate the situation, these are the relevant results my desktop returns from the DosDevices code vs the current registry implementation:

QDD common name	QDD internal name	current equivalent
COM1	\Device\Serial0	COM1
ACPI#PNP0501#1#{86e0d1e0-8089-11d0-9ce4-08003e301f73}	\Device\00000031	COM1
COM4	\Device\USBSER000	COM4
USB#VID_0658&PID_0200#6&13add0ab&0&4#{86e0d1e0-8089-11d0-9ce4-08003e301f73}	\Device\USBPDO-10	COM4

This is what Win IoT returns with the same USB device attached:

QDD common name	QDD internal name	current equivalent
ACPI#BCM2837#4#{86e0d1e0-8089-11d0-9ce4-08003e301f73}	\Device\0000001e
USB#VID_0658&PID_0200#5&3753427a&0&5#{86e0d1e0-8089-11d0-9ce4-08003e301f73}	\Device\00000ede

As you can see, on desktop both the current-style and UWP-style names can be determined from QueryDosDevice, but on Win IoT the COM names simply don't seem to function (even with an educated guess at what they are). Would it be better to always return the same type of name for homogeneity, or return the COM names when available for potential backward compatibility reasons? (Alternately, we could return both when available, even though they seem to function identically.)

Second, I did find other places in the registry (on both desktop and Win IoT) that might lead to an alternative implementation. While QueryDosDevice seems to execute more quickly, what I read in pinvoke-checker.md suggests that it is best to avoid adding new native calls. I don't know how much more cross-platform-friendly registry calls would be, but I would appreciate a rundown on what the best practice is here.

[danmoseley]

@cStorm @shaggygi I changed the title to be specific to Windows IoT since as far as I know the general issue to get SerialPort on Unix would enable Raspberry Pi also.

Here's some information I've gotten from the dev in Windows who owns this:

QueryDosDevice does not read the key,it asks the kernel object manager directly. SERIALCOMM key isn’t there on IOT because we don’t automatically assign COM names on IOT (b/c we don’t allow installer code to run on device setup on IOT). No COM name, no SERIALCOMM entry. This article shows devs how to fix it up after the fact.

We probably want to continue using SERIALCOMM because it seems preferred, it's proven, and it also may be easier for us on UWP. Would it make sense to fall back to try QueryDosDevice if SERIALCOMM gives us nothing? Or fall back only on IoT? I don't have an IoT device handy so it's very helpful that you are logging the results you get.

[cStorm]

After some changes, I currently have it only calling QueryDosDevice if the SERIALCOMM key does not exist, and this works for me as it did before.

Only doing this on Win IoT seems like it might cheat other, unforeseen scenarios out of the fix, and I don't know how common it would be for that key to be absent on non-IoT machines anyway (if we're worried about wasting cycles). That said, I don't object to limiting the scope of the change if that's preferred and someone knows the right way to detect the Windows IoT platform.

Also, if there is anything someone wants checked or verified on Windows IoT (for this or another issue), I'm happy to do that and report back.

[willdean]

@cStorm Personally I wouldn't worry about "cheating other scenarios out of the fix" - I think that's less important than not breaking existing stuff.

Re what kind of name needs to be returned, it should definitely be a string which can then be passed, unmodified, to the SerialPort constructor - any other requirement is secondary to that. If there are multiple possibilities for strings which meet that requirement (like there would be on full Windows), then we should choose the shortest string which is most like "COMx".

You can't return an artificial "COMx" name on IoT if that then doesn't work to open the port (it sounds like it wouldn't).

[willdean]

@danmosemsft Said:

@cStorm @shaggygi I changed the title to be specific to Windows IoT since as far as I know the general issue to get SerialPort on Unix would enable Raspberry Pi also.

I don't think this is true, as there's little (other than physical hardware) which relates Windows IoT on Raspberry Pi to the conventional Linux-running Raspberry Pi.

We clearly (as the OP in this thread mused) need two Raspberry Pi threads, one for Linux and one for Win IoT. This one has now become the latter, so that's fine.

[willdean]

@cStorm

Also, if there is anything someone wants checked or verified on Windows IoT (for this or another issue), I'm happy to do that and report back.

How much of the test suite have you been able to run?

[cStorm]

@willdean Right! In my current implementation, if SERIALCOMM exists, all results are from there and in the COMx format. If not, we presume there is no other recourse and fall back to returning the longer names from QueryDosDevice. Either of these name formats can be passed to SerialPort on Windows desktop, but only the latter seems to work on Win IoT.

You have a good point about targeting the change to Win IoT to reduce the potential for unexpected errors on other platforms, but it's worth considering that the platform detection code itself would increase that potential. Is there an established way to do this that would be considered safer that the QueryDosDevice call?

As for the test suite, I've only run it on desktop so far (having updated SerialPort constructor's argument-checking test) and have just manually tested specific changes on Win IoT. I was under the impression that the build wouldn't run on WinIoT/ARM, and haven't gotten the chance to explore other ways of running the tests yet. If there is any information on this I'd be eager to have it.

As an aside, does anyone know if there is a name for that longer, specifically-formatted style of identifier (e.g. USB#VID_0658&PID_0200#5&3753427a&0&5#{86e0d1e0-8089-11d0-9ce4-08003e301f73}) that UWP and IoT use?

[willdean]

@cStorm I think this is considered the definitive reference on all the ways of enumerating ports.

I don't know the proper name of the huge horrible string - it's appears to be a superset of what Device Manager calls the 'Device Instance Path', which might be a good enough name for your purposes.

The thing to be wary of with the test suite is that it adapts how much of it runs depending on what hardware it finds - if you run it with a single port with nothing connected, then only a small amount of stuff runs - you get much more if you arrange the port to loop-back the data, and only get the whole lot if you have two ports connected with a null-modem cable.

The test is also very picky about specific UART behaviour, so only fully passes with a 16x50 UART at at least one end of the null-modem - I'm not sure what the UART details on the Raspberry Pi are, but come back if you have problems because we may need to make the tests smarter to get through it.

[los93sol]

Should this be working in Iot Core? I'm trying with an ASP.NET Core application using the daily builds and targetting win10-arm fails to build so I have to target win-arm instead. The issue I have then is that when deployed I get "System.IO.Ports is currently only supported on Windows.". Reading this thread I'm under the impression this is expected to work with this scenario.

[willdean]

@los93sol I'm not sure you should read this thread as being any claim it works at the moment. I think @cStorm has done some work locally on enumerating ports on Win10-arm-iot or whatever it's called officially, but I don't think that's ever been committed back.

[danmoseley]

@weshaggard I'm curious why this message appears on IoT. It comes from <GeneratePlatformNotSupportedAssemblyMessage Condition="'$(TargetGroup)' == 'netstandard' AND '$(TargetsWindows)' != 'true'">SR.PlatformNotSupported_IOPorts</GeneratePlatformNotSupportedAssemblyMessage> ... but TargetsWindows seems like it should be true?

[los93sol]

After looking at the source it appears as thought the issue I was experiencing is actually by design and is explicitly blocked in this case. I was calling SerialPort.GetPortNames() where the exception was being returned.

The following is where it appears to be trapping.... if (PlatformDetection.IsWinRT) { return new string[0]; // we are waiting for a Win32 new QueryDosDevice API since the current doesn't work for Uap https://github.com/dotnet/corefx/issues/21156 }

[los93sol]

Thinking about it though, that still doesn't explain why I had to target win-arm instead of win10-arm, I would expect iot core to be a win10-arm target, but I supposed that is probably the RID for full windows 10 on arm?

[danmoseley]

@ericstj can you answer my question about $(TargetsWindows) and @los93sol 's question about the RID of Windows IoT? Where are we at with that target platform?

[danmoseley]

The following is where it appears to be trapping....

@los93sol that should only happen if you're running in app container (eg., a UWP style app). The message you reported "System.IO.Ports is currently only supported on Windows." is not from there.

[Petermarcu]

@los93sol , we simplified in .NET Core 2.0. win-arm should be all you need to use.

[los93sol]

@danmosemsft You are correct, I was seeing that snippet in the test and mixed it up. I have ended up doing similar to @cStorm and modified it to use QueryDosDevice to list the ports and that is working as expected. I then modified SerialStream to eliminate the check that the portName starts with "COM" as was suggested earlier in this thread. I'm not able to instantiate the SerialPort object but when I go to open it things get sketchy and I'm troubleshooting blindly. The remote debugger totally crashes and open powershell connections to the device die as well. @cStorm Can you give any clue if there were any other modifications necessary to get it working?

[ericstj]

I'm curious why this message appears on IoT. It comes from SR.PlatformNotSupported_IOPorts ... but TargetsWindows seems like it should be true?

TargetsWindows is a build time condition based on the project configuration. That just means that any non-windows binary (including AnyOS binary) will get the not-supported-assembly. You'd see this if your app if you weren't restoring the correct asset from the package. That'd probably mean that the RID used for restore was wrong, or you had an out of date RID mapping from Microsoft.NETCore.Platforms. If you shared your project.assets.json/project.lock.json I could look at it to tell you why.

[los93sol]

@ericstj Thank you, the file is too large for me to post here, what's the best way to get it to you?

[ericstj]

Just put it in a gist.

[los93sol]

@ericstj Eric, I have posted it Here

[los93sol]

Looking at it myself, I'm pretty sure you are correct, Microsoft.Netcore.Platforms was something like 1.0.1. Can I just update it through nuget or is there some preferred way we are supposed to manage that?

[ericstj]

Here are your three targets:

    ".NETCoreApp,Version=v2.0": {
...

        "runtime": {
          "lib/netstandard2.0/System.IO.Ports.dll": {}
        },
        "runtimeTargets": {
          "runtimes/win/lib/netstandard2.0/System.IO.Ports.dll": {
            "assetType": "runtime",
            "rid": "win"
          }
        }
...
    ".NETCoreApp,Version=v2.0/win-arm": {
...
        "runtime": {
          "lib/netstandard2.0/System.IO.Ports.dll": {}
        }
...
    ".NETCoreApp,Version=v2.0/win10": {
...
        "runtime": {
          "runtimes/win/lib/netstandard2.0/System.IO.Ports.dll": {}
        }

This tells me that NuGet doesn't think win-arm is compatible with win.

Why? Because the RID graph is old. win-arm was added in https://github.com/dotnet/corefx/commit/179f1f25cccae75989a2b95c85e1ef6ce5cb6165, on May 23. "Microsoft.NETCore.Platforms/2.1.0-preview2-25319-04" referenced by this project is from May 19.

Can you change your reference to System.IO.Ports 4.5.0-preview1-25430-01? Even better, just choose the same build that matches the NETCore.App you are using.

PS: The confusing 4.5.0-preview2 versions ought to be deleted. Those are from a couple builds that happened after we branched for 2.0 but forgot to roll the pre-release back to preview1. I filed https://github.com/dotnet/corefx/issues/21781 to track that.

[los93sol]

@ericstj Thanks for creating that and getting some action on it so quickly, you guys are awesome!

[los93sol]

Interesting little development, I have two devices, one is a Serial->USB device with an FTDI chipset that I can communicate with using a custom built version of System.IO.Ports that just relaxes the COM requirement in SerialStream. The other is a UART->USB device with a Silicon Labs chipset (it's the adafruit one). Using that device as soon as SerialPort.Open is called the Pi reboots. If anyone has a suggestion on how to obtain some logs so I can work through it that would be extremely helpful. It is probably worth noting that using a UWP app both devices work as expected so I'm thinking there's something else in System.IO.Ports that's the culprit here, I just have no way of stepping it to know exactly where it's dying.

[los93sol]

Dug a little deeper and while I still don't know a good way to step it, I was able to comment enough code out and keep running to isolate the crash. It looks like in SerialStream.EventLoopRunner.WaitForCommEvent the line causing issues appears to be...

if (Interop.Kernel32.WaitCommEvent(handle, eventsOccurredPtr, intOverlapped) == false)

Figured I'd post it now in case anyone has any info about why that might causing the Pi to reboot with the Adafruit UART->USB adapter

[los93sol]

I'm stumped as it looks like it's crashing when pinvoking Interop.Kernel32.WaitCommEvent. I really don't understand why it would crash with one device and work fine with the other. I do have a kernel dump from the device, but not sure how to use windbg to figure anything out from it.

[willdean]

@los93sol The stack looks like this when you're using USB->Serial drivers:

1 .NET SerialPort - a fairly thin wrapper on Win32 the serial port API (MS / OSS) 2 Win32 serial port API - an very thin wrapper on the kernel mode driver (MS) 3 KM driver - an extremely complex kernel mode driver written by the USB chip vendor (chip vendor)

Your problem is almost certainly in '3' (hence one device and not the other) - it's unlikely you're going to get much help with it on this GH repo.

If you have a BSOD crash dump then you can open it in WinDbg and type something like (it tells you anyway) !analyze and it will give you a bunch of sometimes-useful info, but most likely you'll just see a stack trace pointing the finger at the SiLabs driver.

[los93sol]

@willdean Thank you for that information, that's pretty much what I was expecting, but being new to working at this level I wasn't sure. I was able to get the crash dump, but WinDbg failed to load symbols so I wasn't able to get anything useful from it yet. I'll do some more digging to try to figure that part out. In the meantime, the only thing that leads me to believe it is not manufacturer driver related is the fact that using the UWP UARTSerialSample that leverages the SerialDevice API works fine.

I was looking to see if the UWP API's were open sourced to see what'd different between how UWP monitors for com events and how .NET core API's are doing it, but I couldn't find it anywhere on GitHub.

[willdean]

@los93sol Some symbol server info which might be useful is here: https://msdn.microsoft.com/en-us/library/windows/desktop/ms681416(v=vs.85).aspx

It's not always easy to repro these sort of problems (and also partly why it's hard to write good USB->Serial drivers) because the Win32 serial API is deceptively complex, and different applications use the port in lots of different ways - there are a bazillion IOCTLs and some fairly complex timers, and all sorts of different ways to use the port, particularly with respect to waiting for things. It's possible/likely that the UWP sample uses the port differently to .NET SerialPort - neither is necessarily wrong, it's just that one works better with that specific driver than the other.

[los93sol]

@danmosemsft @ericstj Can either of you tell me how UWP's SerialDevice API monitors for activity? I know SerialStream pinvokes WaitCommEvent but using this particular Adafruit adapter it causes the device to reboot so curious to see what SerialDevice is doing differently since I'm not having the issue on that API surface.

[danmoseley]

@los93sol unfortunately we have no knowledge of how the stack works below the .NET level.

[shaggygi]

@los93sol @ptorr-msft might be able to help, but not sure.

[danmoseley]

Note - https://github.com/dotnet/corefx/issues/21156 covers making S.IO.Ports work in UWP/AppContainer, which has its own issues.

[danmoseley]

Microsoft's USB serial port driver is named usbser.sys . That should work. If you see something else, it's a 3rd party: you'd have to contact the vendor. They should have the ability to figure out the problem, but they may not support IoT use unfortunately.

[willdean]

usbser.sys is a communication device class (CDC) driver - to the best of my knowledge none of the mainstream USB-Serial products use CDC (they predate CDC and usbser.sys has, historically, been execrable anyway. It was apparently rewritten for Win10, so it might be better now, but I doubt it's used in this case).

A read of the release notes of the SiLabs driver should bring forth a chilly pallor and a few beads of sweat to the forehead: http://www.silabs.com/documents/public/release-notes/v6-7-4-driver-release-notes.txt

[los93sol]

@danmosemsft I think it is worth noting that there is a disconnect here as the device I am using is on the supported list for Windows iot so i would expect the driver would also be supported. I plan on digging into this again in the near future. I was able to get it working, but my solution was to comment most of the code involved to avoid the crashes.

[RoySalisbury]

I think its great that people are working on this issue, even if it is just testing at the moment. However, it would be helpful for thoes having the same issue and wanting to possible help if you were to branch that code and at least commit what you have done/tried so far.

I need the SerialPort class as well because the UWP SerialDevice is even in worse shape.

Thanks!

[biradarsm]

I have connected a Sample Weighing Machine to Raspberry Pi 3 using Serial (RS 232 ) to USB Converter Cable on the USB Port of Raspberry Pi 3, The Serial Port device is not getting Detected in UWP Sample application.

[los93sol]

I think the issue here is that the status of this is not really clear. If someone can point me to how to properly detect IOT core I am willing to do some work get things rolling a bit

[los93sol]

Bueller? Anyone still interested in this?

[JeremyKuhne]

@danmosemsft, @pjanotti can you help unblock @los93sol?

[los93sol]

If it were possible to use the UWP API from .net core then I would be good to go, but as it stands today that is not possible...this isn’t a total blocker for me as I hacked out the checks and forced a custom version of System.IO.Ports but I would like to do this properly and will do the PR for it if someone can point me to how to detect iot core