serialport / node-serialport

Access serial ports with JavaScript. Linux, OSX and Windows. Welcome your robotic JavaScript overlords. Better yet, program them!
https://serialport.io
MIT License
5.81k stars 1.01k forks source link

Virtual Serial Ports #999

Closed reconbot closed 6 years ago

reconbot commented 7 years ago

We need Serial Ports on our CI otherwise they're not doing all they could for us.

Travis ci supports socat on linux and osx I can install it and setup a "virtual Arduino" if someone can can figure a command to set up a pair of virtual ports.

Windows is a bit harder, there are a bunch of emulators but none of them look scriptable.

thom-nic commented 7 years ago

I've figured out how to use socat for testing on Linux & OSX. It works on CircleCI if youapt-get socat in a prebuild step.

The raw command looks like this:

socat -d -d pty,raw,nonblock,echo=0,link=ttyV0 pty,raw,nonblock,echo=0,link=ttyV1

Here's a snippet from a mocha test where I've got it all wrapped together:

const fs = require('fs');
const os = require('os');
const resolve = require('path').resolve;
const child_process = require('child_process');
const SerialPort = require('serialport');

describe('SerialPort', function() {
  let client;
  let hwWrite, hwRead;
  let socat;
  const MASTER_PORT = resolve('./ttyV0');
  const SLAVE_PORT  = resolve('./ttyV1');

  const SOCAT_EXISTS = child_process.spawnSync('socat', ['-h']).status === 0;

  if ( ! SOCAT_EXISTS ) {
    console.warn("`socat` is not installed, skipping serial client tests...");
    const installCmd = os.type() === 'Darwin' ? 'brew install socat' : 'sudo apt-get install socat';
    console.warn(`Please run \`${installCmd}\` to enable these tests!`);
    return;
  }

  beforeEach(done => {
    socat = child_process.spawn('socat', ['-d','-d',
      `pty,raw,nonblock,echo=0,link=${MASTER_PORT}`,
      `pty,raw,nonblock,echo=0,link=${SLAVE_PORT}`],
      {detached: true, stdio: 'ignore'});
    socat.on('close', code => {
      //console.log('socat exited with code', code);
    }).on('error', done);

    // socat needs time to init the fds.  Wait for the expected ports to appear...
    const waitForPorts = () => {
      try {
        if ( fs.statSync(MASTER_PORT).dev && fs.statSync(SLAVE_PORT).dev ) {
          hwWrite = fs.createWriteStream(MASTER_PORT, {autoClose: false});
          hwRead = fs.createReadStream(MASTER_PORT);
          hwRead.on('error', () => {}); // ignore read errors on linux when client side stream is closed.
          client = new SerialPort(SLAVE_PORT, {autoOpen: false});
          done();
        }
      }
      catch( ex ) { // ENOENT if both fds have not been created yet.
        setTimeout(waitForPorts, 100);
      }
    };
    waitForPorts();
  });

  afterEach(done => {
    const cb = err => {
      if ( hwWrite ) hwWrite.close();
      socat.kill('SIGTERM');
      done(err);
    };

    if ( client && client.isOpen() ) client.close(cb);
    else cb();
  });

  // tests go here.  `client` is an initialized serial client and `hwRead` and `hwWrite` are 
  // streams you can use for the 'other' side of the serial client.
});

This manages to cleanup the virtual TTY links that socat creates after each test. If things aren't created and closed in the right order you end up with spurious stream errors but this has been working well for me. Can't help with windows though :/

reconbot commented 7 years ago

I cannot wait to give this a try. I'm in negotiations with a windows serialport product, we'll see!

reconbot commented 7 years ago

I finally got this going on travis. It looks like socat doesn't support everything we test with a physical port (custom baudrate test, and flow control flags so far) but it's much better than nothing!

https://travis-ci.org/EmergingTechnologyAdvisors/node-serialport/jobs/183820609

thom-nic commented 7 years ago

ah yeah it's not perfect but probably will get you at least some of the tests you weren't able to do on CI previously? Guessing you were using an env flag or something to skip some tests on CI?

reconbot commented 7 years ago

Yep

On Wed, Dec 14, 2016, 8:46 AM Thom Nichols notifications@github.com wrote:

ah yeah it's not perfect but probably will get you at least some of the tests you weren't able to do on CI previously? Guessing you were using an env flag or something to skip some tests on CI?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EmergingTechnologyAdvisors/node-serialport/issues/999#issuecomment-267037523, or mute the thread https://github.com/notifications/unsubscribe-auth/AABlbtYRxdZ4wkHYzJGOtUmDVwMkiDWIks5rH_NRgaJpZM4KqJ5M .

reconbot commented 7 years ago

Yep on all accounts

On Wed, Dec 14, 2016, 10:08 AM Francis Gulotta wizard@roborooter.com wrote:

Yep

On Wed, Dec 14, 2016, 8:46 AM Thom Nichols notifications@github.com wrote:

ah yeah it's not perfect but probably will get you at least some of the tests you weren't able to do on CI previously? Guessing you were using an env flag or something to skip some tests on CI?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EmergingTechnologyAdvisors/node-serialport/issues/999#issuecomment-267037523, or mute the thread https://github.com/notifications/unsubscribe-auth/AABlbtYRxdZ4wkHYzJGOtUmDVwMkiDWIks5rH_NRgaJpZM4KqJ5M .

thom-nic commented 7 years ago

Cool, in that case it sounds like you can get most of the tests running on CI at least! Glad I could help.

reconbot commented 7 years ago

I have many more of the tests passing by implementing a way to disable specific tests that aren't supported. (this used to be done by platform, now an environment flag) I'm not launching socat from node (though maybe I should) and I don't have the full behavior of the arduino mimicked yet.

Failing on the read test. https://travis-ci.org/EmergingTechnologyAdvisors/node-serialport/jobs/186489179

fourfathom commented 7 years ago

Windows: SerialPort.list does not list Virtual Serial Ports, at least not those created by Franson Virtual Serial Port or Franson GPSgate. I can manually open a virtual "COM1"(for example): new SerialPort('COM1'....

The virtual ports don't show up on Windows Device Manager, so perhaps there's not an easy way to fix this. I can show the virtual ports in my NavMonPc program, but I'm using Franson SerialTools for this function.

DraTeots commented 7 years ago

The same with com0com created ports. image

In the device manager com0com ports are in their own section.

At the same time, C# sees all the ports. That is SerialPort.cs source code (if it would help)

[ResourceExposure(ResourceScope.Machine)]
        [ResourceConsumption(ResourceScope.Machine)]
        public static string[] GetPortNames() {
            RegistryKey baseKey = null;
            RegistryKey serialKey = null;

            String[] portNames = null;

            RegistryPermission registryPermission = new RegistryPermission(RegistryPermissionAccess.Read, 
                                    @"HKEY_LOCAL_MACHINE\HARDWARE\DEVICEMAP\SERIALCOMM");                                    
            registryPermission.Assert();

            try {
                baseKey = Registry.LocalMachine;
                serialKey = baseKey.OpenSubKey(@"HARDWARE\DEVICEMAP\SERIALCOMM", false);

                if (serialKey != null) {

                    string[] deviceNames = serialKey.GetValueNames();
                    portNames = new String[deviceNames.Length];

                    for (int i=0; i<deviceNames.Length; i++)
                        portNames[i] = (string)serialKey.GetValue(deviceNames[i]);    
                }
            }
            finally {
                if (baseKey != null) 
                    baseKey.Close();

                if (serialKey != null) 
                    serialKey.Close();

                RegistryPermission.RevertAssert();
            }

            // If serialKey didn't exist for some reason
            if (portNames == null) 
                portNames = new String[0];

            return portNames;
        }

P.S. I'm on windows 10 creators update.

vkuehn commented 7 years ago

as I am also interested I did some search. Looks like named pipes can be used for that. Here a dicussion about that Would serialport work with named pipes ?

DraTeots commented 7 years ago

BTW, I tried reading @"HKEY_LOCAL_MACHINE\DEVICEMAP\SERIALCOMM" (inside electron using npm windows package) and indeed there are correct device addresses and port names (of virtual and FTDI ports) stored as keys and values.

console.log(windows.registry('HKEY_LOCAL_MACHINE\\HARDWARE\\DEVICEMAP\\SERIALCOMM'))

image

reconbot commented 7 years ago

Would serialport work with named pipes ?

@vkuehn we append \\.\ to the com port path that you pass in when creating a port. Per that discussion if you pass pipe\COM1 in we'll change it to \\.\pipe\COM1 and you might be set. I don't really know what windows named pipes are but give it a try and report back!

vkuehn commented 7 years ago

named pipes exist on all OS's. E.g. on Xnix/mac mkfifo mypipe creates a 0 bytes entry which can be used as named pipe. You could simply use cat/echo or any other pipe afine software like a serial terminal My Virtual box setting looks like

serialpipe

which can be used with a serial terminal connecting to com1 and from the physical host connecting to com6 .I tried on windows with putty and on mac with minicom both did work with VB named pipes Changing the readline.js from serialport examples to COM6 fails with

serialport:binding:auto-detect loading WindowsBinding +0ms serialport:main opening path: COM6 +18ms serialport:bindings open +3ms serialport:main _read queueing _read for after open +6ms serialport:main Binding #open had an error Error: Opening COM6: File not found at Error (native) +3ms

reconbot commented 7 years ago

Try pipe\COM6?

vkuehn commented 7 years ago

strangely even the putty stuff doesn't work today. Sorry I need a few days to get again a working windows box

vkuehn commented 7 years ago

btw on a mac serial port connection looks like that how would I use serialport with that ?

bildschirmfoto 2017-07-19 um 14 01 04
reconbot commented 7 years ago

Just put the path in there.


Francis Gulotta wizard@roborooter.com

On Wed, Jul 19, 2017 at 8:06 AM, vkuehn notifications@github.com wrote:

btw on a mac serial port connection looks like that how would I use serialport with that ? [image: bildschirmfoto 2017-07-19 um 14 01 04] https://user-images.githubusercontent.com/7905271/28366081-73d3c698-6c8b-11e7-9816-27927a15384b.png

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EmergingTechnologyAdvisors/node-serialport/issues/999#issuecomment-316364764, or mute the thread https://github.com/notifications/unsubscribe-auth/AABlbgTI26Ubs9KwrGGcO7UuIzrsQbsPks5sPfFOgaJpZM4KqJ5M .

vkuehn commented 7 years ago

there minicom works but readline.js from the examples fails with (node:58800) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Error Operation not supported Cannot lock port

reconbot commented 7 years ago

That's an open option that you can disable called lock https://github.com/EmergingTechnologyAdvisors/node-serialport#module_serialport--SerialPort..openOptions

vkuehn commented 7 years ago

now I get (node:60420) UnhandledPromiseRejectionWarning: Unhandled promise rejection (rejection id: 1): Error: Error: Operation not supported on socket setting custom baud rate of 9600 I think setting a baudRate is default but the pip things doesn't allow that...FYI Minitcom is set to 9600 works that way with virtual box

vkuehn commented 7 years ago

so I have a Test environment again with a windows Host and a virtual box with a windows 7 SP1 installation. Again putty on both boxes with the pipe as mentioned above works but serialport 5.0.0-beta8 fails with serialport:binding:auto-detect loading WindowsBinding +0ms serialport:main opening path: pipe\COM6 +20ms serialport:bindings open +2ms serialport:main _read queueing _read for after open +4ms serialport:main Binding #open had an error Error: Open (GetCommState): Unknown error code 1 at Error (native) +2ms

reconbot commented 7 years ago

I don't really know what to do here, GetComState must not be supported for pipes as it has no actually bad rate.

reconbot commented 7 years ago

Alright, This issue was about adding virtual ports to our CI for better testing but has turned into better supporting virtual ports in general. I think.. that's great actually.

This feels super weird, but I think maybe a "skip setup" option might be useful so people can use virtual ports that don't emulate the buadrates and flow control setup commands.

Craytor commented 7 years ago

General question - saw you guys mentioning virtual com ports... Is serialports able to recognize and use virtual com ports for data ingestion?

reconbot commented 7 years ago

Often


Francis Gulotta wizard@roborooter.com

On Tue, Aug 1, 2017 at 11:33 PM, Tyler Youschak notifications@github.com wrote:

General question - saw you guys mentioning virtual com ports... Does serialports able to recognize and use virtual com ports for data ingestion?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/EmergingTechnologyAdvisors/node-serialport/issues/999#issuecomment-319558162, or mute the thread https://github.com/notifications/unsubscribe-auth/AABlbthiL3lJAVSFyplu-5T94UCqmAhIks5sT-4IgaJpZM4KqJ5M .

hanfengcan commented 6 years ago

freevirtualserialports always disconnect when I open a virtual port

fourfathom commented 6 years ago

The virtual serial ports from Franson (GPSgate, etc.) work fine with the NodeRed serial ports. I don't think you can get the Franson virtual ports in a toolkit anymore, but I use them in my NavMonPc program, and have connected to them with the NodeRed serialport.

reconbot commented 6 years ago

Virtual serialports do not usually work as of now. I'm Happy to land patches and help debug but it's not something I know how to fix as each port seems different. If anyone is a paid customer of a virtual serialport that doesn't work, please open a support ticket with the vendor who makes it. I am happy to work with the vendors to add support for their products.