Support usbdmx.com interface

[peternewman]

https://www.usbdmx.com/usb_dmx_interface.html

Protocol docs: https://www.usbdmx.com/protocol.html

Appears to have no bulk update option, so might be rather slow or want some cleverness to only send changed channels. Unit is FTDI based, but with it's own micro, so would need either it's own plugin, or a disabled by default option in the usbdmx plugin so it doesn't normally grab them from the FTDI plugin (see https://github.com/OpenLightingProject/ola/pull/1556 for an example).

@clxjaguar already has some code in C for it (maybe they'd be interested in trying it): https://github.com/clxjaguar/usbdmxcurses/blob/master/usbdmx.c

[clxjaguar]

Oh yes, I would be personally REALLY interested by the feature, and of course, can test it. That device is seen by the OS as a virtual serial port, and need only to be updated with channel(s) with changed value(s).

[peternewman]

Are you interested in having a go at coding it too (given you've already got code, just outside of OLA)? Happy to help out with advice, but I'm rather reluctant to write it due to https://www.openlighting.org/ola/get-help/ola-faq/#Can_you_add_support_for_the_ltinsert-name-heregt_device and similar abandoned code like #400.

The first thing would be a combination of #1556 (for the default FTDI off stuff) and there's even a nice developer guide for adding a new device to this existing plugin: https://github.com/OpenLightingProject/ola/blob/master/plugins/usbdmx/README.developer.md#adding-support-for-a-new-usb-device

[clxjaguar]

I'm sadly not a pro programmer. I wanted to do it since years, but OLA architecture is really hard for me to understand (even with the guide) and I would need some help to achieve this.

[peternewman]

Fair enough @clxjaguar , I'm happy to help with advice if I can, or I could probably do the discovery bit of it, so you'd just need to deal with the generating and sending DMX part?

It does DMX input too doesn't it?

[peternewman]

Are you interested @clxjaguar ? Essentially you'd just need to take a DMXBuffer ( http://docs.openlighting.org/ola/doc/latest/classola_1_1_dmx_buffer.html ) and generate the data to send down the serial port from it, and I should be able to do the rest.

[clxjaguar]

Of course I'm interested. (And btw, just back from holidays so I got back access to a my computer, sorry for the response delay!)

[peternewman]

Of course I'm interested.

Great, take a look at: https://github.com/peternewman/ola/tree/usbdmxcom

Probably worth pulling that into your own branch, then you just need to write CreateFrame within plugins/usbdmx/USBDMXCom.cpp

(And btw, just back from holidays so I got back access to a my computer, sorry for the response delay!)

No worries, hope you had a nice break!

[clxjaguar]

Great, take a look at: https://github.com/peternewman/ola/tree/usbdmxcom

OK, I did git-clone it. But I'm afraid I already ran into my first problem, as ./configure says:

checking for cppunit-config... no
checking for Cppunit - version >= 1.9.6... no
configure: error: Missing cppunit, please install >= 1.9.6

(and it stops there)

I don't have this problem with the current (https://github.com/OpenLightingProject/ola.git) git repository's content:

$ ./configure
...
checking for CPPUNIT... yes
...
OLA Version 0.10.8
...

$ apt-cache show *cppunit* | depile "Package:" "Version:" "Status:"
libcppunit-1.14-0   1.14.0-3
libcppunit-dev  1.14.0-3
libcppunit-doc  1.14.0-3
libcppunit-subunit-dev  1.3.0-1
libcppunit-subunit0 1.3.0-1
libcppunit-1.13-0v5 1.13.2-2.1  deinstall

Probably worth pulling that into your own branch, [...]

Can you give me the git commands to make this properly?

[...] then you just need to write CreateFrame within plugins/usbdmx/USBDMXCom.cpp

Thanks a lot.

[peternewman]

OK, I did git-clone it. But I'm afraid I already ran into my first problem, as ./configure says:

I don't have this problem with the current (https://github.com/OpenLightingProject/ola.git) git repository's content:

Strange, can you share the config.log from both builds. I assume you've run autoreconf -i on the broken one? If you do a fresh checkout of the current ola git repo, does it still build cleanly for you?

$ apt-cache show *cppunit* | depile "Package:" "Version:" "Status:"

Can you share a dpkg -l | grep -i cppunit as I'm far more familiar with that output. I've never heard of, and can't find online, depile.

Probably worth pulling that into your own branch, [...]

Can you give me the git commands to make this properly?

This guide is probably pretty good: https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories

Alternatively in the GitHub website if you just go to edit one of the files in my branch I linked to (e.g. add your name to AUTHORS ), you should end up with a fork of your own of my branch: https://github.com/peternewman/ola/edit/usbdmxcom/AUTHORS

Then you can clone that branch as normal.

[...] then you just need to write CreateFrame within plugins/usbdmx/USBDMXCom.cpp

Thanks a lot.

I think in your case the length of the command string varies, so you might need to slightly tweak what that command generates, but essentially you may as well return the data once and use it in both async and sync modes.

[clxjaguar]

Strange, can you share the config.log from both builds.

config_nok.log config_ok.log

It seems the OLA configure version in the standard repo is 0.10.8, and the one from usbdmxcom tree is 0.9.8

I assume you've run autoreconf -i on the broken one?

I tried but doing the same. What I usually did since some times to compile OLA is:

$ libtoolize && aclocal && autoreconf ; automake --add-missing
$ ./configure
$ make all
$ make check

If you do a fresh checkout of the current ola git repo, does it still build cleanly for you?

Yes.

Can you share a dpkg -l | grep -i cppunit as I'm far more familiar with that output.

$ dpkg -l | grep -i cppunit
rc  libcppunit-1.13-0v5:amd64            1.13.2-2.1                    amd64        Unit Testing Library for C++
ii  libcppunit-1.14-0:amd64              1.14.0-3                      amd64        Unit Testing Library for C++
ii  libcppunit-dev                       1.14.0-3                      amd64        Unit Testing Library for C++
ii  libcppunit-doc                       1.14.0-3                      all          Unit Testing Library for C++
ii  libcppunit-subunit-dev:amd64         1.3.0-1                       amd64        SubunitTestProgressListener for CPPUnit - Development headers
ii  libcppunit-subunit0:amd64            1.3.0-1                       amd64        SubunitTestProgressListener for CPPUnit - C++ shared library

I've never heard of, and can't find online, depile.

"depile" is a small utility i wrote some time ago, in short, searching some data fields in different lines into columns in a single line for a single data entry.

Probably worth pulling that into your own branch, [...]

Can you give me the git commands to make this properly?

This guide is probably pretty good: https://docs.github.com/en/get-started/getting-started-with-git/managing-remote-repositories

Alternatively in the GitHub website if you just go to edit one of the files in my branch I linked to (e.g. add your name to AUTHORS ), you should end up with a fork of your own of my branch: https://github.com/peternewman/ola/edit/usbdmxcom/AUTHORS

Then you can clone that branch as normal.

Okay, i'll try..

[...] then you just need to write CreateFrame within plugins/usbdmx/USBDMXCom.cpp

Thanks a lot.

I think in your case the length of the command string varies, so you might need to slightly tweak what that command generates, but essentially you may as well return the data once and use it in both async and sync modes.

That adapter just need know what channels were updated since last frames. If nothing changed, no need to send anything at all, i'll continue to send the DMX frames. Also, if receive is activated, it will send a small message from virtual serial port each time a channel changed. It would be really cool to implement both send and receive!

[peternewman]

It seems the OLA configure version in the standard repo is 0.10.8, and the one from usbdmxcom tree is 0.9.8

I'd put good money that you've missed the branch argument when checking out my repo, or haven't switched branches. I think you're on my master one: https://github.com/peternewman/ola/blob/master/config/ola_version.m4#L20-L22

Okay, i'll try..

Cool, let us know how you get on with that bit.

That adapter just need know what channels were updated since last frames. If nothing changed, no need to send anything at all, i'll continue to send the DMX frames.

Yes, which means if you go from all full to a blackout, you've got a lot of data changes, and if you just fade one channel out, you've not got much. It just means the function will want a minor tweak, probably returning the live data length too or something.

Also, if receive is activated, it will send a small message from virtual serial port each time a channel changed. It would be really cool to implement both send and receive!

Ah yes, I'd seen, but then forgot about, receive. Lets get send working first, then we can add some bits in to cover that. There's already another usbdmx plugin device that does it, so we can just adapt that.

[clxjaguar]

It seems the OLA configure version in the standard repo is 0.10.8, and the one from usbdmxcom tree is 0.9.8

I'd put good money that you've missed the branch argument when checking out my repo, or haven't switched branches. I think you're on my master one

Oops. You're totally right! Now "./configure" runs (with some warnings?) and olad compiles correctly.

Okay, i'll try.. Cool, let us know how you get on with that bit.

Is this OK? https://github.com/clxjaguar/ola/tree/usbdmxcom

That adapter just need know what channels were updated since last frames. If nothing changed, no need to send anything at all, i'll continue to send the DMX frames.

Yes, which means if you go from all full to a blackout, you've got a lot of data changes, and if you just fade one channel out, you've not got much. It just means the function will want a minor tweak, probably returning the live data length too or something.

Exactly. But it is still only two bytes by channel, so 1024 bytes at worst. Btw, there is an existing "blackout" function, to force temporary all the channels sent values to zero, but i guess we don't want to use that.

Also, if receive is activated, it will send a small message from virtual serial port each time a channel changed. It would be really cool to implement both send and receive!

Ah yes, I'd seen, but then forgot about, receive. Lets get send working first, then we can add some bits in to cover that. There's already another usbdmx plugin device that does it, so we can just adapt that.

Yep. One step at a time.

[peternewman]

Oops. You're totally right! Now "./configure" runs (with some warnings?) and olad compiles correctly.

👍 You can share them, but they're probably nothing to worry about, just saying stuff isn't going to be used as it's not installed and similar.

Is this OK? https://github.com/clxjaguar/ola/tree/usbdmxcom

Yeah, you can see the changes a PR would show: https://github.com/OpenLightingProject/ola/compare/master...clxjaguar:usbdmxcom?expand=1

Exactly. But it is still only two bytes by channel, so 1024 bytes at worst. Btw, there is an existing "blackout" function, to force temporary all the channels sent values to zero, but i guess we don't want to use that.

Yeah, I'm not saying it's huge, just it's different to the existing ones, you certainly don't want to send all 1024 bytes if only one has changed.

That's down to the eternal balance between efficiency and complexity and speed. You could check for a blackout each time, which may happen a reasonable amount and be worth doing if its a few bytes of command (i.e. a big data saving). You could also consider if enough channels have changed to zero that it might be more efficient to blackout and then just set those to values (assuming the device can do atomic commits, i.e. set them all to blackout, then set some channels to non-zero, but without sending any frames in-between. This is definitely another future enhancement step though.

Yep. One step at a time.

👍

[clxjaguar]

I managed to configure and make and now "plugins/usbdmx/USBDMXCom.cpp" seems to be compiled. But olad don't load the great majority of plugins for a some reason...

$ ola_plugin_info
   Id                    Plugin Name
--------------------------------------
    1                    Dummy
    2                    ArtNet
    6                    Enttec Open DMX
   11                    E1.31 (sACN)
--------------------------------------

[peternewman]

Hi @clxjaguar ,

What does your config.log say/what options did you pass to configure? Have you got libusb-dev installed?

If none of the above is an obvious cause, please can you share olad -l 4 logs: https://www.openlighting.org/ola/get-help/ola-faq/#How_do_I_get_olad_-l_4_logs

[clxjaguar]

Hi @peternewman, Sorry again for the long delay, i didn't noticed you replied and was still waiting (did I lost mail notification?). Yes, I have libusb-dev. After the ./configure, i got :

OLA Version 0.10.8

Prefix: '/usr/local'
Compiler: 'g++ -g -O2 -std=gnu++11 -pthread '
Linker: '/usr/bin/ld -m elf_x86_64   -ldl '
Python:

Python API: no
Java API: no
Enable HTTP Server: yes
RDM Responder Tests: no
Ja Rule: no
Enabled Plugins: artnet dummy e131 espnet gpio karate kinet milinst nanoleaf opendmx openpixelcontrol pathport renard sandnet shownet spi spidmx stageprofi uart$
UUCP Lock Directory: /var/lock

config.log

PASS: tools/ola_trigger/FileValidateTest.sh
============================================================================
Testsuite summary for OLA 0.10.8
============================================================================
# TOTAL: 82
# PASS:  82
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================

$ olad/.libs/olad -l 4 -c /ola/confdir 2> olad.log

olad.log

[peternewman]

Hi @peternewman, Sorry again for the long delay, i didn't noticed you replied and was still waiting (did I lost mail notification?).

No worries.

Note how configure reports:

OLA Version 0.10.8

I assume usbdmx was on the end of this list:

Enabled Plugins: artnet dummy e131 espnet gpio karate kinet milinst nanoleaf opendmx openpixelcontrol pathport renard sandnet shownet spi spidmx stageprofi uart$

It's all enabled fine here:

config.log

But your olad run:

olad.log

Says olad/Olad.cpp:98: OLA Daemon version 0.10.6

I suspect you've got an old packaged or manual install of OLA/OLA libs floating around that it's loading when you run it up (which doesn't have usbdmx in).

I assume you haven't disabled anything in your confdir anyway?

[clxjaguar]

Okay, this was because i was running OLA by executing directly /ola/olad/.libs/olad and not /ola/olad/olad (and i don't remember the reason i started to do this). i can't explain why there is another version there, the /ola repo was brand new, also i tried purging existing ola (not running) ola package from my debian, but same thing happening.

Enabled Plugins: artnet dummy e131 espnet gpio karate kinet milinst nanoleaf opendmx openpixelcontrol pathport renard sandnet shownet spi spidmx stageprofi uartdmx usbdmx usbpro

$ /ola/olad/olad --config-dir=/ola/confdir ps => /ola/olad/.libs/olad --config-dir=/ola/confdir version => 0.10.8 ola-usbdmx is here :)

$ /ola/olad/.libs/olad --config-dir=/ola/confdir ps => /ola/olad/.libs/olad --config-dir=/ola/confdir version => 0.10.6 The only plugins are: ArtNet, Dummy, E1.31 (sACN) and Enttec Open DMX

---

Now, i also have another problem. OLA is rejecting my device because: libs/usb/LibUsbAdaptor.cpp:150: Manufacturer mismatch: FTDI != USBDMX.COM Can it could be made possible to choose the device when we add the port, even if it was not the name we were expecting, or a FTDI chip at all ? it would be best if all usb devices which have a virtual serial port were selectable...

[clxjaguar]

About the mystery of the phantom v0.10.6: solved. There was OLA files remaining back from 2018 in /usr/local/libs!

[peternewman]

$ /ola/olad/olad --config-dir=/ola/confdir ps => /ola/olad/.libs/olad --config-dir=/ola/confdir version => 0.10.8 ola-usbdmx is here :)

Excellent!

Now, i also have another problem. OLA is rejecting my device because: libs/usb/LibUsbAdaptor.cpp:150: Manufacturer mismatch: FTDI != USBDMX.COM Can it could be made possible to choose the device when we add the port, even if it was not the name we were expecting, or a FTDI chip at all ? it would be best if all usb devices which have a virtual serial port were selectable...

You just need to update some of these lines, probably just the first two and it should pass: https://github.com/OpenLightingProject/ola/compare/master...clxjaguar:usbdmxcom?expand=1#diff-1baede3b27545f5ec8adbb7a7ec7a3e7589f61ed0eb630698e5b582d92995996R36-R39

We really need to keep the list restrictive, although we can add multiple matching manufacturer/product names to it, as otherwise OLA will try and talk to every FTDI device on your system (e.g. if you've got an FTDI based programmer for your microcontroller it will start trying to use that as a DMX interface).

[clxjaguar]

I've good news! The adapter is now working, and the code is here: https://github.com/clxjaguar/ola/tree/usbdmxcom (https://github.com/clxjaguar/ola/commit/fa09a3b23c2c57f25110b72d13d62484c269542f)

I tested it normally (async, by default), and also with --no-use-async-libusb (there is maybe still a problem here as the CPU consumption is 100% of one core with the no-use-async mode, but -l 4 don't show anything suspect..)

Also, there would be best if we were able to read the adapter's responses (errors handling, and DMX reception)

You just need to update some of these lines, probably just the first two and it should pass: [...] We really need to keep the list restrictive, although we can add multiple matching manufacturer/product names to it, as otherwise OLA will try and talk to every FTDI device on your system (e.g. if you've got an FTDI based programmer for your microcontroller it will start trying to use that as a DMX interface).

Can it be configurable in the config file? Because it's user defined in the building procedure: https://www.usbdmx.com/building_8.html

Last point, i've been surprised my /dev/ttyUSBx virtual com port of the dmx adapter disappears when olad is ran. I guess this is normal?

[clxjaguar]

@peternewman ?