search

OpenLightingProject / ola

The Open Lighting Architecture - The Travel Adaptor for the Lighting Industry
https://www.openlighting.org/ola/
Other
644 stars 204 forks source link

Gracefully handle OSC bundles #1688

Open loansindi opened 3 years ago

loansindi commented 3 years ago

Setup: OLA universe configured with OSC input and DMX output with an Enttec USB DMX Pro

When updating more than a handful channels of DMX over OSC at the same time, olad runs into rate-limiting issues sending DMX over an Enttec DMX USB Pro, and periodically approximately half of a universe of DMX data is corrupted, with slot values shifting around.

Naively I expected bundling all OSC messages into a single bundle would mitigate this, but it seems that OLA sends a frame per slot regardless of bundling.

In the meantime the workaround the solves my particular problem is adding 30-40ms of delay between each OSC update.

peternewman commented 3 years ago

Example bundle packet (with 180 slots): https://hastebin.com/raw/oyahozinep

Sending these 180 slots at 40 frames a second gives 4500 frames a second (100 times a normal DMX universe).

If ola_recorder is linked in we get a segfault of it and olad:

common/io/EPoller.cpp:306: ss process time was 0.000000
common/io/Descriptor.cpp:361: Failed to send on 6: Resource temporarily unavailable
common/rpc/RpcChannel.cpp:308: Failed to send full RPC message, closing channel
ola/OlaClientWrapper.cpp:71: Server closed the connection
common/io/EPoller.cpp:306: ss process time was 0.000000
Received Segmentation fault
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(+0x9fccf)[0x7fe50504dccf]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x11390)[0x7fe5045d7390]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZNK3ola3rpc10RpcChannel10ReadHeaderEPjS2_+0x3f)[0x7fe5050df55f]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola3rpc10RpcChannel15DescriptorReadyEv+0x9e)[0x7fe5050e1efe]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io27BidirectionalFileDescriptor11PerformReadEv+0x42)[0x7fe5050592c2]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io7EPoller15CheckDescriptorEP11epoll_eventPNS0_9EPollDataE+0x67)[0x7fe505065ef7]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io7EPoller4PollEPNS0_14TimeoutManagerERKNS_12TimeIntervalE+0x26b)[0x7fe5050663db]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io12SelectServer14CheckForEventsERKNS_12TimeIntervalE+0xe9)[0x7fe50505f3d9]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io12SelectServer3RunEv+0xab)[0x7fe50505f4bb]
/home/peter/dmx-software/ola-plugfest/examples/.libs/lt-ola_recorder[0x40991d]
/home/peter/dmx-software/ola-plugfest/examples/.libs/lt-ola_recorder[0x4061e6]
/home/peter/dmx-software/ola-plugfest/examples/.libs/lt-ola_recorder[0x404aa4]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7fe50421c840]
/home/peter/dmx-software/ola-plugfest/examples/.libs/lt-ola_recorder[0x404c39]

Also crashes with just ola_dmxmonitor connected:

plugins/osc/OSCNode.cpp:145: Got OSC message for /dmx/universe/0, types are ii
plugins/dummy/DummyPort.cpp:119: Dummy port: got 512 bytes: 0xff 0xcc 0x0 0x0 0x0 0x0 0xff 0xe5 0x0 0x0 
common/io/Descriptor.cpp:361: Failed to send on 27: Resource temporarily unavailable
common/rpc/RpcChannel.cpp:308: Failed to send full RPC message, closing channel
olad/plugin_api/Universe.cpp:360: Sink client 0x1e20b90 has been removed from uni 0
common/io/EPoller.cpp:116: EPOLL_CTL_DEL 27
Received Segmentation fault
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(+0x9fccf)[0x7f46b4c7cccf]
/lib/x86_64-linux-gnu/libc.so.6(+0x354c0)[0x7f46b42b04c0]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(_ZSt18_Rb_tree_incrementPKSt18_Rb_tree_node_base+0x13)[0x7f46b48ffa73]
/home/peter/dmx-software/ola-plugfest/olad/plugin_api/.libs/libolaserverplugininterface.so.0(_ZN3ola8Universe16UpdateDependantsEv+0x84)[0x7f46b0df5374]
/home/peter/dmx-software/ola-plugfest/olad/plugin_api/.libs/libolaserverplugininterface.so.0(_ZN3ola8Universe15PortDataChangedEPNS_9InputPortE+0x7f)[0x7f46b0df555f]
/home/peter/dmx-software/ola-plugfest/plugins/osc/.libs/libolaosc.so.0(_ZN3ola6plugin3osc14OSCDataHandlerEPKcS3_PP6lo_argiPvS7_+0x2e7)[0x7f46b27a2a97]
/usr/lib/x86_64-linux-gnu/liblo.so.7(+0x76fc)[0x7f46af0ec6fc]
/usr/lib/x86_64-linux-gnu/liblo.so.7(+0x92fa)[0x7f46af0ee2fa]
/usr/lib/x86_64-linux-gnu/liblo.so.7(lo_server_recv+0x1e5)[0x7f46af0ee735]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io27BidirectionalFileDescriptor11PerformReadEv+0xe2)[0x7f46b4c88362]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io7EPoller15CheckDescriptorEP11epoll_eventPNS0_9EPollDataE+0x67)[0x7f46b4c94ef7]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io7EPoller4PollEPNS0_14TimeoutManagerERKNS_12TimeIntervalE+0x26b)[0x7f46b4c953db]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io12SelectServer14CheckForEventsERKNS_12TimeIntervalE+0xe9)[0x7f46b4c8e3d9]
/home/peter/dmx-software/ola-plugfest/common/.libs/libolacommon.so.0(_ZN3ola2io12SelectServer3RunEv+0xab)[0x7f46b4c8e4bb]
/home/peter/dmx-software/ola-plugfest/olad/.libs/lt-olad(main+0x790)[0x403780]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf0)[0x7f46b429b840]
/home/peter/dmx-software/ola-plugfest/olad/.libs/lt-olad(_start+0x29)[0x4038a9]
peternewman commented 3 years ago

For bundles see http://liblo.sourceforge.net/docs/group__liblolowlevel.html#ga1411fdb0c8037a4dd3ab19af35df50ac and lo_bundle_start_handler/lo_bundle_end_handler from http://liblo.sourceforge.net/docs/lo__types_8h.html

Since 2013-05-22: Release 0.27

peternewman commented 3 years ago

We probably need to count bundles, store osc_address and then send them all when bundle is back to zero.

peternewman commented 3 years ago

@loansindi has switched to sending a blob for now.

It would probably be nice if we handled bundles atomically for if the user has no/limited control of how the sending software behaves, however for most circumstances, if possible, sending a blob rather than a bundle of individual channel messages will be better. It will also be more efficient on the OSC wire.