Control when DMX buffer is "dirty"

[ahodges9]

Would be great to have some control over when (or if) the DMX buffer gets flushed. This helps with two things:

1. Reduces network traffic when a universe isn't changing. No point in flooding the network with pointless updates
2. Gives better ability to sync lights across universes. I have an LED strip that spans universes and sometimes only one part of the strip updates which looks to be due to the flushing of the DMX buffer happening after I updated universe 1 but before universe 2 has been fully updated.

I think the design for having a background thread flushing at a constant rate is great so would be good to keep it that way and just have a dirty bit on each universe which must be set for the dmx data to actually be sent.

Note: I guess its also possible these issues are not due to the background thread flushing mid stream (I can add some logging in your library to validate) and instead could be something on the network side. I am seeing a good number of sequence and packet errors on the receiver side when using this library vs other E1.31 applications. Thoughts?

UPDATE: So I did some digging and it looks like my first points isn't 100% valid and you have code that reduces the report rate to 1 fps if data isn't changing. I also added more tracing in on the receiver side and it looks like packets are just not making it over. I am seeing missing sequence numbers. Not sure why this is worse with this library vs other tools...

[Hundemeier]

First of all, thank you for your comment. Your first point is a valid one. If your are using multiple universes for one fixture then this lib is currently not designed for it. The output-thread tries to send out the new data as fast as possible. Originally i designed this library for single universe use only but i think this flushing system should be easy to implement. I've added this to my ToDo list.

What do you mean by missing sequence numbers? Are they everytime 0 or is the corresponding byte in the packet just missing? (The last one could explain the errors) Do you have a wireshark log from this situation?

Update: I've added the flushing feature in the manual-flush branch. You can test it out with installation from source, because this change is not in the pypi yet.

[ahodges9]

Thanks for the work, trying to use this for a project I am working on :)

For the missing sequence number I am referring to just a missing packet on the client side. I will capture some wireshark traces to see if there is anything interesting there. However, it seems in general this protocol might not be my best option as it seems difficult to keep cross universe data in sync which limits me to 170 pixel LED strips. There isn't much we can do about packet loss as it will happen no matter what so I guess its just a matter of making it less noticeable.

[Hundemeier]

The E1.31 Standard has a sync message, that means the DMX data gets send out and the receiver receives it, but does not display it immediately. Then a sync-packet arrives and the receiver then can display the received data. See page 20 and 36 for more details: http://tsp.esta.org/tsp/documents/docs/E1-31-2016.pdf

This feature is currently not implementet in this library but could be patched in. Would requiere some work though and packet loss is not solved by this mechanism. Why do you have so much packet loss? Is this a problem of your network or does this library some weird stuff that prevents packets from being send? In a normal small network there is pretty much no packet loss what so ever. Even when transmitted over WLAN (WLAN just increases the ping a little bit). At least thats my experience.

[ahodges9]

Ah interesting I wonder if that is the source of some of the visual glitches even without packet loss. I essentially have the whole string cycling through the same color every second and 1/15 times or so the strip is split between two colors as one universe doesn't update. Looking at the controller I would see a missing packet every ~25 frames, which would account for more than half the glitches.

I will have to take a look and see why I am getting so much packet loss as I suspect there is something odd going on. If I update it to cycle ever 1/10th of a second I really don't see that much increase in the number of packets lost which is... unexpected. I'll dive into it more later today

[Hundemeier]

I have added the sync feature of the sACN protocol to the manual flush feature in the manual-flush branch. See the readme under Internals/Sending for more details and the example under Usage. Please note that not every receiver implementation supports this feature. See page 36 http://tsp.esta.org/tsp/documents/docs/E1-31-2016.pdf for more information

[joaopmrod]

Hi! I'm also very interested in this. Is the example on the readme file working?

[Hundemeier]

Yes. But it was not deeply tested. The problem are the receivers, because not every receiver implements this feature. So if you see no change when using the manual_flush feature, make sure your receiver supports the synchronization packets of the sACN E1.31 protcol. There is an extra example in the Readme under Sending in the manual_flush branch.

If you can confirm that this is working, then please notify me and i will put this feature in the master branch and on pypi.

[joaopmrod]

I get this error when running the example:

  File "C:/Users/joaop/PycharmProjects/centopeia_firm/testeSCAN.py", line 6, in <module>
    sender.activate_output(3)  # start sending out data in the 1st universe
  File "c:\users\joaop\pycharmprojects\centopeia_firm\src\sacn\sacn\sender.py", line 77, in activate_output
    new_output = Output(DataPacket(cid=self.__CID, sourceName=self.source_name, universe=universe))
  File "c:\users\joaop\pycharmprojects\centopeia_firm\src\sacn\sacn\messages\data_packet.py", line 23, in __init__
    self.syncAddr = sync_universe
  File "c:\users\joaop\pycharmprojects\centopeia_firm\src\sacn\sacn\messages\data_packet.py", line 60, in syncAddr
    raise TypeError(f'sync_universe must be [1-63999]! value was {sync_universe}')
TypeError: sync_universe must be [1-63999]! value was 0

I had to change that line to if sync_universe not in range(1, 64000):

[Hundemeier]

I've pushed a small fix and a new Example. That should work now.

[ahodges9]

Seems functional for me. I am seeing less synchronization issues and the only "glitches" I seem to have now is if the packet for a universe is dropped. But overall the number of glitches has dropped significantly (especially on an isolated network)

Also, this fixing another issue I was having with turning off the lights when stopping the sender. Prior I had to sleep few 1 second to ensure the final dmx_data was actually written. Now I can just manually flush that and can remove the delay 👍