search

FXIhub / hummingbird

Monitoring and Analysing flash X-ray imaging experiments
http://fxihub.github.io/hummingbird
BSD 2-Clause "Simplified" License
16 stars 14 forks source link

ZMQ passing #33

Closed daurer closed 8 years ago

daurer commented 8 years ago

What is the intended use case for sending data to the GUI? Looking at the current framework we have:

Data flow: 

#!python
    Data
      |
    LCLSTranslator
         \
       EventTranslator
         /
    LCLSTranslator
      |
    ipc.set_current_event --> broadcast.set_current_event
      |
    conf.onEvent
         \
       analysis.function
         /
    conf.onEvent   
    ...

Should I be calling ZMQ from the "onEvent" function through the broadcast module?

#!python
    Data
      |
    LCLSTranslator
         \
       EventTranslator
         /
    LCLSTranslator
      |
    ipc.set_current_event --> broadcast.set_current_event
      |
    conf.onEvent
         \
       analysis.function
         /
    conf.onEvent  --> ipc.set_data --> broadcast.set_data --> GUI
    ...
daurer commented 8 years ago

OK I found the example "plotPulseEnergy". So for each analysis function that wants to send data we send a separate ZMQ message? If so, it would be nice to coordinate this to reduce the number of dropped messages on the GUI...

#!python
    Data
      |
    LCLSTranslator
         \
       EventTranslator
         /
    LCLSTranslator
      |
    ipc.set_current_event --> broadcast.set_current_event
      |
    conf.onEvent
         \
       analysis.function
         /
    conf.onEvent  
         \
       analysis.function --> ipc.set_data --> broadcast.set_data --> GUI
         /
    ...

Original comment by: andyofmelbourne

daurer commented 8 years ago

The GUI is actually configured in a way that it only receives signals from ZMQ, if you subscribe to a specific data source. So if you don't ask for data, the GUI does not see any messages.

Original comment by: Benedikt Daurer

daurer commented 8 years ago

Sure, that was not question however:)

I was wondering at what point we envisage sending ZMQ messages from the backend. So far it appears that this will be done by an analysis function to be executed by each rank and at every event.

Original comment by: andyofmelbourne

daurer commented 8 years ago

If we really wanted we could aggregate them at zmqserver.py, but at the moment this does not seem to be an issue. I don't think the extra messages give any measurable overhead.

Original comment by: Filipe Maia

daurer commented 8 years ago

I'll close this as "premature optimization is the root of all evil" :)

Original comment by: Filipe Maia

daurer commented 8 years ago

Ha, good call. I really need to understand ZMQ messaging a bit better. That is, under what conditions will messages get dropped and so on...

Original comment by: andyofmelbourne