Bridge & simulation speed

[takluyver]

I just had a chat with someone (whose name I forgot to ask) who was using the Karabo bridge simulator to develop some code, and wanted it to go faster.

On my computer, with the --gen zeros option and a Python for loop calling client.next() to pull messages as quickly as possible, I'm getting about 600ms per message - i.e. about 1.7 Hz.

Profiling with line_profiler indicates that 69% of the time is spent in socket.recv(), and 31% in socket.send_multipart(). The rest of the lines in start_gen are negligible in comparison. So the server is spending a lot of time waiting for the client to receive the message, deserialise it, and ask for the next one.

A few things I can see that we might try:

• Investigate using ZMQ push/pull or pub/sub sockets instead of req/rep, so the server can queue up messages without waiting for the client to request them.
  • Or go back to emulating this in Python with a queue of messages ready to send.
• Use zero-copy send to transmit the big array data more efficiently.
• Options for simulating smaller data sources, e.g. a single detector module, which take less time to process.

[takluyver]

With the changes in #44, it can now send empty data for the full AGIPD detector (16 modules) in about 150ms, so 4x quicker than before. In this case, the server still spends >99% of its time waiting for the client to receive and deserialize data, so there's not much point speeding up the sending code further without speeding up the client.

With other options:

• Generating random data is significantly slower (~1 second per train for full AGIPD detector)
• A new option --detector AGIPDModule to simulate just one AGIPD module can run significantly quicker as it has much less data to send (~20 ms per train).

I think this is enough to close this.