xulman-obsoleteProjects / DAIS-wp1.3

Scijava/imglib2 images are pinged back and forth over a network...
1 stars 1 forks source link

Add challenge-response communication pattern #6

Closed xulman closed 7 years ago

xulman commented 7 years ago

The sender now pushes the data onto a client, locally buffering input data if client is not able to receive them such fast (which is often the case, as locally buffering is faster than sending data over network). If we wait for confirmation packet from the receiver before we give zeromq another piece of Img data -- this would result in buffering of the currently sending piece of data only (not the whole image).

Unlike the push-pull model, this one would require to have the image opened all the time until the sending is over. Besides for very large image (of which another zeromq-side copy cannot fit into the memory), the zeromq likely stops eating the input image and waits until buffer is (partially) sent-out or more memory is available, switching essentially to the above proposed communication model....

xulman commented 7 years ago

One can probably achieve the first paragraph just by making the sending functions blocking (i.e. control is not returned back until the portion is really sent away).

The only moment where such proposed changes would be beneficial is to log a real progress of the transaction. How much should this mini-library be GUI-aware (to report progress)?

xulman commented 7 years ago

resolved in commit 03e863b6f142bc0edbd73f78cca217643cf2afa4

Sending party sends only the headers and waits until it hears back from the receiver. If it does, it starts sending the metadata and voxel data (as fast as ZeroMQ can eat it on the sending side, possibly polluting local cache/free memory). And since the receiver is sure alive, there is a good chance that data will just be transmitted ASAP (instead of being held on the sending side).

I'll might add a final hand-shake to prevent sending function from returning before data is really on the other side.

xulman commented 7 years ago

The final handshake has been added in commit e60819b0db612e6cd95c3f753034ee86a835505e