inlet max_chunklen and pull_chunk max_samples not working as described

[cboulay]

I'm using the pylsl wrapper but there's nothing python specific about this problem (unless it's calling arguments out of order?)

Here's the description of max_chunklen in the pylsl documentation:

max_chunklen -- Optionally the maximum size, in samples, at which 
                        chunks are transmitted (the default corresponds to the 
                        chunk sizes used by the sender). Recording programs  
                        can use a generous size here (leaving it to the network 
                        how to pack things), while real-time applications may 
                        want a finer (perhaps 1-sample) granularity. If left 
                        unspecified (=0), the sender determines the chunk 
                        granularity. (default 0)

and the kwargs for pull_chunk

timeout -- The timeout of the operation; if passed as 0.0, then only 
                   samples available for immediate pickup will be returned. 
                   (default 0.0)
max_samples -- Maximum number of samples to return. (default 
                       1024)

For a real-time application, I want to get new samples as fast as possible so I'm setting max_chunklen=nextpow2(srate/60) which for the 44 kHz audio app turns out to be 1024.

But, in case my real-time application has a hiccup, I want to make sure I can pull as much data as are waiting so I can catch up quickly, so I'm setting max_samples=8192.

My expectation is that if my application is running quickly then most of my pull_chunk operations will return 0 or 1024 samples. If there's a hiccup then I might get anywhere between 1024 and 8192.

What I end up getting is mostly 0 with the ocassional 4096 chunk (sometimes the 4096 chunk is split across 2 pull_chunk operations). e.g.

4096, 0,0,0,0,0,0,4096,0,0,0,3940,156,0,0,4096

I take this to mean that max_chunklen isn't working as I expected. This happens for max_chunklen 8, 16, up to 8192. If I go much higher (e.g., 16384) then I end up getting clusters of 8192 (max_samples) with 0's in between.

If I set max_chunklen=1 then it does work closer to what I expected:

9,0,1047,1271,1539,239,0,0,434,0,1285,

Here is where maxchunklen actually does anything: server_stream << "Max-Chunk-Length: " << max_chunklen_ << "\r\n";

Received on the other end here.

for (auto &c : hdrline) c = ::tolower(c);
std::string type = trim(hdrline.substr(0, colon)),
                    rest = trim(hdrline.substr(colon + 1));
if (type == "max-chunk-length") chunk_granularity_ = std::stoi(rest);

And ultimately: if (chunk_granularity_) samp->pushthrough = (((++seqn) % (uint32_t)chunk_granularity_) == 0);

This all looks OK. So what's going on?