Update Pupil Labs LSL Relay Docs

[gmierz]

This patch adds information about how to find different data fields in the data returned from Pupil Labs using the LSL Relay Plugin, and what they refer to in the Pupil Labs data format. It also gives a bit of extra information about the diameter field. The following issue has been created for this: https://github.com/pupil-labs/pupil/issues/1281

[cboulay]

What does "Use the Python representation to get " mean in the context of labstreaminglayer?

[cboulay]

Sorry, I wasn't clear. If you keep reading the docs then maybe it makes sense, but at part of the docs where you refer to the Python Representation, it hasn't been mentioned yet, so then one starts to wonder "wait, do I need to examine these in Python?".

Not just your addition, but I think that whole LSL Outlets part of the README needs to be restructured a bit. If you would like to do it then I would appreciate that, but please try to squash it down to 1 commit.

[gmierz]

Yup, I can restructure it and squash it to 1 commit.

The docs I link to have no information about Python representation's and what data fields would be in it. That doc just has the data fields that can normally be found in the data format obtained from Pupil Capture - so it could be different to what the 'Python representation' can contain. I see now why it makes sense for this whole section to be changed since this can be confusing or ambiguous.

And that's a good question: "wait, do I need to examine these in Python?" I'm not sure because I use them in Python and I don't know if it can be read everywhere - the repr() object is a string that can be parsed into a Python dict with eval() so I imagine this could be done in other languages, but it would definitely be harder. @mkassner, or @willpatera, would you be able to answer this question? And would you have any suggestions for how it should be restructured?

[cboulay]

Also tagging @papr .

[papr]

I agree that this should be changed to msgpack in order to be parable in other languages.

[gmierz]

I've tested that the creation of data works correctly with msgpack.pack(), but I haven't checked if reading it will be fine. I will post again once I've finished testing that.

[gmierz]

Just a little update. The data is being created, but I can't read it in python. It seems like XDF/LSL are poorly interacting with the byte format of messagepack. For instance, I had to make this change in pylsl.py where v.encode(...) is called for all types even when they don't have an encode attribute like the bytes python object:

    def push_sample(self, x, timestamp=0.0, pushthrough=True):
        """Push a sample into the outlet.

        Each entry in the list corresponds to one channel.

        Keyword arguments:
        x -- A list of values to push (one per channel).
        timestamp -- Optionally the capture time of the sample, in agreement 
                     with local_clock(); if omitted, the current 
                     time is used. (default 0.0)
        pushthrough -- Whether to push the sample through to the receivers  
                       instead of buffering it with subsequent samples. 
                       Note that the chunk_size, if specified at outlet 
                       construction, takes precedence over the pushthrough flag.
                       (default True)

        """
        if len(x) == self.channel_count:
            if self.channel_format == cf_string:
                x = [v.encode('utf-8') if type(v) != bytes else str(v).encode('utf-8') for v in x]
            handle_error(self.do_push_sample(self.obj, self.sample_type(*x),
                                             c_double(timestamp),
                                             c_int(pushthrough)))
        else:
            raise ValueError("length of the data must correspond to the "
                             "stream's channel count.")

It might not be correct though because I can get bytes from reading the XDF file, but I can't convert the samples back to their original forms through messagepack because I always end on the same error: msgpack.exceptions.ExtraData: unpack(b) received extra data.

I am trying to access the data in the sample with the following code - unpack requires a byte object, and we have a string object from the XDF file with no encoding which is required to get the bytes object:

decoded_sample = unpack(bytes(sample.encode('utf-8')))

Does anyone have any thoughts or ideas? I'm thinking of testing this with the use_bin_type flag set to true to see if that will help: https://msgpack-python.readthedocs.io/en/latest/api.html#msgpack.Packer

[cboulay]

@gmierz In the very near future we are going to be rewriting the git history of sccn/labstreaminglayer to clean it up, then we'll be using submodules. Unfortunately that means that this pull request will be broken.

Can you instead please make this a pull request against https://github.com/labstreaminglayer/App-PupilLabs and then close this PR when you've done so?

[cboulay]

I've made a diff of this PR. I will make a new PR using the diff after we overwrite the repo.