Add support for string streams

[abcsds]

Hi @cbrnr! I think I got the string markers working. Do you have any comments or suggestions about it? I followed the specs as close as I can, but might have missed something: https://github.com/sccn/xdf/wiki/Specifications Wrote a couple tests for it too.

[cbrnr]

I am not sure if this is correctly reading a string stream. I have a couple of test files which contain regularly sampled string streams (i.e. streams with a sampling frequency other than zero), and the results don't seem quite right. In one file, I get the stream footer as the first sample, whereas in another file I even get an undefined reference.

I think the next steps here should be to

1. test the behavior on some test files which I can provide
2. see what the pyxdf package does with regularly sampled string streams.

I checked 2. with the two files I mentioned before, and for the first file (which has the text of a stream footer as its first sample) I get the same result with pyxdf. However, for the second file I do not get an undefined reference but a regular string, so something definitely seems to be off.

In general, do you have a specific use case where you need a regularly sampled string stream? I have never seen this in a real-world XDF file other than some artificial toy examples.

[cbrnr]

BTW, string markers (fs = 0) are already implemented and work – what didn't work so far were string streams with regular sampling frequency (fs > 0).

[abcsds]

fs = 0 means irregular markers, right? those that have a timestamp. In the main branch right now, in the second pass of the reader, when you try to read a chunk of tag 3 (data chunk), be it irregular or a regular string marker, this line stops that from happening. I've been wanting to fix this to be able to read my experiment's files, because I write my markers with descriptive strings.

I went step by step reading the specs, and added the functionality to read the string marker timestamp (if existent) or set the last sample time plus a time delta (if the chunk contained no timestamp). But this is the exact same code you use for setting the timestamp for non-string chunks, so when refactoring I just moved the condition of whether or not the data was of type string to how the data is read.

I understand the modification you made is because several markers can happen at the same time stamp, right? Or would that happen to be for several channels?

I tested with the minimal example file, from the @xdf-modules. It's supposed to have it's own header as the first string marker. Here's a small description: https://github.com/xdf-modules/example-files/blob/master/minimal.md

I see now that I have opened a file from one of my experiments that it has an extra ; when printing the "data" field from the stream dictionary on the terminal:

julia> streams[2]
Dict{String, Any} with 9 entries:
  "nchannels" => 1
  "name"      => "PsychoPy"
  "dtype"     => String
   ...
  "data"      => ["Intro"; "Instructions1"; … ; "img/1"; "BlockEnd";;]
  "srate"     => 0.0
   ...
  "type"      => "Markers"

But when printing the last value of the array I don't see an #undef:

julia> x[2]["data"][end]
"BlockEnd"

Could you make the file you mention available with an url? I'll add it to the tests and move from there. I can also check it with pyxdf. I did use it as a guide at the beginning, but your code was a much better reference. It looks clean and readable. Thanks!

[cbrnr]

fs = 0 means irregular markers, right? those that have a timestamp.

Yes. Regular streams can also have timestamps, but irregular streams must have a timestamp for each sample.

In the main branch right now, in the second pass of the reader, when you try to read a chunk of tag 3 (data chunk), be it irregular or a regular string marker, this line stops that from happening. I've been wanting to fix this to be able to read my experiment's files, because I write my markers with descriptive strings.

Yes, this prevents string streams from being parsed – not string markers. However, you are right that even for string markers, their "data" value did not get populated. Your fix takes care of that, and if you add my suggestion then string streams (which can have multiple channels) also work.

I see now that I have opened a file from one of my experiments that it has an extra ; when printing the "data" field from the stream dictionary on the terminal:

That is just the new way Julia writes a column vector (a Matrix).

[cbrnr]

Thanks @abcsds!