search

labstreaminglayer / App-LabRecorder

An application for streaming one or more LSL streams to disk in XDF file format.
MIT License
123 stars 45 forks source link

LabRecorder crashed while closing the file; saved XDF cannot be read by load_xdf #107

Open mcvain opened 10 months ago

mcvain commented 10 months ago

Hello, I am using load_xdf for Matlab to import some XDF files. It of course works well, but I noticed that my LabRecorder was crashing when writing files sometimes (conditions to replicate unknown; Windows 11). On the console window, I believe it crashed before the usual "Closing the file" message; rather, it would get stuck at one of the "Wrote footer for... " lines. But I don't see/have any crash logs. When this crash happens, the XDF file still gets saved, and it has data (filesize appears normal), but it cannot be read by load_xdf. It throws the following error: 

xdf_viewer
Error using xmlread
Java exception occurred:
org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1838; XML document structures must start and end within the same entity.

    at org.apache.xerces.util.ErrorHandlerWrapper.createSAXParseException(Unknown Source)

    at org.apache.xerces.util.ErrorHandlerWrapper.fatalError(Unknown Source)

    at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)

    at org.apache.xerces.impl.XMLErrorReporter.reportError(Unknown Source)

    at org.apache.xerces.impl.XMLScanner.reportFatalError(Unknown Source)

    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.endEntity(Unknown Source)

    at org.apache.xerces.impl.XMLDocumentScannerImpl.endEntity(Unknown Source)

    at org.apache.xerces.impl.XMLEntityManager.endEntity(Unknown Source)

    at org.apache.xerces.impl.XMLEntityScanner.load(Unknown Source)

    at org.apache.xerces.impl.XMLEntityScanner.skipString(Unknown Source)

    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanEndElement(Unknown Source)

    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown Source)

    at org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown Source)

    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

    at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)

    at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)

    at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)

    at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)

Error in load_xdf>parse_xml_struct (line 766)
result = parseChildNodes(xmlread(tmp));

Error in load_xdf (line 411)
            footer = parse_xml_struct(fread(f,len-6,'*char')');

Error in xdf_viewer (line 8)
[streams,fileheader] = load_xdf(strcat(filepath, filename));

LabRecorder version info: image

I was wondering if this is a known issue, and if there is a way to recover any data from these XDF files. It appears that it contains data but was not formatted properly for load_xdf to read. I have attached an example corrupted XDF file. 004.zip

cboulay commented 10 months ago

Here is where LabRecorder writes the footer: https://github.com/labstreaminglayer/App-LabRecorder/blob/becc09e8bf2ff7b83c6b161a55469fa0a898d879/src/recording.cpp#L258-L270 You can see that it's writing offset_lists_, which are important for synchronizing streams from different originating PCs.

However, LabRecorder also has a thread that writes offsets continuously throughout the recording, here: https://github.com/labstreaminglayer/App-LabRecorder/blob/becc09e8bf2ff7b83c6b161a55469fa0a898d879/src/recording.cpp#L314

As long as your XDF importer is capable of grabbing the clock offsets distributed throughout the file, and it does not rely on the footer, then the footer loading is optional and that can be fixed in the XDF loader.