search

catalystneuro / roiextractors

Python-based module for extracting from, converting between, and handling optical imaging data from several file formats. Inspired by SpikeInterface.
https://roiextractors.readthedocs.io/en/latest/index.html
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

Use lxml for Bruker xml parsing #346

Closed h-mayorquin closed 2 months ago

h-mayorquin commented 2 months ago

I am having troubles testing if the recent changes in https://github.com/catalystneuro/roiextractors/pull/343 work well with the data that I got from Clandinin. The problem is that the data takes too long too load (around 20 minutes). I profiled and the bottleneck is xml parsing.

This PR uses xpath from lxml instead of the standard parser from the library. There is a speed up of around ~100 times

The firs run in the screenshot is using this PR and the other one is using the base branch here:

image

h-mayorquin commented 2 months ago

Note, adapt get stream to use lxml

h-mayorquin commented 2 months ago

This is ready to go, I also changed is_volumetric to use lxml