search

knossos-project / knossos_utils

Python library for interacting with KNOSSOS data sets and annotation files
GNU General Public License v2.0
9 stars 9 forks source link

xml parsing issue with load_kzip_seg #30

Open smcelroy97 opened 1 year ago

smcelroy97 commented 1 year ago

I get an error xml.etree.ElementTree.ParseError: not well-formed (invalid token): line 17, column 128 Which is an issue in the format of the xml file when parsed. I am trying to import a k.zip file that was exported from Knossos, but this import function as well as skeleton().fromNML give the same error (when attempting to use either k.zip or .nml). I used the example from this repo, so is this an issue that arises from the export?

my-tien commented 1 year ago

I assume you are referring to the same data you copy-pasted us over here.

In the pasted data we can see the dataset config path: https://agglomeration.ariadne.ai/dataset/2021-12-24.pyk.conf We can download this file and look at its contents to find out that the segmentation you are looking at is streamed from a server. When you save to .k.zip, you actually only save the changes you make to this streamed data, but you cannot simply save the entire segmentation, because it is far too large to fit into memory.

You can however load the streaming dataset directly with knossos_utlis:

dataset = KnossosDataset("/path/to/downloaded/2021-12-24.pyk.conf")

And then load the chunks you are interested in one at a time:

dataset.load_seg(
    offset=dataset.boundary // 2,
    size=(512,512,512),
    mag=1
)

So e.g. to analyze the segmentation of one cell, you would have start at one chunk containing the cell’s subobject ID and then iteratively explore adjacent chunks for occurences of the same ID.