issue with star file reading?

[McHaillet]

Hi Lorenzo!

I wanted to try Blik as I was curious if there were any nice napari based plugins for opening tomograms with STAR format particle annotations, but I ran into the issue below. Do you have any idea what I am doing wrong? I also pasted the header of my STAR file below (it is a homecooked file), perhaps its not the right format? I did try reading it via cryohub and that seems to work fine.

Command to open tomogram and particle list:

napari -w blik -- output/particles.star output/tomo_filt.mrc

Terminal output:

Traceback (most recent call last):
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/layers/points/points.py", line 806, in size
    self._size = np.broadcast_to(size, len(self.data)).copy()
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/numpy/lib/stride_tricks.py", line 413, in broadcast_to
    return _broadcast_to(array, shape, subok=subok, readonly=True)
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/numpy/lib/stride_tricks.py", line 349, in _broadcast_to
    it = np.nditer(
ValueError: operands could not be broadcast together with remapped shapes [original->remapped]: (3,)  and requested shape (144,)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/components/viewer_model.py", line 1202, in _open_or_raise_error
    added = self._add_layers_with_plugins(
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/components/viewer_model.py", line 1318, in _add_layers_with_plugins
    added.extend(self._add_layer_from_data(*_data))
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/components/viewer_model.py", line 1392, in _add_layer_from_data
    layer = add_method(data, **(meta or {}))
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/components/viewer_model.py", line 4, in add_points
    import itertools
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/layers/points/points.py", line 506, in __init__
    self.size = size
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/layers/points/points.py", line 814, in size
    raise ValueError(
ValueError: Size is not compatible for broadcasting

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/_qt/qt_viewer.py", line 953, in _qt_open
    self.viewer.open(
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/components/viewer_model.py", line 1102, in open
    layers = self._open_or_raise_error(
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/components/viewer_model.py", line 1211, in _open_or_raise_error
    raise ReaderPluginError(
napari.errors.reader_errors.ReaderPluginError: Tried opening with blik, but failed.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/marten/miniconda3/envs/blik/bin/napari", line 8, in <module>
    sys.exit(main())
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/__main__.py", line 564, in main
    _run()
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/__main__.py", line 340, in _run
    viewer._window._qt_viewer._qt_open(
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/_qt/qt_viewer.py", line 961, in _qt_open
    handle_gui_reading(
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/_qt/dialogs/qt_reader_dialog.py", line 190, in handle_gui_reading
    readers = prepare_remaining_readers(paths, plugin_name, error)
  File "/home/marten/miniconda3/envs/blik/lib/python3.10/site-packages/napari/_qt/dialogs/qt_reader_dialog.py", line 245, in prepare_remaining_readers
    raise ReaderPluginError(
napari.errors.reader_errors.ReaderPluginError: Tried to read [output/particles.star, ...] with plugin blik, because it was associated with that file extension/because it is the only plugin capable of reading that path, but it gave an error. Try associating a different plugin or installing a different plugin for this kind of file.

My starfile looks like this:

# Created by the starfile Python package (version 0.4.12) at 20:17:22 on 25/07/2023

data_

loop_
_rlnCoordinateX #1
_rlnCoordinateY #2
_rlnCoordinateZ #3
_rlnAngleRot #4
_rlnAngleTilt #5
_rlnAnglePsi #6
_ptmLCCmax #7
_rlnDetectorPixelSize #8
_rlnMicrographName #9
_rlnMagnification #10
_rlnGroupNumber #11
181 163 86  -36.460765  -161.427020 160.142076  0.455640    13.790000   tomo200528_110.mrc  10000.000000    0
416 343 126 97.352181   -102.015107 38.385390   0.441689    13.790000   tomo200528_110.mrc  10000.000000    0
152 56  114 -159.478539 -104.188364 121.661090  0.424355    13.790000   tomo200528_110.mrc  10000.000000    0
449 180 104 -141.136705 -95.532863  -153.204503 0.422766    13.790000   tomo200528_110.mrc  10000.000000    0

[brisvag]

Hey @McHaillet, nice to see you here! :) I've been making several changes lately and haven't pushed a new version yet. Unfortunately I also need to bump the cryohub version and maybe something else, so it's not quite ready to go out.

However, you might be able to install everything you need with:

pip install "git+git://github.com/gutsche-lab/blik.git#egg=blik[all]"
pip install "napari[all]==0.4.18"
pip install "git+git://github.com/brisvag/cryohub.git@fix/io-star#egg=cryohub"

And this should work. I'm in the process of cleaning up and fixing bugs, so please tell me anything that goes wrong!

[McHaillet]

Had to switch from git+git: to git+https: to do the pip install directly from the repo, but now its working.

The command to open the starfile and tomogram now works and the visualization looks very nice!

Some things I noticed at first glance:

1. when changing the opacity of the coordinate points and particle orientation vectors they become black dots on my tomographic volume, instead of becoming translucent.
2. changing the slider of the coordinate point size or symbol does not appear to do anything. For the orientation vectors it works fine.

I can also make a github issue of these things if you want, let me know ;)

[McHaillet]

About the second point, just noticed I had to first select all the coordinate points before I could change size and shape.

[brisvag]

when changing the opacity of the coordinate points and particle orientation vectors they become black dots on my tomographic volume, instead of becoming translucent.

Does not happen for me :/ can you send a small video of it, and record exactly which steps you took from loading to the bug? And yes, would be best to make a new issue :)

About the second point, just noticed I had to first select all the coordinate points before I could change size and shape.

Not a bug, just a napari weirdness that I've been fighting to change but people disagree with :P those options refer to either the "selected points" and the points created from now on.

[McHaillet]

I made a new issue with the video attached!

Not a bug, just a napari weirdness that I've been fighting to change but people disagree with :P those options refer to either the "selected points" and the points created from now on.

Hmm, that seems a bit counter-intuitive. Now you also need to first switch to 3D viewing to efficiently select all the points.

[brisvag]

Yup, I agree. Relevant discussion: https://github.com/napari/napari/issues/5748; feel free to add weigh in there so maybe we can convince folks :P

Now you also need to first switch to 3D viewing to efficiently select all the points.

You can actually select all points in the points layer with shift+a. Take a look at all the shortucts in File->preferences->shortcuts, there's some useful stuff there!

[brisvag]

I just released a new version (0.5.0), so all the workarounds above are no longer needed!