[WIP] add camera spline widget

[kevinyamauchi]

This PR adds a widget for annotating a spline path through the data and then using it to set the camera path.

[kevinyamauchi]

I need to cut a new release of napari-animation napari-threedee for this PR to work. I will do so this week.

[alisterburt]

sorry, do you mean a new release of napari-threedee? 🙂

[kevinyamauchi]

Sorry - yes!

[psobolewskiPhD]

Test fails are due to hard-coded PyQt5 import, I have made a PR upstream to fix: https://github.com/napari-threedee/napari-threedee/pull/147

[codecov[bot]]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Comparison is base (521d0b3) 86.23% compared to head (3d27ee4) 86.36%.

Files	Patch %	Lines
napari_animation/_qt/camera_spline_widget.py	93.33%	1 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #159 +/- ## ========================================== + Coverage 86.23% 86.36% +0.13% ========================================== Files 26 27 +1 Lines 1075 1093 +18 ========================================== + Hits 927 944 +17 - Misses 148 149 +1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[psobolewskiPhD]

Maybe I'm not using it properly, but I used the camera spline path to make a spline path and that works (modulo the broken set from current view button). I then tried to add keyframes and I get a Traceback: ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

``` ValueError Traceback (most recent call last) File ~/Documents/dev/napari-animation/napari_animation/_qt/animation_widget.py:118, in AnimationWidget._capture_keyframe_callback(self=, event=False) 116 def _capture_keyframe_callback(self, event=None): 117 """Record current key-frame""" --> 118 self.animation.capture_keyframe(**self._input_state()) self.animation = self = File ~/Documents/dev/napari-animation/napari_animation/animation.py:98, in Animation.capture_keyframe(self=, steps=15, ease=)>, insert=True, position=-1) 95 new_frame.name = f"Key Frame {next(self._keyframe_counter)}" 97 if insert: ---> 98 self.key_frames.insert(position + 1, new_frame) new_frame = position = -1 self = position + 1 = 0 self.key_frames = [] 99 else: 100 self.key_frames[position] = new_frame File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/containers/_selectable_list.py:69, in SelectableEventedList.insert(self=[], index=0, value=) 66 super().insert(index, value) 67 if self._activate_on_insert: 68 # Make layer selected and unselect all others ---> 69 self.selection.active = value value = self = [] File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/containers/_selection.py:108, in Selection.active(self=Selection({}), value=) 106 self.clear() if value is None else self.select_only(value) 107 self._current = value --> 108 self.events.active(value=value) value = self = Selection({}) self.events.active = self.events = File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/event.py:768, in EventEmitter.__call__(self=, *args=(), **kwargs={'value': }) 765 self._block_counter.update([cb]) 766 continue --> 768 self._invoke_callback(cb, event if pass_event else None) event = self = cb = > pass_event = True 769 if event.blocked: 770 break File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/event.py:806, in EventEmitter._invoke_callback(self=, cb=>, event=) 804 self.disconnect(cb) 805 return --> 806 _handle_exception( self = event = cb = > (cb, event) = (>, ) 807 self.ignore_callback_errors, 808 self.print_callback_errors, 809 self, 810 cb_event=(cb, event), 811 ) File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/napari/utils/events/event.py:793, in EventEmitter._invoke_callback(self=, cb=>, event=) 791 try: 792 if event is not None: --> 793 cb(event) event = cb = > 794 else: 795 cb() File ~/Documents/dev/napari-animation/napari_animation/animation.py:268, in Animation._on_active_keyframe_changed(self=, event=) 266 if active_keyframe: 267 keyframe_index = self.key_frames.index(active_keyframe) --> 268 self.set_key_frame_index(keyframe_index) keyframe_index = 0 self = File ~/Documents/dev/napari-animation/napari_animation/animation.py:120, in Animation.set_key_frame_index(self=, index=0) 118 def set_key_frame_index(self, index: int): 119 frame_index = self._keyframe_frame_index(index) --> 120 self.set_movie_frame_index(frame_index) frame_index = 0 self = File ~/Documents/dev/napari-animation/napari_animation/animation.py:134, in Animation.set_movie_frame_index(self=, index=0) 131 if self.key_frames.selection.active != key_frame: 132 self.key_frames.selection.active = key_frame --> 134 self._frames.set_movie_frame_index(self.viewer, index) index = 0 self.viewer = Viewer(camera=Camera(center=(62.99999999999997, 77.71788787841795, 64.83293151855469), zoom=3.5162095392142843, angles=(0.6395245182826729, 33.01808292794204, 1.3184949621363917), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(143.90210915518512, 135.01486633648213, 155.84283798168246), scaled=True, size=1, style=), dims=Dims(ndim=3, ndisplay=3, last_used=0, range=((0.0, 128.0, 1.0), (0.0, 128.0, 1.0), (0.0, 128.0, 1.0)), current_step=(76, 63, 63), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[, , ], help='use <2> for transform', status='', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={'Alt-F': >, 'Alt-R': >, 'Alt-D': >, 'Alt-A': . at 0x287bc5d80>, 'Alt-B': . at 0x287bc7400>}) self = self._frames = 135 self._current_frame = index 137 except KeyError: File ~/Documents/dev/napari-animation/napari_animation/frame_sequence.py:162, in FrameSequence.set_movie_frame_index(self=, viewer=Viewer(camera=Camera(center=(62.99999999999997, ...ind_callbacks.. at 0x287bc7400>}), index=0) 161 def set_movie_frame_index(self, viewer: napari.viewer.Viewer, index: int): --> 162 self[index].apply(viewer) index = 0 viewer = Viewer(camera=Camera(center=(62.99999999999997, 77.71788787841795, 64.83293151855469), zoom=3.5162095392142843, angles=(0.6395245182826729, 33.01808292794204, 1.3184949621363917), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(143.90210915518512, 135.01486633648213, 155.84283798168246), scaled=True, size=1, style=), dims=Dims(ndim=3, ndisplay=3, last_used=0, range=((0.0, 128.0, 1.0), (0.0, 128.0, 1.0), (0.0, 128.0, 1.0)), current_step=(76, 63, 63), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[, , ], help='use <2> for transform', status='', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={'Alt-F': >, 'Alt-R': >, 'Alt-D': >, 'Alt-A': . at 0x287bc5d80>, 'Alt-B': . at 0x287bc7400>}) self = 163 self._current_index = index File ~/Documents/dev/napari-animation/napari_animation/viewer_state.py:62, in ViewerState.apply(self=ViewerState(camera={'center': (62.99999999999997...aults': Empty DataFrame Columns: [] Index: [0]}}), viewer=Viewer(camera=Camera(center=(62.99999999999997, ...ind_callbacks.. at 0x287bc7400>})) 59 original_value = layer_attributes[attribute_name] 60 # Only setattr if value has changed to avoid expensive redraws 61 # dicts can hold arrays, e.g. `color`, requiring comparisons of key/value pairs ---> 62 if layer_attribute_changed(value, original_value): value = {'path_id': (6,) int64, 'spline_color': (6,) object} original_value = {'path_id': (6,) int64, 'spline_color': (6,) object} 63 with contextlib.suppress(AttributeError): 64 setattr(layer, attribute_name, value) File ~/Documents/dev/napari-animation/napari_animation/utils.py:44, in layer_attribute_changed(value={'path_id': (6,) int64, 'spline_color': (6,) object}, original_value={'path_id': (6,) int64, 'spline_color': (6,) object}) 39 if ( 40 not isinstance(original_value, dict) 41 or value.keys() != original_value.keys() 42 ): 43 return True ---> 44 return any( value = {'path_id': (6,) int64, 'spline_color': (6,) object} original_value = {'path_id': (6,) int64, 'spline_color': (6,) object} 45 layer_attribute_changed(value[key], original_value[key]) 46 for key in value 47 ) 48 return not np.array_equal(value, original_value) File ~/Documents/dev/napari-animation/napari_animation/utils.py:45, in (.0=) 39 if ( 40 not isinstance(original_value, dict) 41 or value.keys() != original_value.keys() 42 ): 43 return True 44 return any( ---> 45 layer_attribute_changed(value[key], original_value[key]) value = {'path_id': (6,) int64, 'spline_color': (6,) object} original_value = {'path_id': (6,) int64, 'spline_color': (6,) object} value[key] = (6,) object original_value[key] = (6,) object key = 'spline_color' 46 for key in value 47 ) 48 return not np.array_equal(value, original_value) File ~/Documents/dev/napari-animation/napari_animation/utils.py:48, in layer_attribute_changed(value= (6,) object, original_value= (6,) object) 43 return True 44 return any( 45 layer_attribute_changed(value[key], original_value[key]) 46 for key in value 47 ) ---> 48 return not np.array_equal(value, original_value) value = (6,) object np.array_equal = original_value = (6,) object np = File ~/micromamba/envs/napari-418/lib/python3.10/site-packages/numpy/core/numeric.py:2439, in array_equal(a1= (6,) object, a2= (6,) object, equal_nan=False) 2437 return False 2438 if not equal_nan: -> 2439 return bool(asarray(a1 == a2).all()) a1 = (6,) object a2 = (6,) object 2440 # Handling NaN values if equal_nan is True 2441 a1nan, a2nan = isnan(a1), isnan(a2) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() ```

Looks like what I implemented in: https://github.com/napari/napari-animation/pull/181 doesn't cover path_id and spline_color properly.

Edit: Aha, they're features in the points layer: viewer.layers['n3d paths'].as_layer_data_tuple()

(array([[ 63.        ,  77.71789097,  64.83292989],
        [ 63.        , 109.17496836,  37.81894252],
        [ 63.        ,  19.60227341,  98.4226905 ],
        [ 76.        ,  83.93821701, 102.86578053],
        [ 76.        , 108.99724476,  71.76415033],
        [ 76.        ,  62.25593767,  21.46837122]]),
 {'name': 'n3d paths',
  'metadata': {'n3d_metadata': {'annotation_type': 'path'}},
  'scale': [1.0, 1.0, 1.0],
  'translate': [0.0, 0.0, 0.0],
  'rotate': [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0], [0.0, 0.0, 1.0]],
  'shear': [0.0, 0.0, 0.0],
  'affine': array([[1., 0., 0., 0.],
         [0., 1., 0., 0.],
         [0., 0., 1., 0.],
         [0., 0., 0., 1.]]),
  'opacity': 1.0,
  'blending': 'translucent',
  'visible': True,
  'experimental_clipping_planes': [],
  'symbol': array([<Symbol.DISC: 'disc'>, <Symbol.DISC: 'disc'>,
         <Symbol.DISC: 'disc'>, <Symbol.DISC: 'disc'>,
         <Symbol.DISC: 'disc'>, <Symbol.DISC: 'disc'>], dtype=object),
  'edge_width': array([0.05, 0.05, 0.05, 0.05, 0.05, 0.05]),
  'edge_width_is_relative': True,
  'face_color': array([[0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [0.12156863, 0.46666667, 0.7058824 , 1.        ]], dtype=float32),
  'face_color_cycle': array([[0.12156863, 0.46666667, 0.7058824 , 1.        ],
         [1.        , 0.49803922, 0.05490196, 1.        ],
         [0.17254902, 0.627451  , 0.17254902, 1.        ],
         [0.8392157 , 0.15294118, 0.15686275, 1.        ],
         [0.5803922 , 0.40392157, 0.7411765 , 1.        ],
         [0.54901963, 0.3372549 , 0.29411766, 1.        ],
         [0.8901961 , 0.46666667, 0.7607843 , 1.        ],
         [0.49803922, 0.49803922, 0.49803922, 1.        ],
         [0.7372549 , 0.7411765 , 0.13333334, 1.        ],
         [0.09019608, 0.74509805, 0.8117647 , 1.        ]], dtype=float32),
  'face_colormap': 'viridis',
  'face_contrast_limits': None,
  'edge_color': array([[0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ],
         [0.41176471, 0.41176471, 0.41176471, 1.        ]]),
  'edge_color_cycle': array([[1., 1., 1., 1.]], dtype=float32),
  'edge_colormap': 'viridis',
  'edge_contrast_limits': None,
  'properties': {'path_id': array([0, 0, 0, 0, 0, 0]),
   'spline_color': array([array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32),
          array([0.12156863, 0.46666667, 0.7058824 , 1.        ], dtype=float32)],
         dtype=object)},
  'property_choices': {},
  'text': {'string': {'constant': array('', dtype='<U1'),
    'encoding_type': 'ConstantStringEncoding'},
   'color': {'constant': array([0., 1., 1., 1.], dtype=float32),
    'encoding_type': 'ConstantColorEncoding'},
   'visible': True,
   'size': 12,
   'blending': <Blending.TRANSLUCENT: 'translucent'>,
   'anchor': <Anchor.CENTER: 'center'>,
   'translation': array(0.),
   'rotation': 0.0},
  'out_of_slice_display': False,
  'n_dimensional': False,
  'size': array([10, 10, 10, 10, 10, 10]),
  'ndim': 3,
  'features':    path_id                              spline_color
  0        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  1        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  2        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  3        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  4        0  [0.12156863, 0.46666667, 0.7058824, 1.0]
  5        0  [0.12156863, 0.46666667, 0.7058824, 1.0],
  'feature_defaults':    path_id  spline_color
  0        0           NaN,
  'shading': <Shading.NONE: 'none'>,
  'antialiasing': 1,
  'canvas_size_limits': (2.0, 10000.0),
  'shown': array([ True,  True,  True,  True,  True,  True])},
 'points')

...and features are Pandas:

type(viewer.layers['n3d paths'].as_layer_data_tuple()[1]['features'])
Out[8]: pandas.core.frame.DataFrame

[kevinyamauchi]

Thanks for the feedback @psobolewskiPhD ! This might be related to the properties -> features migration on the napari side.

Unfortunately, I don't think I will have the time to push this across the line in the coming weeks, so I think it's best that we close this PR for now. Hopefully I (or someone else) can pick it up in the future!