HyperspyUI for 4D STEM

[CSSFrancis]

@ericpre I was wondering what changes might be necessary to properly handle lazy datasets/4D STEM.

I think with the Lazy plotting and annotations this might be a little easier/ smoother than when first tried.

General Idea:

1 - The navigator should be reused for all transformations of some lazy signal using the map function to not repeatedly compute the navigator. 2 - Any map-reduce functions should only compute when asked to (ie. Virtual imaging) 3 - Vectors signals are just displayed over the signal.

This should allow use to port most of the functions in pyxem into HyperspyUI and have pretty good functionality there.

One thing that might be nice is to have the UI running a seperate event loop and then have dask run all of the parallelization etc. I'm not sure that is the case at the moment. But with lazy signals you tend to get a lot of spinning wheels.

[CSSFrancis]

@magnunor Any thoughts as well?

I've increasingly been having problems getting people to use pyxem/ hyperspy and there is a fairly big force that wants to do processing on the computer next to the microscope.

[ericpre]

1 - The navigator should be reused for all transformations of some lazy signal using the map function to not repeatedly compute the navigator.

Would it better to handle this in hyperspy?

2 - Any map-reduce functions should only compute when asked to (ie. Virtual imaging)

Could it be done by adding a docked widget which allow the user to define manually or automatically?

3 - Vectors signals are just displayed over the signal.

Do you mean adding a plugin to support a specify workflow (if I infer correctly: handling several signals, one being a vector signal?), otherwise, I guess this should all be handling in hyperspy/pyxem?

This should allow use to port most of the functions in pyxem into HyperspyUI and have pretty good functionality there.

You should already be able to run everything that you do in pyxem within HyperspyUI by using the embedded qtconsole or a script. One think that would be good is to integrate with jupyter notebook and they could two ways:

• running notebook within HyperSpyUI, a good example of that is notebook in spyder
• Use HyperSpyUI as "matplotlib backend" to manage the matplotlib figure within a jupyter notebook

In both cases, the dependent process needs to connect the already running jupyter kernel when starting.

One thing that might be nice is to have the UI running a seperate event loop and then have dask run all of the parallelization etc. I'm not sure that is the case at the moment. But with lazy signals you tend to get a lot of spinning wheels.

It should be possible to do that within HyperSpyUI because we should have good control of the event loop. In Ipython qtconsole or jupyter lab/notebook, we may not have that much control.

[ericpre]

• Use HyperSpyUI as "matplotlib backend" to manage the matplotlib figure within a jupyter notebook

xref https://github.com/hyperspy/hyperspyUI/issues/129

[CSSFrancis]

Would it better to handle this in hyperspy?

Yes I think I generally the navigation image should be reused especially for lazy images.

2 - Any map-reduce functions should only compute when asked to (ie. Virtual imaging)

Could it be done by adding a docked widget which allow the user to define manually or automatically?

That's a really good idea. Although we have to be better about not freezing the application when someone tries to compute when they shouldn't.

3 - Vectors signals are just displayed over the signal.

Do you mean adding a plugin to support a specify workflow (if I infer correctly: handling several signals, one being a vector signal?), otherwise, I guess this should all be handling in hyperspy/pyxem?

Ultimately I would like to be able to do template matching or strain mapping etc.

The problem with operations that return vectors is how do you display them in order to have a selectable signal object. We can now just plot them over a signal.

This should allow use to port most of the functions in pyxem into HyperspyUI and have pretty good functionality there.

You should already be able to run everything that you do in pyxem within HyperspyUI by using the embedded qtconsole or a script.

Yea but it might be nice to have the ability to right click and apply any function that the signal can apply.

One think that would be good is to integrate with jupyter notebook and they could two ways:

• running notebook within HyperSpyUI, a good example of that is notebook in spyder

• Use HyperSpyUI as "matplotlib backend" to manage the matplotlib figure within a jupyter notebook

In both cases, the dependent process needs to connect the already running jupyter kernel when starting.

Hmm I might look into how to do this!

One thing that might be nice is to have the UI running a seperate event loop and then have dask run all of the parallelization etc. I'm not sure that is the case at the moment. But with lazy signals you tend to get a lot of spinning wheels.

It should be possible to do that within HyperSpyUI because we should have good control of the event loop. In Ipython qtconsole or jupyter lab/notebook, we may not have that much control.

Another thing to look into I guess.

[ericpre]

That's a really good idea. Although we have to be better about not freezing the application when someone tries to compute when they shouldn't.

It should be possible and there is functionality for this, see for example: https://github.com/hyperspy/hyperspyUI/blob/fce8b44aa731d76364a42cd82449206a57afe10a/hyperspyui/uiprogressbar.py#L60-L70 It may be rough on the edge quite possible some code needs to be updated here to take into account changes in hyperspy.

Any operation that it long enough should have a progress bar, for example in case of the decomposition: https://github.com/hyperspy/hyperspyUI/blob/fce8b44aa731d76364a42cd82449206a57afe10a/hyperspyui/plugins/mva.py#L255

[CSSFrancis]

Just some other thoughts related to this:

@ericpre I'm curious about the "tree"/ DataViewWidget. What I would like to do is to "connect" signals together where you might have a workflow like this:

4D STEM Signal --> Centered 4D STEM Signal -->Polar 4D STEM Signal --> Orientation Match

In this case, it would be nice to represent this entire workflow as a single signal, with the ability to switch between viewing each of the different "steps".

[CSSFrancis]

Just curious if this is worth doing? If I was starting new for 4D STEM the way that I would tackle this is:

1. Add typing in hyperspy/pyxem. This would allow us to auto create dialogues for different functions.
2. Run everything through dask. The full lazy workflow including markers now in hyperspy would allow us to interact with data without computing for any function using map.
3. Make every fuction editable as well. Meaning that you could edit some functions parameters and it would update the resultant signals. This would make it easier to build up a full workflow.
4. Try to implement more gpu integrations. That would make the plotting a bit more interactive as you could speed up the calculations on a single chunk.