Support more viewer options in the ImJoy API

[oeway]

Would be nice to be able to pass viewer options when creating the viewer window including cmap, ui_collapsed, rotate, axes, mode etc. Just like the options accepted by the view function in the itkwidgets,

This can be passed as config:

const viewer = await api.createWindow({
  src: "https://kitware.github.io/itk-vtk-viewer/app/",
  config: {...viewer options...},
  data: { image: imageArray }
})

For updating the options later, perhaps would be also nice to have setConfig (and getConfig).

[thewtex]

:+1: :+1: support for this in #343

[oeway]

BTW, is there an option that we can pass the pixel size for different axes so the scale bar are shown in actual units (e,g. um, nm)?

[thewtex]

@oeway :+1: added here: f21851c (#343)

[oeway]

@thewtex Could you also add an option for setting the contrast stretching (e.g. min and max for linear stretching) and the cmap?

cc @muellerflorian

[thewtex]

@oeway :+1: yes, will do.

[oeway]

@thewtex I just noticed that when we have a slow internet or calling setImage with a larger image, because the transmission of data takes time, the user will have to seeing a fixed interface for quite some time before the image. In this case, I think it would be nice if we can expose an API for showing the spinner in the viewer (e.g. setLoader, setSpinner, showLoading etc.). What do you think?

[thewtex]

Could you also add an option for setting the contrast stretching (e.g. min and max for linear stretching) and the camp?

Update: this is now implemented in https://github.com/Kitware/itk-vtk-viewer/pull/343 as:

setImageColorRange(range, component, name)

getImageColorRange(component, name)

Set/get the [min, max] range of intensity values mapped to colors for the given image component identified by name.

setImageColorRangeBounds(bounds, component, name)

getImageColorRangeBounds(component, name)

Set/get the [min, max] range of intensity values for color maps that provide a bounds for user inputs.

setImageColorMap(colorMap, componentIndex, name)

 ### getImageColorMap(componentIndex, name)  Set/get the color map for the given component/channel.

[thewtex]

with a larger image, because the transmission of data takes time, the user will have to seeing a fixed interface for quite some time before the image.

With the multi-scale, we can avoid this :-), but there may be causes where the data served is not multi-resolution.

In this case, I think it would be nice if we can expose an API for showing the spinner in the viewer (e.g. setLoader, setSpinner, showLoading etc.). What do you think?

@oeway cool idea. We could have it part of the API or we could have the spinner or progress bar show when useful? Is there an example use case with the data that you are using and how you are loading it?

A similar case for spinners is a freeze / unfreeze behavior where we save a screenshot and the config to JSON for later restoration. When using in Jupyter, e.g. many viewer instances can be created on a page. WebGL only supports so many resource allocations, and it will crash the page when there are too many. We need to freeze / unfreeze viewers so only the most recently used are active.

[oeway]

You can take this one for example: https://imjoy.io/docs/#/gallery?id=itkvtk-viewer The first code block reading a 2048x2048 image into the viewer, because it’s on Binder so it takes a few seconds at least to see the image. It can easily generate confusion for the first time user.

For the multi-scale, if it’s a large single scale image, perhaps we can use the encoder to automatically provide a multi-resolution interface then send it to the viewer?

Yes! freeze / unfreeze would be also nice, I imagine it will be helpful in a long interactive ImJoy doc with many viewers open along the page.

On Tue 15 Dec 2020 at 16:53, Matt McCormick notifications@github.com wrote:

with a larger image, because the transmission of data takes time, the user will have to seeing a fixed interface for quite some time before the image.

With the multi-scale, we can avoid this :-), but there may be causes where the data served is not multi-resolution.

In this case, I think it would be nice if we can expose an API for showing the spinner in the viewer (e.g. setLoader, setSpinner, showLoading etc.). What do you think?

@oeway https://github.com/oeway cool idea. We could have it part of the API or we could have the spinner or progress bar show when useful? Is there an example use case with the data that you are using and how you are loading it?

A similar case for spinners is a freeze / unfreeze behavior where we save a screenshot and the config to JSON for later restoration. When using in Jupyter, e.g. many viewer instances can be created on a page. WebGL only supports so many resource allocations, and it will crash the page when there are too many. We need to freeze / unfreeze viewers so only the most recently used are active.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/Kitware/itk-vtk-viewer/issues/348#issuecomment-745382270, or unsubscribe https://github.com/notifications/unsubscribe-auth/AADU3S4WZIRJPFFMEGOEDU3SU6A7VANCNFSM4TJLXTGA .

[thewtex]

You can take this one for example: imjoy.io/docs/#/gallery?id=itkvtk-viewer

I like that example :blush:

The first code block reading a 2048x2048 image into the viewer, because it’s on Binder so it takes a few seconds at least to see the image. It can easily generate confusion for the first time user.

For the multi-scale, if it’s a large single scale image, perhaps we can use the encoder to automatically provide a multi-resolution interface then send it to the viewer?

Yes, we will definitely eventually have a multi-scale, compressed encoder if we have a Python kernel :-).

The pause for the viewer initialization appears to be in the data transfer. Is there a way that the imjoy-rpc codec system could be used to pass progress callbacks? We could then not only show a spinner but also the transfer progress for the default network transfer. Also, a custom codec (zarr-blosc) could emit progress information based on the number of transferred, decompressed, chunks?

[oeway]

The pause for the viewer initialization appears to be in the data transfer. Is there a way that the imjoy-rpc codec system could be used to pass progress callbacks? We could then not only show a spinner but also the transfer progress for the default network transfer. Also, a custom codec (zarr-blosc) could emit progress information based on the number of transferred, decompressed, chunks?

For custom codec, we can directly emit progress in the encoder/decoder function. For the transfer progress, typically we just send the data through a certain transport layer (e.g. jupyter or socketio) without knowing the progress. However, I think we can easily support that by changing the way we send data, for example, say we want to send a numpy array to another peer, instead of directly sending the data, we can send a getChunk function together with its total length, then the other peer can fetch data chunk by chunk. We can then easily add a callback function in the getChunk function to report the progress. This can be done by a custom codec, or we can do it directly by default, we will need to think about the api though. Any idea on how the api should look like?

[thewtex]

@oeway thanks for the updates and thoughts.

We can then easily add a callback function in the getChunk function to report the progress. This can be done by a custom codec, or we can do it directly by default, we will need to think about the api though. Any idea on how the api should look like?

We often have multi-dimensional chunks, but, for progress reporting, we often just want to know the total chunk progress... after implementing multiple imjoy interfaces for that use multi-dimensional chunks (in progress), we can deduce what makes most sense for a common API.