[Bug] ipyplot breaks input() functionality in Jupyter Lab

[BotScutters]

Hey! First off I just wanted to thank you for building this tool, I've found it really useful for some exploratory work I've been doing.

That said, I just spent a few hours trying to solve an issue that ended up being due to something in the ipyplot implementation.

My Environment

Python 3.7.5 Jupyter Lab 2.2.4 Ubuntu 16.04 inside WSL 2 on Windows 10

The Issue

When running input() in a Jupyter cell, the text-entry box would fail to render, leaving the interpreter frozen waiting for a response and no place to enter it.

The Unexpected Solution

It seems this was tied in some way to how ipyplot modifies the HTML, as I found that simply deleting the cells in my notebook that had previously run ipyplot.plot_class_tabs() made the text entry box for input() suddenly appear. The impact of this was surprisingly expansive, as it would actually cause entirely separate notebooks to exhibit this issue. I figured it out when I saw that closing a notebook containing ipyplot code caused a separate, new notebook to all of a sudden work correctly.

Just wanted to post this here in case you know of a fix, and in case anybody else encounters this issue.

[karolzak]

Hi @BotScutters ! Thanks for stopping by and sharing your issue. I haven't noticed that problem before but I'll give it a closer look. IPyPlot is generating HTML code under the hood and that probably causes some issues you mentioned although I specifically removed any JS from the picture which could affect other parts of the notebook. Could you please check for me IPyPlot version that you're using?

[BotScutters]

Hey @karolzak , sure thing, the version I have installed is 1.0.5

[karolzak]

@BotScutters so I found the root of this problem and as I suspected it's related to HTML styles that IPyPlot requires to display everything correctly. I made a huge mistake by changing the default input class style params instead of creating a child class specifically for IPyPlot. This caused HTML input box from input() not display correctly..

I already fixed this issue and pushed updated version into the branch where I'm working on v1.1 for IPyPlot. If you want to try it out you can install this new version directly from github branch with this command:

pip install git+https://github.com/karolzak/ipyplot.git@release-1-1 --upgrade

Let me know if that works for you and thanks again for raising that up! Cheers, Karol

[BotScutters]

Awesome, thanks @karolzak ! I'll test this out soon and let you know if I have further issues with it. While you're working on it, there were a couple of other things I encountered while getting started that maybe you can take a look at? Particularly I'm wondering if you'd add support for additional image filetypes besides jpeg, as well as using different iterables besides numpy arrays, such as a Pandas series. It seems like these are being prevented through a list of assert statements, but I wonder if they could be pretty easily implemented without much issue. Thanks again for your help on this, awesome tool!

[karolzak]

add support for additional image filetypes besides jpeg

Hmm the way it is implemented right now it supports the following: Supported image formats:

• Array of local storage URLs, e.g. [your/dir/img1.jpg]
• Array of remote URLs, e.g. [http://yourimages.com/img1.jpg]
• Array of PIL.Image objects
• Array of images as numpy.ndarray objects

All of the above can be anything image-like (e.g. png, tiff, jpg). Could you share a scenario or an example where you tried to use something different then jpeg and it didn't work?

different iterables besides numpy arrays, such as a Pandas series

This is definitely achievable. It should be possible to support iterables like lists, arrays, numpy arrays or pandas series. I added a new issue to track this feature request here #10

[BotScutters]

Sure, let me map out a description of my use case:

• I've got a DataFrame containing tens of thousands of rows where each row corresponds to an image.
• For each image I have a filepath along with some features and metadata
• Image files come in many formats (e.g. png, tiff, jpg, etc

In my ideal case, I can do something like the following:

ipyplot.plot_class_tabs(df['image_file_paths'], labels=df['image_class'], max_imgs_per_tab=20, img_width=150)

where the df['image_file_paths'] series may be many thousands of rows long and consist of many different filetypes. The max_imgs_per_tab parameter will limit how many images ever actually get loaded, so I don't end up with thousands of images unnecessarily loaded in RAM.

Currently it looks like ipyplot will only accept filepaths if they're JPGs, so that's what led to me asking about additional filetypes.

Ah, and one further question/request--currently, plot_class_tabs will show all of the images with an index number printed above the image. Is it possible in the current version to instead provide labels to be printed above each image instead of the indices?

Thanks for all your help!

[karolzak]

Thanks for detailed explanation of your use case!

I will think about how to allow Pandas.Series as input type however in the meantime a simple workaround would be to just use this:

ipyplot.plot_class_tabs(df['image_file_paths'].to_numpy(), labels=df['image_class'].to_numpy(), max_imgs_per_tab=20, img_width=150)

.values instead of .to_numpy() would work too.

Currently it looks like ipyplot will only accept filepaths if they're JPGs, so that's what led to me asking about additional filetypes.

Actually IPyPlot supports all file types that are supported in HTML. There should be no problems with JPG/PNG, however for certain browsers there might some issues displaying TIFF files (most browsers do not handle TIFFs in browser - IE worked for me :) ). Have you tried running IPyPlot with different file types and did it fail for you? Care to share an example?

Ah, and one further question/request--currently, plot_class_tabs will show all of the images with an index number printed above the image. Is it possible in the current version to instead provide labels to be printed above each image instead of the indices?

Currently for plot_class_tabs class names are displayed as tab labels as below: Is this not satisfactory for you? Why would you want to replace indexes with label names since the label name is the same for all the images in current active tab?

[BotScutters]

Awesome, yeah, using .to_numpy() is how I've been making it work thus far. For the distinction between JPG/PNG, I could have sworn I had previously seen an assert statement enforcing that the filetype be jpg, but I'm not finding it now... so maybe I was just hallucinating on that point 😛 If I come across that again I'll bring it up here.

As for the issue around modifying what's displayed on the indices, the use case I'm imagining isn't putting the labels there, but actually values or text that are specific to each image. For example, in using this as a tool for viewing image classification results, I could display the confidence score for each image in that location.

[karolzak]

For the distinction between JPG/PNG, I could have sworn I had previously seen an assert statement enforcing that the filetype be jpg, but I'm not finding it now... so maybe I was just hallucinating on that point 😛 If I come across that again I'll bring it up here.

No worries :D. Maybe you were not hallucinating but it could have been related to another issue like in this case #11 where IPyPlot is not displaying images for URLs with special characters like % or #.

As for the issue around modifying what's displayed on the indices, the use case I'm imagining isn't putting the labels there, but actually values or text that are specific to each image. For example, in using this as a tool for viewing image classification results, I could display the confidence score for each image in that location.

This is good, this is very good! I love this idea! Do you want to create a separate feature request as the "father" of that idea? :)