[Visualisation] Find a best way for activation (feature) map visualisation

[zaleslaw]

Currently, we have a very poor example of feature map visualization in the examples module for LeNet-5.

Hope these examples could be extended and rewritten in a more clear, Kotlin-idiomatic, and universal manner to visualize it on Swing.

But let's don't limit yourself to good old Swing, what is about modern approaches, HTML + JS tracer, or maybe Compose or you could suggest the better options here.

Let's start our research here and implement the best practices

[avan1235]

I will try to work on this feature using https://github.com/JetBrains/lets-plot-kotlin which from firsts tries works really good and gives me the ability to work on Datalore (I don't need to plot anything with Swing locally etc.)

From my perspective all kotlin-dl samples could (maybe even should) have been proposed in notebook version (if they are simple enough) because it is easier to run notebook in ready environment and not use your own CPU to train these toy models or work with them.

What do you think of creating the sample nets in jupyter notebooks format?

We could propose extra bindings to plotting library and include these funtions in kotlin-dl but still show the example nets (with their visualisations) with jupyter notebooks

[zaleslaw]

I will try to work on this feature using https://github.com/JetBrains/lets-plot-kotlin which from firsts tries works really good and gives me the ability to work on Datalore (I don't need to plot anything with Swing locally etc.)

Datalore probably have an outdated Kotlin kernel (it will be updated soon), but Kotlin DL should work without any issues Let's plot could be a perspective direction for research, but I don't remember, could it visualize images?

From my perspective all kotlin-dl samples could (maybe even should) have been proposed in notebook version (if they are simple enough) because it is easier to run notebook in ready environment and not use your own CPU to train these toy models or work with them.

The current examples in the examples module serve two purposes:

• The first goal is to be ready-made chunks of code for a huge number of custom scripts so that they can be easily integrated into custom projects.
• The second goal is to be integration tests for testing combinations of various neural network architectures, datasets, and preprocessing.

The number of examples is large, as long as they lie like code and not like a Jupiter Notebook, refactoring is easy in the IDEA, but once they become notebooks, it become just text, it will become much more expensive to maintain them, test coverage will be significantly reduced.

What do you think of creating the sample nets in jupyter notebooks format?

However, the best and most unique examples are really worth translating into notebooks and updating them once, before release, for example, making them part of the documentation (as in the docs folder), and not part of the code.

I think I should create a ticket about that and add links to a few examples that should be ported to the notebooks.

We could propose extra bindings to plotting library and include these functions in kotlin-dl but still show the example nets (with their visualizations) with Jupiter notebooks

Yes, we could, if you will have any prototypes, we could go to the lets-plot guys to discuss with them and verify your hyposethis

[avan1235]

I will try to work on this feature using https://github.com/JetBrains/lets-plot-kotlin which from firsts tries works really good and gives me the ability to work on Datalore (I don't need to plot anything with Swing locally etc.)

Datalore probably have an outdated Kotlin kernel (it will be updated soon), but Kotlin DL should work without any issues Let's plot could be a perspective direction for research, but I don't remember, could it visualize images?

I managed to visualize mnist number images as tile plots where every single pixel got its value on tile plot. I haven't seen yet the possibility to plot for example png images but imo it could be done as these images also can be represented as metrices

From my perspective all kotlin-dl samples could (maybe even should) have been proposed in notebook version (if they are simple enough) because it is easier to run notebook in ready environment and not use your own CPU to train these toy models or work with them.

The current examples in the examples module serve two purposes:

• The first goal is to be ready-made chunks of code for a huge number of custom scripts so that they can be easily integrated into custom projects.
• The second goal is to be integration tests for testing combinations of various neural network architectures, datasets, and preprocessing.

The number of examples is large, as long as they lie like code and not like a Jupiter Notebook, refactoring is easy in the IDEA, but once they become notebooks, it become just text, it will become much more expensive to maintain them, test coverage will be significantly reduced.

What do you think of creating the sample nets in jupyter notebooks format?

However, the best and most unique examples are really worth translating into notebooks and updating them once, before release, for example, making them part of the documentation (as in the docs folder), and not part of the code.

I think I should create a ticket about that and add links to a few examples that should be ported to the notebooks.

It would be great and I think it has a big educational role because it is easier to run notebook than to have the IDE run locally and downloading the whole github kotlindl project to have it structured properly.

We could propose extra bindings to plotting library and include these functions in kotlin-dl but still show the example nets (with their visualizations) with Jupiter notebooks

Yes, we could, if you will have any prototypes, we could go to the lets-plot guys to discuss with them and verify your hyposethis

Let's see at my propositions in the notebook that I shared with you for the mnist numbers classification with lenet5

[zaleslaw]

It would be great and I think it has a big educational role because it is easier to run notebook than to have the IDE run locally and downloading the whole github kotlindl project to have it structured properly.

It's a fresh point of view for me, thanks, I'm an old-school guy here and fondly tied to code downloaded and run locally on the laptop

[avan1235]

Here are my drafts of letsplot usage with lenet5 model, you just have to be logged in and click "Edit copy" to see the notebook in your space (I didn't manage to share it in another way)

About running things in notebook, it's only toy problems purpose for me and playing with framework because the auto-import feature here missed is somehow challenging. But using these "magic lines" for imports without having any knowledge of gradle may be convincing at the begining of playing with framework