Ordering by number incorrect behaviour

[vanhumbeecka]

🐛 Bug

Minor UI bug (see screenshot). When logging images with context, the UI is not ordering them correctly when the context values are of type int. The UI seems to convert ints to strings when sorting them.

To reproduce

Log images with context where values are of type int. (e.g. batch_idx in screenshot). Witness the wrong sort order in the UI

Expected behavior

I expect to see the order in terms of: batch_idx=0 batch_idx=1 batch_idx=2 ...

and NOT batch_idx=0 batch_idx=1 batch_idx=10 ...

Environment

• Aim Version (3.16.0)

Additional context

Loving this project! Keep it up 👍

[roubkar]

Hey @vanhumbeecka, thanks for reporting this issue. Indeed context dicts are being stringified and then sorted.

Moving this currently to the backlog due to the team capacity.

Also currently we're working on a new base for explorers and starting to migrate explorers to that base. We're thinking to do the same for the Run page tabs, so the experience of exploration would be the same for Images Explorer and Images tab in the Run page.

Would love to hear your feedback on this

[vanhumbeecka]

I'm only using it since last week, so my feedback might be skewed. Here is a topic I'm currently struggling with:

I'm having a hard time slicing-and-dicing stored images from runs. There are multiple dimensions to consider (name, caption, context, steps, epochs, ...), on top of 2 different views (in the image tab of a specific run, and in the images explorer tab) which allow you to do different things. It's nice to see lots of options, but this goes with tradeoffs with getting your head around it.

Very concrete use-case atm: I'm trying to visualize "reconstructed" images from variational-autoencoders. I'd like to some inputs ranked for each epoch, so I'm able to see the progress of the reconstruction (I was able to do that after multiple attempts to store the images with the 'correct' dimensions attached. What would be nice, is to have a way to put the 'original' image somewhere next to it. Currently that's only possible if you 'hack' it, by dumping your original image in the beginning/end of a run with those same dimensions attached, so you could slice-and-dice it with this included. Not sure if this is useful info though.

[gorarakelyan]

@vanhumbeecka thanks a lot for an insightful feedback, it makes a total sense! We are going to add an ability to arrange images in a grid, which will enable also dividing images by columns. Thus it will enable easily setting up a side-by-side view for original and reconstructed images.

wrt the overall experience: I got the point, do you have some approaches in mind that can provide more seamless experience for slicing-and-dicing?

[vanhumbeecka]

Hi @gorarakelyan that's a difficult question to answer. Considering this on a high-level, Aim will not be able to solve all these specific problems for everyone, hence I would focus on good integration with tools that already exist out there to cover some of that burden. Integration with tools like Hydra (config management), Optima (hyperparameter tuning), Submitit (slurm schedular), etc.. would be very nice to see. Especially for the 'slicing-and-dicing' of parameters, Hydra and Optima could be important to consider. This is of course a very subjective answer 😅

[gorarakelyan]

@vanhumbeecka sounds great! Aim already has integrations with OmegaConf and Optuna (https://github.com/optuna/optuna). Could you please share a link to Optima? I couldn't manage to find it on GitHub 🤔

[vanhumbeecka]

Sorry, I meant Optuna instead of Optima. Cool to hear, I'll need to dig some further into those integration options it seems