Widget idea: table showing current point coordinates and residuals for each point

[jni]

(Related: #21)

During a pair session today @andreasmarnold came up with what I think is a really good idea: during alignment, have a table widget showing all the points added so far in each layer, together with the associated residuals. Then, one can go back and remove high-residual points from the table, which may have been added less precisely than others. This is easier than manually going through points, especially because the conditions on the "move layers" callback (which alternately bring the reference and moving images and points to the front) are not very sophisticated, and can get weirdly out of sync if you start removing points in the "wrong" order. By removing points in corresponding pairs through the table, we circumvent this issue altogether.

The magicgui table widget might be usable here. It might be a matter of:

• Adding a(n initially empty) table as a return value from the start_affinder function
• Add a callback to add rows to the table as points get added
• Add a callback for when row values get deleted to actually delete the whole row and the corresponding points in the points layers

The last bit is a bit tricky because by default, selecting a row in a magicgui table and pressing backspace clears the row values, rather than deleting the row. We'll have to look at the internal representation of the table to see how easy it is to delete a row "in place". I also expect that people will expect that editing the values in the table actually edits the point coordinates, but let's call that a stretch goal for now. 😂

CC @tlambert03 in case you have some comments about that proposed implementation / the Table widget API. 🙏

[tlambert03]

We'll have to look at the internal representation of the table to see how easy it is to delete a row "in place".

there's a private method that does it ... so it should just be a matter of handing the UI interaction

[tlambert03]

I also expect that people will expect that editing the values in the table actually edits the point coordinates, but let's call that a stretch goal for now.

for that you could connect to the Table.changed signal:

it emits a dict like this:

In [6]: t.changed.connect(print)
Out[6]: <function print at 0x1039c5080>

{'data': 8, 'row': 1, 'column': 1, 'column_header': 'b', 'row_header': '1'}
{'data': 12, 'row': 0, 'column': 1, 'column_header': 'b', 'row_header': '0'}
{'data': None, 'row': 1, 'column': 0, 'column_header': 'a', 'row_header': '1'}
{'data': 34, 'row': 1, 'column': 0, 'column_header': 'a', 'row_header': '1'}

probably one of the first annoyances you'll run into is restriction/validation on column types (https://github.com/pyapp-kit/magicgui/issues/216)

[jni]

there's a private method that does it ... so it should just be a matter of handing the UI interaction

I'll play with it, but do you think that this should be part of the public API? But I'm happy to use at my own risk for now.

probably one of the first annoyances you'll run into is restriction/validation on column types

Thanks for the link! Since these are all float columns I don't expect to run into it — if a user types some non-float thing I'll happily ignore it and restore the table to the previous value. 😂 If I understand the issue correctly? Actually maybe I don't — if the table is all floats as input and I write a float in a cell, does the data become a string or does it stay float? And if the latter, what happens if I type some text?

As a side note, is an output widget the right model here @tlambert03? Or maybe I should move to a magic_class...? I'm seeing now that the result_widget appears to be "hardcoded" to a LineEdit widget? Or is that just a docstring issue?

[tlambert03]

I'll play with it, but do you think that this should be part of the public API? But I'm happy to use at my own risk for now.

I can see a world in which it's added, but have a couple thoughts:

• most of the magicgui API is about stuff that modifies the state of the widget. less so about stuff that mutates the data itself. (questions of performance aside: you could, for example, use any regular method of removing a row from your data and reset the widget.value)
• so, if I understand correctly, this particular question seems to be more about the UI, and less about the programmatic API right? you wanted the user to be able to delete a row in the widget, right? That part seems to be a bit more about what additional buttons/keybindings we add?

anyway, I think it's important to nail down exactly what the goal is here before determining what APIs are needed.

if the table is all floats as input and I write a float in a cell, does the data become a string or does it stay float?

pretty sure it stays a float. I think that was the one type that was dealt with

As a side note, is an output widget the right model here @tlambert03?

since it's something that you want the user to be able to edit directly, it kinda seems more like an input widget? I don't know, I probably need to look into what you're discussing more, but I probably wouldn't have started with an output widget. Just rather made an additional table widget and added it to the container. Not sure about magic_class and what API's it would be offering here... what part of magic class were you thinking might help here?

[jni]

you could, for example, use any regular method of removing a row from your data and reset the widget.value

you wanted the user to be able to delete a row in the widget, right? That part seems to be a bit more about what additional buttons/keybindings we add?

Well, yes. Basically, my plan before you pointed me to that method, after playing around with backspace clearing the whole row, was indeed to hook into the Table.changed event, check for any rows that are all None, and then updating the whole table. This way, in my specific widget, clearing a row with backspace would immediately delete it. So, yes, it's about keybindings, but in the absence of keybindings, I could just use an API (ideally public), which could have a performance advantage vs updating the whole table, depending on the internal implementation.

pretty sure it stays a float. I think that was the one type that was dealt with

🥳

Edit: accidentally pressed cmd enter, rest of response:

since it's something that you want the user to be able to edit directly, it kinda seems more like an input widget?

Not sure about magic_class and what API's it would be offering here... what part of magic class were you thinking might help here?

Sure, I was calling it an output widget because affinder is an early (maybe the earliest) magic_factory implementation, you might recall that you added the self-reference in magic_factory for this 😃, so it's a function that takes in image layers, and then makes points layers, and the user adds points — at which point we get a table. So from a functional description, it seems like an output.

I brought up magic_class because the above flow reminded me of our discussion where you pointed out that maybe functions were the wrong model for this — when you have repeated interactions where you modify the data, rather than inputs and outputs, a class that holds all the state and gets modified might be a more natural model?

[tlambert03]

the above flow reminded me of our discussion where you pointed out that maybe functions were the wrong model for this — when you have repeated interactions where you modify the data, rather than inputs and outputs, a class that holds all the state and gets modified might be a more natural model?

yeah, that's why I added the dataclass representation: https://pyapp-kit.github.io/magicgui/dataclasses/