Suggestion on how UI works

[roel-v]

Can I offer a suggestion as to how dragging the grid could be made (in my opinion) more intuitive: show 'drag handles' (circles, the same thing that is shown now for the 'selected corner') at each of the corners. Make the cursor change to a 'move' (arrows in all directions) cursor when the user hovers over the drag handle. Only then are the handles dragable.

Right now (as I understand it), clicking anywhere selects the corner that is closest to wherever the user clicked. Dragging then is done by moving the cursor to the selected drag handle, and dragging from there. However when you drag from a random spot on the plane, what happens is that the closest corner is selected and then moved there at the first drag event. This makes it very easy to over stretch the projection plane. Unless there is a use case for doing this that I'm missing? I would think moving the four corners is the only thing you really want to do.

[courtneypattison]

@roel-v Thank you so much for the suggestions! I absolutely agree that it is not obvious that the calibration canvas can/should be moved. I think marking the other corners that aren’t selected in some way is a great idea. Maybe circles that are black to show that they aren’t selected or small blue circles that grow to big ones when selected?

The reason why I chose a target cursor is because it doesn’t cover up the corner but gives a single point in the middle to align the corner of the cutting mat to. I originally had a cross hair cursor but it was harder to line up. I think the move cursor might have the same issue. One idea I have is maybe make the cursor disappear on click so it feels like you’re just moving the selected corner. I haven’t tried this out yet.

You’re right that the closest corner is selected. Originally I had it like you suggested but changed it because sometimes corners can get lost off screen. Like if a tablet user rotates the screen. It was super frustrating! Or it could happen if someone moves too far with the arrow keys.

Please let me know what you think!

[hluedeke]

With the pan feature in the newlook branch, the "hotspot" for corners is reduced to around a 150 pixel radius around the corner, so this "drag to infinity" goes away. As far as changing the icon on hover of a hotspot, I am definitely in favor of that. I think the easiest way to implement is to have an invisible div element the size of the hotspot over each corner. Then you can set a cursor hover style on that div to change the cursor.

Another way might be to test if onMouseMove, the cursor is within hotspot range of any corner and to change the cursor with js. I imagine this way is more resource intensive on the browser.

[roel-v]

OK I see about why you selected the cursor you did; I do think that having the corner area stand out in some way, with some other way of indicating which one is the current selection, is more in line with expectations from other software. Say, draw the grid in black, the not selected corners in blue, the selected one in red, something like that? And have the cursor be the regular pointing arrow except when hovering over one of the circles, then make it be the crosshair for the reasons you indicate (maybe change the cursor only when it's over the currently selected one? Or change the cursor to the 'hand' (like that for hovering over links), and change it to the crosshair when dragging starts? I'm finding the cursor being non-standard when I'm not in an 'I'm actively doing something' mode to be quite foreign. In my mind, the shape of the cursor always indicates that either something will happen when I click, or something is already happening because I'm dragging something.

But I'm not quite understanding the issue with corners being lost off screen. Do you mean that there is no longer an indication that there is more than what is currently showing, and it's no longer clear what a next click will do? I've had that happen too when I didn't go to full screen mode, is that what you mean? Because I think in cases like that, it would be more helpful to e.g. have a banner to says 'not your whole working area is showing, please go to full screen mode'. Although in the case of the user rotating a tablet it would be caused by the viewport needing to be reset? In general it seems to me that the whole mode of operation is that everything always works in screen space coordinates (instead of viewport coordinates, but I'm using Windows desktop programming terms here, maybe that's not what they're called in web dev terms) and upon rotating, the zoom level should be adjusted so that no corner is ever outside the screen space?

I'm not sure how I can easily test a branch on my local machine without a full node js setup, so I haven't tried, but does 'pan feature' mean that my previous paragraph no longer holds, i.e. the user can work in a smaller viewport within the browser and move the raster that represents the cutting mat and from which the projection matrix is derived now becomes a movable, resizable object?

[hluedeke]

And have the cursor be the regular pointing arrow except when hovering over one of the circles, then make it be the crosshair for the reasons you indicate (maybe change the cursor only when it's over the currently selected one? Or change the cursor to the 'hand' (like that for hovering over links), and change it to the crosshair when dragging starts? I'm finding the cursor being non-standard when I'm not in an 'I'm actively doing something' mode to be quite foreign.

I think we can address this pretty easily, though it will just take some time to implement. This is what I was suggesting in my previous comment - I think adjusting the cursor (for desktop anyway) will be a better indicator of where the clickable spots are, whether or not the actual bounding dots get implemented.

But I'm not quite understanding the issue with corners being lost off screen.

I could see this happening. The coordinates are being calculated off of screen width, but they are being adjusted to the browser window. Since browsers can scroll in both directions, the viewport coordinates can get really tricky to deal with. I have not seen this project on an iPad, either the current version or the newlook branch, so maybe @courtneypattison would be better to address this.

In general it seems to me that the whole mode of operation is that everything always works in screen space coordinates (instead of viewport coordinates, but I'm using Windows desktop programming terms here, maybe that's not what they're called in web dev terms) and upon rotating, the zoom level should be adjusted so that no corner is ever outside the screen space?

I would be hesitant to "rezoom", as any auto-adjustments to the calibration grid would affect the projection ratio and be pretty frustrating to the user. Once again, I have not tested this on an iPad, so it's hard to say what the best solution is here.

I'm not sure how I can easily test a branch on my local machine without a full node js setup, so I haven't tried, but does 'pan feature' mean that my previous paragraph no longer holds, i.e. the user can work in a smaller viewport within the browser and move the raster that represents the cutting mat and from which the projection matrix is derived now becomes a movable, resizable object?

Yep!

[courtneypattison]

And have the cursor be the regular pointing arrow except when hovering over one of the circles, then make it be the crosshair for the reasons you indicate (maybe change the cursor only when it's over the currently selected one? Or change the cursor to the 'hand' (like that for hovering over links), and change it to the crosshair when dragging starts? I'm finding the cursor being non-standard when I'm not in an 'I'm actively doing something' mode to be quite foreign.

I think we can address this pretty easily, though it will just take some time to implement. This is what I was suggesting in my previous comment - I think adjusting the cursor (for desktop anyway) will be a better indicator of where the clickable spots are, whether or not the actual bounding dots get implemented.

I agree with both of you. Standard cursor in calibration mode, move cursor (or something similar) when in the area where you can drag corners (or just in the circle), and crosshair (or disappear) when actually dragging corner.

But I'm not quite understanding the issue with corners being lost off screen.

I could see this happening. The coordinates are being calculated off of screen width, but they are being adjusted to the browser window. Since browsers can scroll in both directions, the viewport coordinates can get really tricky to deal with. I have not seen this project on an iPad, either the current version or the newlook branch, so maybe @courtneypattison would be better to address this.

Ugh, the screen and viewport coordinate conversion is a nightmare. I think I broke touch support for tablets at some point this week. I've mostly just been testing with simulated mobile devices in dev tools. Losing the points off screen was an issue when I didn't have a reset/delete button, but you're right that it isn't an issue anymore if you just reset the points.

In general it seems to me that the whole mode of operation is that everything always works in screen space coordinates (instead of viewport coordinates, but I'm using Windows desktop programming terms here, maybe that's not what they're called in web dev terms) and upon rotating, the zoom level should be adjusted so that no corner is ever outside the screen space?

I would be hesitant to "rezoom", as any auto-adjustments to the calibration grid would affect the projection ratio and be pretty frustrating to the user. Once again, I have not tested this on an iPad, so it's hard to say what the best solution is here.

I would be scared about adding zooming. I'm worried it will throw off people's calibrations. I've only tested on an iPad once when my brother in law was over. I think maybe just reset points? Also, I was thinking maybe four buttons like arrow keys for people using touch screens or anyone who doesn't know the arrow keys can be used for fine tuning. Thoughts?

I'm not sure how I can easily test a branch on my local machine without a full node js setup, so I haven't tried, but does 'pan feature' mean that my previous paragraph no longer holds, i.e. the user can work in a smaller viewport within the browser and move the raster that represents the cutting mat and from which the projection matrix is derived now becomes a movable, resizable object?

Yep!

The newlook branch is live now, so you can check it out! I should update the README with install instructions. I'm sorry that they aren't there.

[hluedeke]

Also, I was thinking maybe four buttons like arrow keys for people using touch screens or anyone who doesn't know the arrow keys can be used for fine tuning. Thoughts?

Sounds like a great idea!

[roel-v]

Also, I was thinking maybe four buttons like arrow keys for people using touch screens or anyone who doesn't know the arrow keys can be used for fine tuning. Thoughts?

Oh I think this is the best idea so far. This way it will work the same for both touch and mouse input, right? It will leave the actual area where you need to find the corner of the mat free, and it provides an area where it can be made obvious what to do. I'm obviously no artist but I just played around in Inkscape a bit and something like this:

on each edge makes it (well, to me, maybe others think differently :) ) immediately obvious that you can drag the whole 'aim' widget by moving the lower circle. With mouse input, cursor wise, I would maybe make this work like 'pan' - change the cursor to an open hand once you bring the mouse cursor over it, and a closed/grabbing hand while dragging, much like you'd navigate say a pdf when you're panning with the 'hand' tool (as it's known in Foxit PDF, I seem to remember Adobe also calls it that).

This is a fun UX exercise :)

[courtneypattison]

Also, I was thinking maybe four buttons like arrow keys for people using touch screens or anyone who doesn't know the arrow keys can be used for fine tuning. Thoughts?

Oh I think this is the best idea so far. This way it will work the same for both touch and mouse input, right? It will leave the actual area where you need to find the corner of the mat free, and it provides an area where it can be made obvious what to do. I'm obviously no artist but I just played around in Inkscape a bit and something like this:

on each edge makes it (well, to me, maybe others think differently :) ) immediately obvious that you can drag the whole 'aim' widget by moving the lower circle. With mouse input, cursor wise, I would maybe make this work like 'pan' - change the cursor to an open hand once you bring the mouse cursor over it, and a closed/grabbing hand while dragging, much like you'd navigate say a pdf when you're panning with the 'hand' tool (as it's known in Foxit PDF, I seem to remember Adobe also calls it that).

Are you thinking that the arrow keys would be tied to the corner that is selected? I do think that would make it obvious what to do but may be annoying when pressing because the buttons would keep moving on you

This is a fun UX exercise :)

Gosh, I'm so happy to have the help. The extent of my UI/UX knowledge is reading the material design and iOS guidelines a few years ago. Would you folks be ok with me adding you to the organization? I don't want to ask too much but it would be really nice if you could assign yourselves any issues you're looking into and you could always remove yourself if you're not interested anymore.

[hluedeke]

Gosh, I'm so happy to have the help. The extent of my UI/UX knowledge is reading the material design and iOS guidelines a few years ago. Would you folks be ok with me adding you to the organization? I don't want to ask too much but it would be really nice if you could assign yourselves any issues you're looking into and you could always remove yourself if you're not interested anymore.

Sure! I think that would be easier, as we won't accidentally work on the same things this way. Thanks for inviting me!

[roel-v]

Are you thinking that the arrow keys would be tied to the corner that is selected? I do think that would make it obvious what to do but may be annoying when pressing because the buttons would keep moving on you

To be honest, I wasn't thinking about keyboard input at all, just mouse and touch. I was thinking of doing away with the concept of having a 'selected' handle all together, but that's a very touch/mouse-only concept. For people who do have a keyboard I agree you need to be able to see what element will be affected when you press the arrow keys, be it a handle or the grid itself. But clicking/touching the handle image or grid could make the selection, and it would be obvious which one is selected as that would be the one in a color that stands out. Here's a quick mockup of what I had in mind with the handles always being visible:

(The ugly pink is only like that because it was easy to derive from the blue by changing the hue, I'm not suggesting anything to look like this)

I've had a look at how other applications deal with this, but none of them (Blender, Sketchup, paint.net, Bambu Studio) really model the same idea as here. The thing that comes closest is the Blender node editor, which has a regular arrow until you start dragging an element, at which point the cursor changes into a 4 arrow cursor. Maybe that is the most intuitive way.

Now all of this is armchair quarterbacking from my side, I should really just try out a few versions before going on about it. Now that I have a dev version working, I can see if I can find out how hard some of these things are to implement.

Would you folks be ok with me adding you to the organization?

Sure!

[courtneypattison]

From an email I received:

The past two days I am having an issue with calibration. I can grab the corners of the mat from 2-3 corners but never 4 corners. The anomaly almost always happens in the lower left corner. Whereas I can grab one of the corners of the calibration grid and the hand icon will turn into a bullseye icon, which allows me to drag and drop to the appropriate corner of my mat per the projector. The lower left-hand corner when clicked, does not change to a bullseye and will not move the corner. That corner, when touched will move the whole calibration grid. It did not have this behavior on Wednesday.

If we change to only being able to move the corners from within the radius of the selected corner and change the panning to being more central, then I think this would be fixed. Similar to what we’ve been discussing, I think, lol!

[hluedeke]

@courtneypattison This might be due to the bug I just opened up. The offset for testing grid corners is off in full screen mode. I wish I knew if this person was using fullscreen or not!

I also think maybe the hover cursor should be shown based off the hotspot calculation. That way it will be obvious when it's off. I will work on that next.

[courtneypattison]

Agreed! I sent her a message to ask if it happens only in full screen. I’ll update when I hear back from her.

[roel-v]

I think this can be closed now? My initial point was made irrelevant by the current, superior interface, and the last few comments are afaik fixed by now?

[courtneypattison]

Let’s do it!