search

adafruit / circuitpython

CircuitPython - a Python implementation for teaching coding with microcontrollers
https://circuitpython.org
Other
4.09k stars 1.21k forks source link

Make displayio touchscreen friendly #1598

Open TG-Techie opened 5 years ago

TG-Techie commented 5 years ago

I was thinking about how to make a displayio object touchscreen (and thus menu) compatible. In my work I add the object to a list of currently onscreen (and touchable) objects whenever it is 'placed' on the screen and remove when 'unplaced'. To me, this does not sound objecty like displayio is. so, upon further thought, i was thinking about how to make dispio object 'touchable' here are my thoughts and then my idea for implementation:

my thoughts on how to implement touch, in general, is to have a list of 'touch zone's whether the zones are the objects themselves or an instance of a touch zone class made by the touchable object. then when the screen is touched the list is scanned for any touch boxes that are around the point.

I was thinking about a solution and realized there is already a list of objects on the screen, the currently shown group. then all that needs to be known is which objects can be touched. i was thinking that adding a .touchable boolean to shapes and such could achieve this. in the background, the touchscreen would be read then loop through the group(backward) to see which objects are touchable and have the point inside their space.

For shapes they have bounds and the point can be scanned for in there, text is square, and so are images (well rectangular).

siddacious commented 5 years ago

@TG-Techie Please clean up your initial request/comment to include more information about what you're trying to achieve and how what you're asking for will help achieve your goals.

TG-Techie commented 5 years ago

my apologies, more detail for not issues too. better explanation?

siddacious commented 5 years ago

Much better, thank you.

TG-Techie commented 5 years ago

how do you add labels? or is it a permissions thing?

tannewt commented 5 years ago

We should add a function to Group that returns the child with the opaque pixel from a given coordinate.

TG-Techie commented 5 years ago

Agreed, "fetch", "pull", "of"?

dhalbert commented 5 years ago

child_at ?

TG-Techie commented 5 years ago

When you say child do you mean a pixel color or an object?

kmatch98 commented 4 years ago

@tannewt I’m interested in working on this one.

There should be a way of tapping into (or copying some of) the display redrawing code and the display elements get_pixel routines you determine which is opaque and take advantage of any x,y and mirror, rotation transform code that is already there. If you suggest an approach I will try to figure out how to make it work.

As for operation of this child_at(x,y) function, how should it deal with a child element that is a Group?

  1. If the child Group is at that pixel should it return the child Group and you would have to keep iterating to get to the actual display object? If so, the child_at function will need to accept a Group as an optional argument so that you could search for the pixel location in a specific Group (for example iterating through nested Groups) something like child_at(x1,y1,group=childGroup).

  2. Or else should child_at iterate all the way down through any Group hierarchy and find the display element that is a “leaf” of the hierarchy tree?

tannewt commented 4 years ago

@kmatch98 That's a great question! My first instinct is to return a TileGrid that matched. However, the group may actually be the best unit. Perhaps the missing piece is adding a depth kwarg that could be set to indicate that a group is fine. I imagine that would be best for a button grid for example.

kmatch98 commented 4 years ago

Ok, I'm still forming my thoughts around this concept. I think this can be done in python, but adding the opacity check may required some additions to the core.

Function definition: child_at(x, y, group, search_level=0, **require_opacity=False**)

This function iterates through this group and its subgroups to determine the top display element that is located at the given x,y pixel position. This display element could be a Group, a TileGrid or a Shape. This function will return the object depending upon the search_level. Returns None if no display elements are found at this location.

Other things that may be needed:

  1. Opacity-check functions: Need an is_opaque? or is_transparent? palette function.

  2. Invisible button mask: Need a display element that is transparent that can be used as a transparent button-mask. Could associate a function with it that could be called when the button is pressed.

If you are creating a touch-based input scheme, you may want to create boundaries that do not exactly match what is displayed on the screen. For example, you might want to have rectangular buttons, but have some visual space between the buttons. Even if the press point was just outside the button, it should respond to the closest button that was pressed.

So, you might want to have a visual display of elements, and a separate "button-mask" that is associated with these button actions. In essence an invisible layer that manages the touch handling.

  1. Return multiple display elements at the x,y location: Include an option to return all the display elements at that given position, in case your interface required some overlapping checks (e.g. tapping on a location when your rocket ship collides with an asteroid to shield it from destruction). This could be something like max_elements=7 where you return the 7 top display elements at that location, or use None to return all the elements.
tannewt commented 4 years ago

I don't think it's worth doing in Python. Instead, layer_at could be a function on the native Group object. It'll be faster and have raw access to displayio internal APIs.

I do think you'll want to handle transparency.

To handle fuzzy touch you could call layer_at for surrounding pixels as well and then decide amongst all of the results what was touched.