Touch control

[XITRIX]

I've made a basic touch-event system, hope it could be useful.

[natinusala]

Hey, thanks for the PR! I'm sorry I didn't get to review it the first time, seeing that it's pretty large.

It's going to take me some time to process everything, can you maybe explain quickly how your implementation works / how is its architecture? Thanks!

[natinusala]

It does not look like your branch is building for Switch, I see glfw / OpenGL stuff being built 🙁

[XITRIX]

That's strange... Merged yoga branch right now and checked, make clean / make -j works fine.

[natinusala]

You added glfw back to the Switch Makefile it seems

[XITRIX]

Oh, I remember, in one of your old commits it was there, and I could not build nro, so I removed it by my self but not commited it, cause I thought you need it for any reason. Is my last commit building? I checked, where is no glfw in makefile right now.

[natinusala]

I was able to try it by removing the glfw line in the Makefile.

It works quite well, good job! Don't forget to add your name to the copyrights if you didn't already 😄

I don't know how far you wanted to go with your code, but there are a few things I noticed:

• when we use touch, the highlight doesn't disappear, while it does in HOS, it can be fixed by having a "touch" mode that's enabled when we use touch and disabled when we use the controller
• some gestures are a bit sloppy, especially scrolling, it doesn't accelerate when we flick the finger for instance. I think RetroArch has some pretty good acceleration code that's GPL so we can use it!
• pressing things only moves the highlight cursor, it doesn't actually "click" the thing. I think it should run the "A" (or "click") action if there's any on the clicked view. If you look on HOS, touching things with the finger actually clicks them.
• if you try to touch something that's not fully in view (so if it's clipped by scrolling), you won't be able to touch it, whereas in what you did it becomes focused and scrolls to the center. That's tied to the "fancy" scrolling that's not yet implemented in borealis so we can take care of it later I think.
• there's no audio but that's polishing for later 😛

How do you want to proceed? Do you want me to look at the code and review it before changing anything? Or do you want to tackle some of these issues first?

[natinusala]

In qlaunch when you touch on things, you need to touch them once to select them THEN touch them another time to click them. So actually borealis views should have two modes: a "touch to click" one and a "touch to highlight, touch another time to click" mode

[XITRIX]

I think if you'll review it, it will be better cause I'm new in c++ (I'm c#/swift developer). I tried to implement something similar like iOS handles touches, and I had a problem with idea of how acceleration on scroll should work so I'll try to find how it works in RetroArch. I know about current problems it have, cause I just used default giveFocus on items without any modifications. If you think it's ok, and you'd like me to continue touch development, I'll be happy to help with it :)

[XITRIX]

Also I have no sounds in borealis at all, do you have any idea of why that could be?

[natinusala]

Sound is only implemented on Switch right now, are you trying on Switch? It works for me, does it not work for you?

[XITRIX]

Yes, I'm trying on switch with atmosphere 17.1, hos 11.1

[natinusala]

Using touch or controller? Do you see anything relevant in the logs?

[XITRIX]

Controller, I know that I've not implemented sound with touch. That what inside logs, I'm not sure what qlaunch is, may be it's some kind related:

[natinusala]

Is your hbloader old by any chance?

[XITRIX]

It's v3.4.0

[XITRIX]

oh, it's hbmenu I'll check right now, but I think it's latest one upd. no way it's outdated, it was installed with latest atmosphere 18.0 (updated half an hour ago)

[HookedBehemoth]

This is a filesystem permission issue. The qlaunch romfs is accessible with hbloader since v2.3.3.

Don't use forwarders.

[XITRIX]

I got it, I don't know why, but I cannot run hbmenu by holding R while running any title (albums works fine, but it's an applet), so I created a forwarder for hbmenu. It worked fine when I was on SXOS, but after updating on Atmosphere it stoped working, do you know any reason of why that could happen?

[natinusala]

It looks like a botched Atmos install and/or config to me 👀

I'm going to read and review the changes when I can, it's a big PR so it might take me a while to take in lol

[XITRIX]

I converted window coordinates to app's one. I tested it on my laptop with cursor, I scaled window in different ways and everything worked fine.

[XITRIX]

Will you add an horizontal scroll any time soon? I would like to test touch inputs with complex view hierarchy like an array of touchable images inside horizontal scroll inside vertical scroll with some text in it, like in eShop app page. I tried to implement it by my self and it almost worked, but every child view was aligned by parent's scroll y == 0, no matter where by y was this child horizontal scroll was.

[XITRIX]

How did you took the names of sounds from qlaunch.bfsar? I spend several hours already and still cannot understand how to do that...

[natinusala]

Sorry for not responding earlier, I'm in the process of replacing some files to relicense the library. I want to do that before accepting more contributions, and so I'm going to take care of the frame context stuff later, when the relicense is done.

Will you add an horizontal scroll any time soon?

Soon I don't know, but I'm eventually going to need it, if you can you can show me what you have and I can comment / help out.

Keep in mind that for lists of repeated items it's better to use a recycling mechanism, and that's not yet implemented (but you can probably do it yourself).

How did you took the names of sounds from qlaunch.bfsar?

I believe the pulsar library has a function to list sound names of an archive.

[XITRIX]

I believe the pulsar library has a function to list sound names of an archive.

I had not found it, but finally I've made it myself, just ran through lots of ids and list their names if it exists.

Ok, may be I'll try to make them myself, when I'll finish with touch.

[natinusala]

I will look at everything you did in a giffy.

I have to warn you that the license of the library has been changed from GPLv3 to Apache 2.0, if you don't agree to that change please close the PR. Otherwise you can add your name to the authors file if you haven't already (and resolve the conflicts 👀 ).

[natinusala]

FYI I just pushed a change that removes controllerState from the global Application state

[XITRIX]

I have to warn you that the license of the library has been changed from GPLv3 to Apache 2.0

I don't even know what does that mean, so it is not a problem for me :)

[natinusala]

Not that I doubt of the answer but I have to ask for big contributions like that, the code is from you right? It doesn't come from any external project, or even inspired from (code washing) external projects, or their design?

[XITRIX]

I have written that code for that project from scratch. I used only concept of how it works in iOS, but most of it is just my idea of how it works. I used some names from real iOS functions, but it's only names.

[natinusala]

@XITRIX do you plan to add anything else to the metrics folder?

I wonder if, for the sake of simplicity, we shouldn't move it to core/geometry.hpp with everything inside? And a core/geometry.cpp file with the code, I want to avoid header-only code as much as possible now.

[XITRIX]

Ok, not a problem, I'll move it.

[EmmmaTech]

Looking at the files, I noticed that the scrolling example text (in resources/i18n/en-US/scroll.json) is super long and is in Russian. I'm curious, is there a reason for this? You could make it shorter, like a lot shorter.

[natinusala]

Tell me how you would test scrolling without a super long text?

[XITRIX]

+1, I used it to test long scrolling view. True, it's on Russia, but I could replace it with lorem ipsum text. It was the first I found in google with "super long text" request.

Also I've found, that there is the problem with cyrillic character "Ц" in glfw version of font, it shows like " ," without "U".

[EmmmaTech]

It's understandable if the super long text is for the scrolling, but yeah the text should be changed to the lorem ipsum example.

You could also move the current example to another folder for Russian text.

[EmmmaTech]

Tell me how you would test scrolling without a super long text?

Well, I thought an alternative could be a bunch of elements that you scroll down on. For example, a bunch of pictures on the screen, maybe some big buttons that do nothing, rectangles? And yes, I'm aware my example is similar to the first tab in the demo.

[XITRIX]

I've changed sound processing a bit. Now I'm not calling play() inside recognizer's callback anymore, now I'm setting sound to the pointer, provided by recognitionLoop() and play it inside Application. If there are several recognizers with custom sound, only the closest to the firstFesponder will be taken.

[EmmmaTech]

I wanted to try this out, and I like it. It controls pretty smoothly, even on a Mac. Only one suggestion (might be hard to implement), I wish the scrolling on GLFW platforms was easier to do (what I mean is adding support for the scroll-wheel).

[XITRIX]

I thought a lot about scrolling by wheel but I cannot realise how could I do that without any problem for users of Borealis... The main idea is to allow pan gesture recognizer work with both scenarious, like it works on iOS apps ported on Mac, it's gestures automatically translates to Mac's equivalents. I think first of all I need to separate mouse from touches and implement separate recognition scenarious for it.

[natinusala]

Borealis is not about mouse, it's about controllers and touch. As mouse is yet another "input type" with different control schemes, I wouldn't focus on making mouse usable for now.

Le jeu. 25 mars 2021 à 09:55, Vinogradov Daniil @.***> a écrit :

I thought a lot about scrolling by wheel but I cannot realise how I could do that without any problem for users of Borealis... The main idea is to allow pan gesture recognizer work with both scenarious, like it works on iOS apps ported on Mac, it's gestures automatically translates to Mac's equivalents.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/natinusala/borealis/pull/77#issuecomment-806477373, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAI3CKCXSUXWQTTRSBDG77LTFL26HANCNFSM4XBXJ4ZA .

[XITRIX]

I've added mouse scrolling, tried to use as less code as possible. Review it please.

[natinusala]

Is it implemented as part of touch or separately? Because scrolling using the wheel doesn't make sense in a touch context, since... touch screens don't have a mouse wheel. If it works it's fine but it's weird to have a hybrid touch emulation / scroll wheel on a computer.

I would rather implememt mouse as mouse with its own defined behavior and stuff

[XITRIX]

As you could guess, I'm using ios and macos as reference. Apple's UIKit has auto adaptation of its gesture recognizers for touch and mouse, so I can use the same recognizer on both platforms. I wanted to implement something like that, but right now I've implemented only scrolling wheel as separate part for mouse. May be I should fully separate their inputs and combine them in single struct i.e. "ScreenInput" and toss it through recognizers, but that will double the code of recognizers, cause then we'll need to handle both touch and mouse. I'm still not sure what will be the best solution, so decided to write something what will not have too many lines of code and will be easy to replace later.

[EmmmaTech]

Wow, it works really fluently! It feels like i'm using a web browser, which is good.

[natinusala]

@XITRIX And what purpose does it serve at the moment? Does mouse still act as a touch pointer on PC? Isn't it weird to have a half touch half mouse thing?

[XITRIX]

You could test it, it works perfectly fine now, like it should work on PC. Tap works the same on touch and mouse click, Pan also works the same like drag and drop event, and I've made another recognizer - Scroll one, which with touch works the same way as pan, but on PC also handles scrolling wheel. Also I've added NO_TOUCH_SCROLLING define for platforms without touch input, so mouse will not be able to scroll by dragging that window.

[natinusala]

TODO for me after I'm done reviewing the code:

• Remember to remove all logs
• See if everything behaves likes HOS
• See if sounds behave like HOS
• Think of how to easily integrate the "tap" gesture to the "click" action we already have, so that it just works (tm)