[BUG] Keyboard hides Pane with Input

justjoshin83 commented 1 year ago

Describe the bug I have an Ionic app and I am using Cupertino Pane as part of my "Item Post" process. There is a slider inside of the pane, and each slide has a different input. Everything works really well until the keyboard is activated on the physical device (I'm testing on Android). When the keyboard is activated, the Pane is hidden behind the keyboard itself. The input has focus at this point, and when I start typing, the pane is then pushed just barely above the keyboard, but still not where it needs to be. This also pushes the followerElement off-screen.

If I then dismiss the keyboard and click back into the input on the pane, the keyboard is shown again, but this time, the pane starts slightly above the keyboard (but still not where it needs to be), and again pushes the followerElement off-screen.

To Reproduce Steps to reproduce the behavior:

View this sample on a mobile phone
The first slide is simply a sample with no input, so you can see how it looks when it's working well.
The second slide has a single input element. Click into that element (but don't start typing yet). This should make the keyboard appear
Notice that once the keyboard appears the Pane is hidden behind it. Instead, the keyboard should effectively resize the viewport (which I have set in the keyboard options in ionic). This should mean that the bottom of the Pane is exactly at the top of the keyboard.

Expected behavior The bottom of the Pane should sit at the top of the Keyboard when the keyboard is active. When the keyboard is dismissed, the Pane should sit at the bottom of the viewport (just as it was before the keyboard was activated).

Screenshots If applicable, add screenshots to help explain your problem.

Smartphone (please complete the following information):

Device: Android Pixel 5 emulator
OS: Android 33
Browser: Chrome

Additional context You can see a demo of this here, but you need to run it on your phone in order to experience the issue (everything works perfectly while in the browser, because there isn't an onscreen keyboard in that case).

roman-rr commented 1 year ago

@justjoshin83 Thank you for report. Let's fix it together. I seen that on your example only 1 slide for swiper.

Please record sample video with exact bug demonstrating from your device.

justjoshin83 commented 1 year ago

Hey @roman-rr I just sent you access to this video via your gmail account: https://drive.google.com/file/d/1_LQjN6pPvf0ox_hvT58mzyx5yTYEtha8/view?usp=sharing

Please let me know if you have any issues viewing it. Hopefully this helps demonstrate the issue. Also, please note that the app shown in the video is private and not to be shared or replicated. I don't think you would do that, but I need to say that just to protect my intellectual property :) Thanks again!

roman-rr commented 1 year ago

@justjoshin83 do not worry about privacy.

Right now I'm pretty straight forward come out with reason of chrome screen-resizing mechanics for on-screen keyboard (OSK).

I continue investigation asap, and here some useful links for me:

Latest chrome tests

https://developer.chrome.com/blog/viewport-resize-behavior/ https://viewport-resize-behavior.netlify.app/

Previous versions bug fix

https://medium.com/@sruthisreemenon/avoid-ui-distortions-during-keyboard-display-for-a-mobile-friendly-webpage-86eb99590a13

roman-rr commented 1 year ago

Chrome v.108+

Confirmed that since v.108 on chrome mobile with interactive-widget=resizes-overlays all bugs are disappeared, but keyboard should be handled within default keyboard will open functions.

roman-rr commented 1 year ago

Steps for this issue:

[x] Fix <v.108 releases
[x] Made a future patch for v.108+

roman-rr commented 1 year ago

@justjoshin83 Please check v1.3.2 with recent fix and let me know if it's works better.

justjoshin83 commented 1 year ago

Hi @roman-rr I'm so so sorry for the delayed response; I've had a lot of changes in my life recently and had to step away from my project for a couple of months. But I'm back now and just tested the latest version (1.3.2) and it seems to work much better. thank you so much for the quick response and great fix :)

justjoshin83 commented 1 year ago

Hi @roman-rr so sorry to bother you again, but I've run into an issue again. The old behavior is happening when I test my app as a native app on Android. Basically, I'm building my app with Ionic and have been testing in the browser (latest version of chrome) during development. Then I test on the device via a website (also using chrome). Both of these scenarios everything works really well.

But, when I build the app as a native Android app, and try to test, I am experiencing the same issues as noted in my original bug. Perhaps native android apps use a different web browser internally? I can send you an APK for you to install if you have an android phone so you can see the issue personally. Otherwise I'm happy to send a video. Sorry again for the trouble, and thank you so much for your help!

All the best, Josh

roman-rr commented 1 year ago

Hello @justjoshin83 Let's just start from the video.

justjoshin83 commented 1 year ago

Thanks so much, @roman-rr Please see a video of the issue here:

https://drive.google.com/file/d/1GJdm-KzmrELPwFkIDTCSkW93mWpoJICS/view?usp=sharing

If you have any questions or anything I can help with, please let me know. Thanks again!

roman-rr commented 1 year ago

Thanks so much, @roman-rr Please see a video of the issue here:

https://drive.google.com/file/d/1GJdm-KzmrELPwFkIDTCSkW93mWpoJICS/view?usp=sharing

If you have any questions or anything I can help with, please let me know. Thanks again!

Please specify the version of OS WebView using from Settings > Apps > Android System WebView (bottom of the page), and also Chrome version

justjoshin83 commented 1 year ago

Hi, thanks for the quick reply!

The web view version is: 112.0.5615.135 My chrome version says it's also: 112.0.5615.135 More info about my device: Android 13; Pixel 4a Build/TQ2A.230405.003

One other thing I wanted to mention, is that I am using Capacitor instead of cordova. I'm not sure if that matters, but I saw some references to "this.device.cordova" in your code and wanted to mention it just in case. Thanks again!

I hope this helps!

roman-rr commented 1 year ago

@justjoshin83 I just pushed update, you might try master branch version. Let me know!

justjoshin83 commented 1 year ago

Hey @roman-rr ! Thanks so much for the update! I installed the latest version from the master branch and am a little confused haha... Basically, it seemed to work really well the first time I installed my app with the latest code. Then, I closed my app and opened it again, and the original problem is still happening. It's very odd.

Would it matter that I manually call cupertinoPane.moveToHeight after the sheet is presented? For instance, I have a swiper in the pane, and when the user slides to a different slide, i call moveToHeight to match the height of the content in the current slide. That part works really well... it's just that when the keyboard gets presented it seems to behave inconsistently.

I'm happy to send you a build of the app so you can see, and/or if it would help to see my code I'm happy to share that as well. thank you again for your help with everything; I'm available to assist however possible.

roman-rr commented 1 year ago

@justjoshin83 Did you clear the cache and remove the previous version from node_modules?

justjoshin83 commented 1 year ago

Hey @roman-rr ! I did remove my previous node_modules... I'm actually using a CI/D platform to build the app fresh with every commit. That said, I believe your recent fix did help! The follow-element is no longer flying off the screen when the keyboard is active, which is great!

Unfortunately there is still one odd issue where the entire pane is shifted off the screen in a specific situation. The video below (starting around 1:02) shows what I'm experiencing now. I hope this helps! Sorry for so many issues, but I really really appreciate your help!

https://drive.google.com/file/d/1It9dG7f5S-j5Gt7zSUv6MTGK7MbNwctZ/view?usp=sharing

roman-rr commented 1 year ago

@justjoshin83 Thank you, lets solve it.

I think calling this method after changing slide will solve the issue (because with changing slides, some DOM heights also change, and it requires to be recalculated):

this.setBreakpoints({...});

But if you unable to do that, faster way is to make another codesandbox. Just a simple similar slider and input on second step. I will debug it much faster.

justjoshin83 commented 1 year ago

Ok great, I will try calling the set breakpoints method this afternoon and let you know how it goes.

In the meantime, unfortunately, the original issue is happening on iOS. I hadn't tested on iOS until today and noticed the follow element is flying off screen, and the pane isn't adjusting when the keyboard is active on iOS. This with the latest code in the master branch.

justjoshin83 commented 1 year ago

Hi @roman-rr , I tried calling the setBreakpoints method after the moveToHeight method, but it introduced other issues. I've created a sandbox environment that matches mine as closely as possible (https://codesandbox.io/p/sandbox/swiper-default-angular-forked-3k9z2z). That said, in order for you to actually see the issue, this code will need to be running as a native app on android or ios. The issue does not occur if running the code in Chrome (even if you're on android or ios).

If you're available, I would be willing to pay you to screenshare with me so we can work on this in real-time. This is the last issue I need to resolve before launching my app, so if you would be able to screenshare that would be amazing :)

roman-rr commented 1 year ago

@justjoshin83 The bug is fixed in v1.3.21.

Note: Configuration doesn't save in memory pane position that you specified with moveToHeight(...). This function used for one-time position change.

If you want to make pane return to position that you specified with moveToHeight(...), you can consider to set breakpoints for each slide changes. For example on first slide visible:

this.setBreakpoints({
middle: {
      enabled: true,
      height: 400,
      bounce: true
  }
});

On second slide visible:

this.setBreakpoints({
middle: {
      enabled: true,
      height: 450,
      bounce: true
  }
});

Than pane will remember breakpoints and coming back to them on keyboard close event. But, if at least few people will ask about it, we can make setHeight stored in memory, as well.

Debug tool for issue test-pane.zip

justjoshin83 commented 1 year ago

Hey @roman-rr ! Thank you so much for the detailed response! I was not using the right values when calling the setBreakpoints method before, but using your code above has fixed almost all of my problems. Thank you so so much! The only two issues that remain:

On iOS the pane still does not move with the keyboard sometimes (even when calling the setBreakpoints method). The problem does not happen on Android; Android is perfect now :). The issue on iOS can be seen in this video (https://drive.google.com/file/d/1OD-wCi25JsdALhLEfhlfssbyGfzGq2HU/view?usp=sharing)
Since using the setBreakpoints method, the follow element does not seem to follow the pane if it changes height. This isn't a big issue for me, but I wanted to let you know. This is also demonstrated in the video.

Thank you again; we're really close to having this be perfect!

roman-rr commented 1 year ago

@justjoshin83 Ok, I'm discovered body resizing with on-screen keyboard for iOS as well. That's pushed follower out of screen. Should be fixed on master branch last commit.

Other issue with iOS should be reviewed in details one-by-one. I suggest for you to add scroll-y="false" to ion-content directive element, and set parentElement: document.body to settings. This gonna fix majority of your issues, please try.

Debug tool for issue test-pane-v2.zip

justjoshin83 commented 1 year ago

@roman-rr Thank you again for the help! I just tried the latest build, and set scroll-y to false on ion-content, and set parentElement to document.body in the CupertinoSettings, and now, the keyboard doesn't appear at all haha. It's very bizarre. Perhaps another one of my settings is causing this problem. The settings for the pane in my last video are below:

paneSettings: CupertinoSettings = {
    buttonDestroy: false,
    fitScreenHeight: false,
    lowerThanBottom: false,
    animationDuration: this.paneAnimationDuration,
    parentElement: document.body,
    animationType: 'ease-in-out',
    preventClicks: true,
    showDraggable: false,
    maxFitHeight: Math.min(615, this.maxHeightWithinViewport),
    bottomClose: false,
    followerElement: '#pane-follower',
    events: {
        onTransitionEnd: (ev) => {
            this.setBreakpoints();
        }
    }
};

roman-rr commented 1 year ago

@justjoshin83 Input focused, but screen keyboard not visible on iOS devices ?

justjoshin83 commented 1 year ago

@roman-rr It appears that the input may never gain focus... This is actually happening on all platforms (ios, android, and web). Here is a video of what's happening.

That issue only happens if I set parentElement: document.body. Without that setting the keyboard shows as expected. On Android everything is working great (except the followerElement doesn't follow the pane anymore after calling the setBreakpoints method).

On iOS, the keyboard occasionally covers the pane still for the example above. It works on my other panes very well... it's just this one that I have in the video. The settings for that pane are in my previous comment. Thank you again! I'm going to send you another donation tomorrow for all of your help; I really appreciate it :)

roman-rr commented 1 year ago

@justjoshin83 Very odd. Try to place your wrapper and all pane content inside of ion-content.

<ion-content>
...
  <div class="cupertino-pane dreamr-sheet feed-actions-sheet">
  ... 
  </div>
</div>

justjoshin83 commented 1 year ago

@roman-rr OH MY GOSH!!! I'm so foolish :/ I just realized I had the pane inside the ion-content already. As soon as I moved it out of the ion-content element the keyboard works perfectly on iOS. I'm so sorry for the trouble on this one. But the good news is that now, iOS and Android work wonderfully when the keyboard is active :) Thank you so so much for your help!!

roman-rr commented 1 year ago

@justjoshin83 That's a very good news, and thank you to help make an improvements into package.

tech-systems / panes