search

johnfactotum / foliate

Read e-books in style
https://johnfactotum.github.io/foliate/
GNU General Public License v3.0
6.39k stars 293 forks source link

Touchscreen text selection/highlighting #207

Open JamesSwift opened 4 years ago

JamesSwift commented 4 years ago

I've just started using foliate and I love it! The devs have done an awesome job.

When using my tablet/laptop combo I can't highlight or select text when using the touchscreen. Is there any plan to implement this, as highlighting is something I do an awful lot and its often more comfortable to do so in tablet mode?

johnfactotum commented 4 years ago

It seems that WebKitGTK doesn't currently support touch-friendly text selection at all (which would mean that other applications like GNOME Web, Devhelp, etc., also have this issue). Unfortunately I don't have a touchscreen device so I will need help implementing/testing this.

JamesSwift commented 4 years ago

I thought that would be the issue. I am happy to help in testing touch functionality on my tablet. Just say when.

22 Nov 2019 04:44:07 John Factotum notifications@github.com:

It seems that WebKitGTK doesn't currently support touch-friendly text selection at all (which would mean that other applications like GNOME Web, Devhelp, etc., also have this issue). Unfortunately I don't have a touchscreen device so I will need help implementing/testing this.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub[https://github.com/johnfactotum/foliate/issues/207?email_source=notifications&email_token=AAP33TJFCS3OBL4XOL64OKTQU5PQVA5CNFSM4JQJ2VPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE4QWJY#issuecomment-557386535], or unsubscribe[https://github.com/notifications/unsubscribe-auth/AAP33TI4DFTI5Q75JTSTH3DQU5PQVANCNFSM4JQJ2VPA].[https://github.com/notifications/beacon/AAP33TIN65XF63H4HA5VTGLQU5PQVA5CNFSM4JQJ2VPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEE4QWJY.gif]

johnfactotum commented 4 years ago

Not really related to text selection, but it'd be great if you can test out the master branch and see if page turning and controls work on a touchscreen. They use mouse events, but I think WebKit should also trigger mouse events when tapping on a touchscreen.

Tapping on the left third of the page should navigate you to the previous page, and the right third to the next page. The middle third should toggle the headerbar and navbar, like this figure shows: image

JamesSwift commented 4 years ago

Hi, sorry for the delay. We've just had a baby which is making it difficult to get much done (I'm literally typing this while holding and feeding the baby with one hand). I'll try to test it soon. If you could provide an appimage, rpm or deb that would speed me along lots. But in either case I'll get to it then report back eventually. :-)

JamesSwift commented 4 years ago

Hi, I'm trying to compile the master branch to test it on my tablet, but I get this error:

james@acer3:~/foliate$ meson build --prefix=/usr
The Meson build system
Version: 0.51.2
Source dir: /home/james/foliate
Build dir: /home/james/foliate/build
Build type: native build
Project name: foliate
Project version: 2.0.0
Build machine cpu family: x86_64
Build machine cpu: x86_64
Program desktop-file-validate found: YES (/usr/bin/desktop-file-validate)
Program appstream-util found: YES (/usr/bin/appstream-util)
Program glib-compile-schemas found: YES (/usr/bin/glib-compile-schemas)
Found pkg-config: /usr/bin/pkg-config (0.29.1)
WARNING: Could not detect glib version, assuming 2.54. You may get build errors if your glib is older.
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonmain.py", line 127, in run
    return options.run_func(options)
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 241, in run
    app.generate()
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 159, in generate
    self._generate(env)
  File "/usr/lib/python3/dist-packages/mesonbuild/msetup.py", line 188, in _generate
    intr.run()
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreter.py", line 3924, in run
    super().run()
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 412, in run
    self.evaluate_codeblock(self.ast, start=1)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 436, in evaluate_codeblock
    raise e
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 430, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 441, in evaluate_statement
    return self.function_call(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 778, in function_call
    return func(node, posargs, kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 285, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 174, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreter.py", line 3465, in func_subdir
    self.evaluate_codeblock(codeblock)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 436, in evaluate_codeblock
    raise e
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 430, in evaluate_codeblock
    self.evaluate_statement(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 443, in evaluate_statement
    return self.assignment(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 1053, in assignment
    value = self.evaluate_statement(node.value)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 445, in evaluate_statement
    return self.method_call(cur)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 815, in method_call
    return obj.method_call(method_name, args, kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreter.py", line 1712, in method_call
    value = fn(state, args, kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 285, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/interpreterbase.py", line 174, in wrapped
    return f(*wrapped_args, **wrapped_kwargs)
  File "/usr/lib/python3/dist-packages/mesonbuild/modules/gnome.py", line 162, in compile_resources
    state, ifile, source_dirs, dependencies)
  File "/usr/lib/python3/dist-packages/mesonbuild/modules/gnome.py", line 245, in _get_gresource_dependencies
    pc, stdout, stderr = Popen_safe(cmd, cwd=state.environment.get_source_dir())
  File "/usr/lib/python3/dist-packages/mesonbuild/mesonlib.py", line 967, in Popen_safe
    stdout=stdout, stderr=stderr, **kwargs)
  File "/usr/lib/python3.7/subprocess.py", line 800, in __init__
    restore_signals, start_new_session)
  File "/usr/lib/python3.7/subprocess.py", line 1551, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'glib-compile-resources': 'glib-compile-resources'
itprojects commented 4 years ago

What is your distribution you're running this on (acer3)?

It says in the final line:

FileNotFoundError: [Errno 2] No such file or directory: 'glib-compile-resources': 'glib-compile-resources'

Do you have a package libglib2.0-dev? If not, install that and repeat the instructions.

JamesSwift commented 4 years ago

Hi, I'm testing this on Pop OS. You were right, I was missing libglib2.0-dev as well as gir1.2-webkit2-4.0 (which took some finding). I have compiled and tested it now.

You are correct, tapping the screen as per your diagram triggers the correct action. Swiping left or right on any area of the screen also works as expected. Good job!

There is still no way to highlight text with the touchscreen (as expected), as any tapping action or tapping and swiping action triggers a page turn.

johnfactotum commented 4 years ago

Thanks for testing this!

Interesting that swiping works. Foliate doesn't handle any touch events, so that means currently any swiping gesture is registered as a (mouse) wheel event. There's really nothing we can do about this. Will need to ask the WebKitGTK devs and/or open a bug report with them.

touchscreenuser commented 4 years ago

Hi John, thanks for making Foliate!

I'm using Surface Go with current Arch Linux and Cinnamon desktop running Foliate 1.5.3.

In touchscreen mode I made the following observations: a) when using a finger for touching the screen selection seems to work when double tapping. But it's not reliable. Often the result is just a page turn as James already stated before. It seems to be necessary to hit a letter when double tapping and maybe it depends on double tapping speed. b) when using Surface pen selection by double tapping works flawlessly but with a pen it's not possible to make a page turn (or lets say I didn't find a way besides using the icons in status bar).

I hope I don't capture the issue but I have another problem/feature request related to touchscreen use: When detaching the keyboard and going in fullscreen mode it's not possible to tap search and leave fullscreen icon because the top bar is hidden and keyboard shortcuts for obvious reasons don't work. It's necessary to re-attach keyboard in order to leave Foliate. Is it possible to make an option to display top bar in fullscreen mode or (in order to save more space on the small 10 inch display) to add these buttons to statusbar in fullscreen mode?

Of course I can help testing as well ...

Edit: I tried again with very high font size value. It's not necessary to hit the letter. It's just necessary to take care that the taps are not interpreted as swipe. The three areas you described above are not working with single tap for me (as long as tap is not interpreted as swipe). Controls in status bar are working with taps flawlessly. Try to check with master branch version as well ...

touchscreenuser commented 4 years ago

Tested master branch now (sorry, should have done it in the first place). All three areas are working with single tap (finger and pen). The middle tap solves my problem to leave full screen as well :-) Swipe works and all controls (finger and pen). Selecting text by double tapping with finger is still an issue but it is a lot more reliable than in version 1.5.3. Great work!

JamesSwift commented 4 years ago

Hi @johnfactotum, Great work on version 2! I'm just checking in again to see if there is any progress on touchscreen support for selecting/highlighting? In testing on the flatpak version, I can double tap to select a word but there is currently no way to extend the selection to more than one word. Are we still waiting on webkitgtk support?

johnfactotum commented 4 years ago

Yeah, WebKitGTK needs to implement selection handles. Although, coming to think of it, I think it might be possible to make our own selection handles.

JamesSwift commented 4 years ago

Ohh, that would be a good stop gap.

On Sun, 17 May, 2020 at 4:16 am, John Factotum notifications@github.com wrote:

Yeah, WebKitGTK needs to implement selection handles. Although, coming to think of it, I think it might be possible to make our own selection handles.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/johnfactotum/foliate/issues/207#issuecomment-629780763, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAP33TN6UK3OE7QZTQEI2T3RR7BQPANCNFSM4JQJ2VPA.

JamesSwift commented 4 years ago

I've opened an issue on WebKitGTK to ask about touch handles. We'll see what happens. https://bugs.webkit.org/show_bug.cgi?id=212048

hashworks commented 4 years ago

While double tab works it's quite fiddly to select a word. It would be great if the area could be increased. Long press instead of double tap (like with a Kindle) would be great as well, especially since there is an option to move to the next page with a single tap.

rik-shaw commented 4 years ago

@johnfactotum could the community help fund a pinephone for you as a test device? I know it is relatively slow, but with its broad development interest I think it would be worthwhile to test foliate against it to help identify issues that could help steer development to be optimized for lower power touchscreen devices. I would be happy to contribute to you for this, as I suspect many others would be.

johnfactotum commented 1 year ago

Just a quick note on how to implement selection handles (which can be helpful for mouse users, too, for tweaking the selection range without having to reselect). It's actually not too difficult.

  1. You'd need something to drag, either in GTK or the WebView.
  2. With the coordinates of this handle, adjusting it so that it's relative to the iframe, use Document.caretRangeFromPoint() (or Document.caretPositionFromPoint()) to get the node and offset.
  3. Update the selection range with the node and offset.

Now, for long press selection, I think you'd do something similar to get a collapsed range from a point, which you can then add to selection and extend it with Selection.modify().