not smooth input with MacOS - Wacom Pen not detected as Input System

[birba]

Setup:

• tablet/pen: Wacom Intuos M BT
• OS: MacOX Catalina 1.15.7 (MacBook Pro 2020)
• Version of Xournal++: 1.0.18
• Installation method: downloaded the ``xournalpp-1.0.18-macos.zip'' (from https://github.com/xournalpp/xournalpp/releases), the system automatically recognise it as an .app which runs. No need to unzip or install at all.

Description of the problem: The strokes are ugly (not smooth, with ''edges'') and there is a notable lag. If I run the app and open: Edit->Preferences->InputSystems I see the message: ''No devices were found. This seems wrong - maybe file a bug report?''

Remark 1) The same Wacom pen on the same laptop writes smoothly on OneNote (terrible software, though). Remark 2)I have Xournal++ on fedora in my office and with the same tablet it is 10times smoother with a machine which is half as powerful as the MacBook. So I think there is a real issue. Remark 3) it looks exactly as this issue: (https://github.com/xournalpp/xournalpp/issues/1531). Also the pictures are the same. The links to the ''these/those'' are broken.

Attempted solution: Download the full ''source code tar.gz'' and try a manual build following this guide: https://github.com/xournalpp/xournalpp/blob/master/readme/MacBuild.md (I did not have installed Homebrew - whatever that is). Took a lot of time, got around 5 errors which are very far from my level of expertise. Most of the commands ended up with ''correct'' or something like that. The following commands did not ended up with ``correct'': ./build-poppler.sh and ./build-portaudio.sh.

Excpeted behaviour: I expect the Xournal++ for Mac to run as smoothly as the Xournal++ I have in the Fedora in the office.

Screenshots of Problem The same screenshots as https://github.com/xournalpp/xournalpp/issues/1531.

[fredowski]

I could fix the detection problem by using the gtk version 3.24.23 which I installed via macports. The "smooth" problem is still there but the strange thing is that when I turn the Wacom pen and use the eraser and then turn back, then it is smooth. Very strange. Behaviour did not change when I chose Eraser via the menu, i.e. I had to turn the pen.

[fredowski]

Hi @birba,

could you maybe try the fork with my changes in the mac-setup directory

https://github.com/fredowski/xournalpp/tree/macapp

to see if you can create the build environment? This takes quite long.

[mschinca]

@fredowski I'm also experiencing the "smooth" issue. I'm using your fork. I'm not using a tablet so I think the lag is unrelated to the input method. I noticed that the lag is proportional to the application window size and is independent from the zoom level. I took a screen capture to show the behavior. I the problem is noticeable only on a high DPI screen (e.g. Mac retina 16) and not on an older screen. Trying to adjust DPI settings didn't help

[rolandlo]

I'm not using a tablet so I think the lag is unrelated to the input method. I noticed that the lag is proportional to the application window size and is independent from the zoom level. I took a screen capture to show the behavior.

I have made the same observations with the latest master (with the updated dependencies).

[fredowski]

Do you also see the behaviour that the lag is gone if you turn the pen, do something with the eraser and turn back?

[rolandlo]

I haven't observed that yet with my 3D Pen (which doesn't have an eraser on the bottom). I have another pen (which has an eraser at the bottom), and will try it, once I can find it.

[mschinca]

Not sure if relevant, but I had the chance to try 1.0.19 and it looks very smooth instead on same hardware. So maybe some performance regression introduced later?

[rolandlo]

Version 1.0.19 looks much smoother indeed. It's especially visible when drawing filled shapes. The question is, whether the performance regression comes from something in Xournal++ or from updated dependencies.

[rolandlo]

Since the lag depends very much on the window size, it might be an issue with the rendering. Maybe lots of rerenders. We will have to investigate this.

[mschinca]

Version 1.0.19 looks much smoother indeed. It's especially visible when drawing filled shapes. The question is, whether the performance regression comes from something in Xournal++ or from updated dependencies.

I suspect it may be the latter. I've just built tag 1.0.19 with the brew method. The drawing is as laggy as latest master. I'm now trying to build the same tag with the bundled gtk+ for comparison.

[mschinca]

I managed to build 1.0.19 with the jhbuild method - with some obstacles due to unpinned dependencies. The drawing was still laggy. I then noticed that the gtk+3 version pulled by gtk-osx-setup.sh was not the same as the one shown by the 'released' 1.0.19

I'm not sure how to build an earlier gtk+3 so I could not confirm that the gtk version is what makes the difference. At this stage it is probably better to investigate the issue on the latest version anyway.

[fredowski]

Hi, the idea of the changing the gtk version was because the wacom pen was not detected: https://github.com/xournalpp/xournalpp/issues/1772#issuecomment-723540523

[rolandlo]

It's no option to go back to GTK version 3.24.10 as pointed out by @fredowski . It might be an option to use a more recent GTK version though. Currently we get 3.24.23 because we use the stable moduleset and the version is hardcoded in

https://gitlab.gnome.org/GNOME/gtk-osx/-/blob/master/modulesets-stable/gtk-osx.modules#L326

We might try out the unstable moduleset, which would currently use GTK version 3.24.28 as seen from

https://gitlab.gnome.org/GNOME/gtk-osx/-/blob/master/modulesets-unstable/gtk-osx.modules#L252

and https://download.gnome.org/sources/gtk+/3.24/

[fredowski]

Yes sure - it is not an option to go back to 3.24.10. My guess is that there are maybe not too many applications which use pen support with gtk on MacOS. Maybe Xournal is the only one? This is the history of the gdk/quartz section in gtk: https://github.com/GNOME/gtk/commits/gtk-3-24/gdk/quartz . Not too much happens there. Maybe it is possible to disect the problem or is the problem the updated pen support https://github.com/GNOME/gtk/commit/ecbed15d47fc29fc0da133c255fb7e4cb62e4129#diff-779883cae9c0feee3d7d2e781b4255a305a14cab7e8c227eb077f32eaec9f8d6 which was the change that fixes the pen detection?

[rolandlo]

Seems I finally got a Xournal++ build for MacOS without the annoying lag on BigSur. All I had to do was to follow the instructions on the wiki page and in the end upgrade Gtk+-3.0 from version 3.24.23 to version 3.24.28 by executing

jhbuild buildone gtk+-3.0

followed by the usual complete-build.sh. The former downloads the latest Gtk+ 3 version and builds it.

Note that I did the build on MacOS Catalina and it works fine there as well. Maybe we can also upgrade Cairo like this (from version 1.16 to 1.17). This might solve some problems with PDF exports.

[rolandlo]

@birba @mschinca @fredowski if you want to try the non-laggy build of Xournal++ on BigSur, I can upload it somewhere for testing purposes. Just let me know.

[mschinca]

@rolandlo great news, sure I'd be happy to test it

[rolandlo]

@mschinca Ok, please download from this link. Password is Xpp and it's valid for 7 days. You need to unzip it after download.

[fredowski]

I tried on MacOS 10.13.6 but there it says:

You compiled on Catalina, right?

[rolandlo]

Yes, I compiled on Catalina. I tested it on both Catalina and BigSur. High Sierra is not supported anymore, see #2888.

[mschinca]

@mschinca Ok, please download from this link. Password is Xpp and it's valid for 7 days. You need to unzip it after download.

@rolandlo I managed to try your build. All in all it looks very smooth so far, no more input lag, which is great news :) My environment is macOS Big Sur 11.2.3

Just a side note on the package, after unzipping I needed to give execution permission in order to launch the program from Finder, like so:

chmod a+x Xournal++.app/Contents/MacOS/xournalpp*

[rolandlo]

@rolandlo I managed to try your build. All in all it looks very smooth so far, no more input lag, which is great news :)

Wonderful! I am glad to hear that. @Febbe will work on updating the binary blob in our Azure pipeline, so that all future versions released on the Github page will use the updated Gtk version (that fixes the lag).

[pegonma]

@rolandlo would you mind making another link to your build? I would like try it too. Thanks!

[rolandlo]

@pegonma Here is another link, valid for another 7 days, same password (Xpp).

[BeaconOnAChairMC]

@rolandlo I keep seeing on my mac "you do not have permission to open this file"

[rolandlo]

@BeaconOnAChairMC So you can't download it? Have you entered the password correctly? Or could you download and unzip it and fail to open it in the finder?

[BeaconOnAChairMC]

@rolandlo nevermind, I downloaded it but couldn't open, but I got it opened. Hmm, seems I am getting input lag on this build as well, but it might be my computer itself, I can see distinct "jagged" curves, even when using trackpad/mouse to draw, hmm

oh well, thanks anyway!

[pegonma]

@rolandlo thank you very much! Sadly, I cannot launch it, even after setting xournalpp as executable. It is likely due to the fact that I am running a way outdated version of MacOS (10.12!). I wonder if it would be too much trouble for you to rebuild it with 10.12 as min requirement...?

[rolandlo]

@pegonma I'm sorry, but rebuilding it with 1.0.12 would be a huge effort.

[rolandlo]

@lupja Here is another link, valid another 7 days, same password Xpp.

[lupja]

@lupja Here is another link, valid another 7 days, same password Xpp.

I managed to open the file. Unfortunately, the issue still persists.

[rolandlo]

I managed to open the file. Unfortunately, the issue still persists.

Strange. Several people have reported that the lag issue is solved. So you observe no difference with respect to the lag at all?

[mschinca]

I took a screen recording on Catalina recent Mac Pro and GTK 3.24.28. I noticed some rare glitches, but overall the strokes are fluid and generally speaking it is a big improvement compared to old version , so I believe GTK 3.24.28 should be used for latest builds.

[lorenzmant]

@lupja Here is another link, valid another 7 days, same password Xpp.

I have tried out Rolandlo's latest build, and I confirm that I'm experiencing a quite evident non-smooth input. I don't think I'm adding anything to what has already been observed so far. Here's what I notice.

a. There's a bit of latency between the input and the displaying of traits. Not terrible but not great for handwriting. b. The app tends to render "curved traits" with "polygonal traits". When I input lines via trackpad the phenomenon is not so annoying, although present. When I draw via an Intuos M BT the phenomenon is way more evident and way more annoying.

Overall I think I am getting something similar to what mschinca shows in his video. If I draw small signs, as mschinca does, I get a similar result. If I draw at the same time more quickly and bigger pictures, the lag and polygonal shape of traits become definitely more evident.

I can provide screenshots/screen recordings if needed; I can also make some specific tests if you think it would be helpful. For the record: I'm on a Macbook Pro M1 under OS 11.2.2.

[rolandlo]

It seems the lag is related to the icons rendering. Can you please try to use the empty toolbar and see if the lag is reduced?

[rolandlo]

Even simpler: Hiding the toolbars (with F9) makes the lag go away. Showing them again (with F9) makes the lag come back again. Can anyone confirm these observations (on BigSur)? @mschinca @fredowski @lorenzmant @lupja

[lorenzmant]

Even simpler: Hiding the toolbars (with F9) makes the lag go away. Showing them again (with F9) makes the lag come back again. Can anyone confirm these observations (on BigSur)? @mschinca @fredowski @lorenzmant @lupja

I've tried to do what you suggested using your version (built on May 2nd at 15:52 with gtk 3.24.28) on OS 11.2.2 on a macbook pro M1. Is this ok? or you compiled a newer version?

With the given version, I don't see any difference between writing with or without toolbar. When I draw with small traits there isn't much latency, and traits are ok. But when I'm drawing large traits the latency increases and the software translates a smooth curved line to a polygonal line. The phenomenon seems to be proportional to the writing speed: the quicker the trait is, the worse the quality of traits.

Ah, is it expected that when I'm using your version of Xournal++, on the menu bar of MacOS there are only two menus? I can only see the apple menu (which is always there), and one single Xournal++ menu. All Xournal's menus are on Xournal++'s window. On the other hand when i use Xournal 1.0.16 the menus aren't in Xournal++ window, but rather on macOS menu bar. May this be connected to the lag?

[rolandlo]

@lorenzmant You are supposed to use a more recent 1.1.0+dev version, not older than 2021, June 16. Ideally try the latest nightly release.

As for the menus: We have removed the Gtk-Mac-Integration. That is probably not related to the lag.

[lorenzmant]

@lorenzmant You are supposed to use a more recent 1.1.0+dev version, not older than 2021, June 16. Ideally try the latest nightly release.

As for the menus: We have removed the Gtk-Mac-Integration. That is probably not related to the lag.

Right, so I retried with the version you suggested, but my comments remain the same. No real difference with or without toolbars.

[rolandlo]

Right, so I retried with the version you suggested, but my comments remain the same. No real difference with or without toolbars.

Thanks for testing. So this issue is even more complicated.

[Febbe]

Currently, Stroke::calcSize() eats 14% of cpu time.

[Febbe]

as seen here: XournalppCursor::updateCursor eats the most CPU time in the input handling on ubuntu linux.

[Febbe]

Can someone of you, who does not experience the toolbar issue, test #3236.

[rolandlo]

Reading a bit about GTK+ performance on BigSur I found the following two sources, which show that Xournal++ is experiencing the same problems as other GTK+ apps (in particular GIMP): GIMP issue GTK issue

As I understood GIMP managed to solve the problem (with a patched GTK+ version?), which was related to color space conversion and full redraws after many events. See these two posts:

first post second post

In particular the observation with full redraws triggered by motion events look relevant.

[ruizbaier]

@lupja Here is another link, valid another 7 days, same password Xpp.

@rolandlo Could you please share the link of that build again so I can try? with my current build I still see the lag.

[rolandlo]

@rolandlo Could you please share the link of that build again so I can try? with my current build I still see the lag.

Here it is (same password Xpp)

[ruizbaier]

@rolandlo Could you please share the link of that build again so I can try? with my current build I still see the lag.

Here it is (same password Xpp)

Thank you!!

[rolandlo]

Please check out the latest nightly release. The lag should be greatly reduced there (both in MacOS Catlina and MacOS BigSur), at least when using the Minimal Left toolbar or when the toolbar is hidden.

[mschinca]

@rolandlo I gave it a shot on Big Sur, Mac Pro 16inch 2019. I think it is getting better, although I can still notice a few lags every now and then when the window is maximized. I made a screen recording

[lorenzmant]

@rolandlo I have given a try to the last nightly build, downloaded from your link, and I had a bit of a surprise in trying it out. Here are some thoughts.

System: MacbookPro 13inch M1, BigSur 11.2.2 Xournal++ version: 1.1.0, built on Jul30 2021 at 00:08:34, Git commit: 97c20f62, libgtk: 3.24.28

1. Regarding the lag: -Very high, no difference with previous versions, on internal display -Very low on external display, no difference with previous version (Nightly build of July 16th) Basically nothing changed in my point of view from version of Jul16th to the last nightly build. What changed was that I tried to use the software on an external display.

2. Regarding the other issues that I see: -High cpu use on both displays -Scrolling the sheet most of the times does not work -The traits are still kind of polygonal, and handwriting looks like "trembling"

3. I have made a screen capture with some comments, available on request. (I'm a bit shy in posting a link with myself saying nonsense)