[Bug]: SPen button on Samsung devices won't be detected

marcel2789 commented 2 years ago

Is your feature request related to a problem? Please describe

When I'm using the preinstalled Samsung Notes app I can use the SPen button to temporarily switch to the eraser. I would like this feature here too.

Describe your feature request!

The setting would allow to choose which tool should be used when the SPen button is held down.

Additional context

No response

Code of Conduct

[X] I agree to follow this project's Code of Conduct

CodeDoctorDE commented 2 years ago

This should already be possible. Can you give me the output when using the button here: https://butterfly-git-debug-stylus-buttons-linwood.vercel.app/? I added a debug dialog that should show the buttons that were pressed

marcel2789 commented 2 years ago

I think something weird is going on. When I'm drawing normally without any button pressed I get {1}. But when I hold the button down and draw I get no dialog box. It's the same when I try to press the button while the pen is hovering over the display. I'm using the Samsung Galaxy Tab s6 lite if that helps.

CodeDoctorDE commented 2 years ago

The dialog gets opened when using "onPointerDown". Can you try pressing the button before touching the display?

marcel2789 commented 2 years ago

I tried that too but the dialog box didn't appear. I also don't know if it would be a problem with the SPen itself because a lot of third party note taking apps also don't accept the button input.

CodeDoctorDE commented 2 years ago

A similar problem was reported on my discord/matrix server. I try a new method and will give you a link. I have a different idea

marcel2789 commented 2 years ago

Came back to this problem. Made a little local website today to test the pointer events of chrome. Turns out chrome doesn't even capture the pen with the button pressed. It can detect touch, mouse and pen input just fine but not with the button pressed. So I'll close the issue since the issue lays in Chromium and/or flutter and not in this app.

Flutter Github Issue

CodeDoctorDE commented 2 years ago

I would keep it open for people having the same issues. I put them in a different label platform-issue to difference it

raffaem commented 11 months ago

To some extent it works in Saber.

Touching the screen with the pen, and pressing the Pen button while touching the screen, will turn the eraser on. See link (confirmed personally on my device).

So, to this extent, is a Butterfly problem, not a general flutter problem.

CodeDoctorDE commented 11 months ago

Touching the screen with the pen, and pressing the Pen button while touching the screen, will turn the eraser on. See link (confirmed personally on my device).

That happens in Saber? I will try to fix that, it's a bit complicated because I don't have a stylus for my phone. I will try to reproduce this on web

markwmuller commented 9 months ago

FWIW, I think this is the commit on Saber where the workaround is introduced: https://github.com/saber-notes/saber/pull/783

CodeDoctorDE commented 9 months ago

Oh, this could be the solution. I didn't check for the inverted stylus input type. I will make a branch for this and it would be nice if anyone can test it

CodeDoctorDE commented 9 months ago

I created a branch to test if this solves the issue. Can you try out this apk after it was built successfully: https://github.com/LinwoodDev/Butterfly/actions/runs/7528706756 (go to artifact and download the apk build. you need to be logged in with github)

markwmuller commented 9 months ago

I've tried it, but it doesn't work as I expect it. To confirm, after installing the APK I have Current Version 2.0.2-rc.0, and the pen writes as usual. When I start writing, and then push the stylus button, nothing changes (the pen keeps writing). The saber behavior would be that the pen starts writing a line, and when I push the button it switches to eraser mode. I didn't change anything in the settings, if that matters.

CodeDoctorDE commented 9 months ago

Thanks for testing, I have currently no spen capable phone. When pressing it, it will go to the second tool starting from left. Can you try setting the eraser at the second index and the pen on the first? If this is already the case or it doesn't solve it, feel free to answer here, then I need to find another way

markwmuller commented 9 months ago

OK, so I've removed the "select" tool that's usually the first on the list of tools at the bottom of the writing screen. Now I have Pen; Path eraser; Undo; Redo; etc.

I can't get the behavior now either. Nothing apparent happens when I push the eraser button.

markwmuller commented 9 months ago

Let me know if there's something I can try to debug on my end. Not sure when I'll have the time, but could try somewhere (program seems overall very nice!). I am able to compile from source, but can't figure out the language to hack anything (wanted to see if I can register the button press, but the code is greek to me...)

CodeDoctorDE commented 9 months ago

Yeah, debugging the app would help me a lot. Can you try adding a breakpoint in https://github.com/LinwoodDev/Butterfly/blob/c7c2fd77e063ab12d5f9b237004f1a855158399f/app/lib/views/view.dart#L304C45-L304C45 and give me the event properties when pressing the spen button and touching the screen?

markwmuller commented 9 months ago

OK, so I'm pretty hapless. I don't know how to set a breakpoint, but I can run the app via command line, to see the output there. Maybe that helps?

Set up: git status: Your branch is up to date with 'origin/feature/inverted-stylus'.

Then build as flutter build apk --debug --flavor nightly, finishes with green checkmark

Run as flutter -v -d XCXXXXX run --flavor 'nightly'

On my phone wait for app to launch. Then open a new doc, then do the below actions. The timestamp should point you to the log file parts:

[11960] select pen tool by and, .
[10523] write a line with stylus, without pushing the button
[19175] write a line with stylus, and push the button halfway through
[13052] hover pen over phone, push button, and then push tip down and draw a line.

I kill flutter with ctrl+c

In short, there is no text output that seems to indicate anything with the button being down; nothing is printed mid-line when I push the button.

Log output below:


[ +440 ms] --------- beginning of main
                    01-16 10:48:56.828 I/SLPASVC (24107): completed get profile info list                                                                     [at
                    com.samsung.euicc.service.EuiccServiceImpl.onGetEuiccProfileInfoList(EuiccServiceImpl.java:115)]
[  +22 ms] executing: /home/mark/code/Android/SDK/platform-tools/adb -s R5CR81KXWLY shell am start -a android.intent.action.MAIN -c android.intent.category.LAUNCHER -f 0x20000000 --ez enable-dart-profiling true
--ez enable-checked-mode true --ez verify-entry-points true dev.linwood.butterfly.nightly/dev.linwood.butterfly.MainActivity
[  +91 ms] Starting: Intent { act=android.intent.action.MAIN cat=[android.intent.category.LAUNCHER] flg=0x20000000 cmp=dev.linwood.butterfly.nightly/dev.linwood.butterfly.MainActivity (has extras) }
[        ] Waiting for VM Service port to be available...
[ +375 ms] VM Service URL on device: http://127.0.0.1:43127/ozKbfwSzPsc=/
[   +1 ms] executing: /home/mark/code/Android/SDK/platform-tools/adb -s R5CR81KXWLY forward tcp:0 tcp:43127
[  +18 ms] 37021
[        ] Forwarded host port 37021 to device port 43127 for VM Service
[   +3 ms] Caching compiled dill
[  +63 ms] Connecting to service protocol: http://127.0.0.1:37021/ozKbfwSzPsc=/
[ +171 ms] Launching a Dart Developer Service (DDS) instance at http://127.0.0.1:0, connecting to VM service at http://127.0.0.1:37021/ozKbfwSzPsc=/.
[ +130 ms] DDS is listening at http://127.0.0.1:33839/sWvxn3uie4U=/.
[  +39 ms] Successfully connected to service protocol: http://127.0.0.1:37021/ozKbfwSzPsc=/
[  +96 ms] DevFS: Creating new filesystem on the device (null)
[  +17 ms] DevFS: Created new filesystem on the device (file:///data/user/0/dev.linwood.butterfly.nightly/code_cache/appKDPOEZ/app/)
[   +1 ms] Updating assets
[ +125 ms] Manifest contained wildcard assets. Inserting missing file into build graph to force rerun. for more information see #56466.
[   +2 ms] Syncing files to device SM N986U1...
[   +1 ms] Compiling dart to kernel with 0 updated files
[        ] Processing bundle.
[        ] <- recompile package:butterfly/main.dart a23550fc-ddd2-434a-82a6-a418c5ffbb43
[        ] <- a23550fc-ddd2-434a-82a6-a418c5ffbb43
[   +3 ms] Bundle processing done.
[ +110 ms] Updating files.
[        ] DevFS: Sync finished
[        ] Syncing files to device SM N986U1... (completed in 115ms)
[        ] Synced 0.0MB.
[        ] <- accept
[   +3 ms] Connected to _flutterView/0xb40000739a0c2800.
[   +1 ms] Flutter run key commands.
[        ] r Hot reload. 🔥🔥🔥
[        ] R Hot restart.
[        ] h List all available interactive commands.
[        ] d Detach (terminate "flutter run" but leave application running).
[        ] c Clear the screen
[        ] q Quit (terminate the application on the device).
[        ] A Dart VM Service on SM N986U1 is available at: http://127.0.0.1:33839/sWvxn3uie4U=/
[  +90 ms] I/flutter (21288): dynamic_color: Core palette detected.
[  +50 ms] The Flutter DevTools debugger and profiler on SM N986U1 is available at: http://127.0.0.1:9100?uri=http://127.0.0.1:33839/sWvxn3uie4U=/
[+1015 ms] I/BLASTBufferQueue(21288): [SurfaceView[dev.linwood.butterfly.nightly/dev.linwood.butterfly.MainActivity]@0#1](f:0,a:0) onFrameAvailable the first frame is available
[        ] D/SurfaceView@acc989d(21288):  setAlpha: mUseAlpha = false alpha=1.0
[        ] D/SurfaceView@acc989d(21288):  updateSurfaceAlpha: setUseAlpha() is not called, ignored.
[   +4 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): Setup new sync id=0
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): Setting syncFrameCallback
[   +1 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): registerCallbacksForSync syncBuffer=false
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): mWNT: t=0xb40000731a0b0a70 mBlastBufferQueue=0xb40000735a0b2b10 fn= 1 caller=
android.view.ViewRootImpl.lambda$applyTransactionOnDraw$11$android-view-ViewRootImpl:12865 android.view.ViewRootImpl$$ExternalSyntheticLambda1.onFrameDraw:4
android.graphics.HardwareRenderer$FrameDrawingCallback.onFrameDraw:924 
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): Received frameDrawingCallback syncResult=0 frameNum=1.
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): Setting up sync and frameCommitCallback
[        ] D/SurfaceView@acc989d(21288): updateSurfacePosition RenderWorker, frameNr = 1, position = [0, 0, 1440, 3035] surfaceSize = 1440x3035
[        ] I/SurfaceView@acc989d(21288): uSP: rtp = Rect(0, 0 - 1440, 3035) rtsw = 1440 rtsh = 3035
[        ] I/SurfaceView@acc989d(21288): onSSPAndSRT: pl = 0 pt = 0 sx = 1.0 sy = 1.0
[        ] I/SurfaceView@acc989d(21288): aOrMT: ViewRootImpl@4d5a037[MainActivity] t = android.view.SurfaceControl$Transaction@3153c34 fN = 1 android.view.SurfaceView.-$$Nest$mapplyOrMergeTransaction:0
android.view.SurfaceView$SurfaceViewPositionUpdateListener.positionChanged:1537 android.graphics.RenderNode$CompositePositionUpdateListener.positionChanged:373 
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): mWNT: t=0xb40000731a0af090 mBlastBufferQueue=0xb40000735a0b2b10 fn= 1 caller= android.view.SurfaceView.applyOrMergeTransaction:1455
android.view.SurfaceView.-$$Nest$mapplyOrMergeTransaction:0 android.view.SurfaceView$SurfaceViewPositionUpdateListener.positionChanged:1537 
[        ] I/BLASTBufferQueue(21288): [ViewRootImpl@4d5a037[MainActivity]#0](f:0,a:0) onFrameAvailable the first frame is available
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): Received frameCommittedCallback lastAttemptedDrawFrameNum=1 didProduceBuffer=true
[        ] D/OpenGLRenderer(21288): CFMS:: SetUp Pid : 21288    Tid : 21311
[        ] W/Parcel  (21288): Expecting binder but got null!
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): onSyncComplete
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): setupSync seqId=0 mSyncId=0 fn=1 caller=android.view.ViewRootImpl$$ExternalSyntheticLambda11.accept:6
android.window.SurfaceSyncer.lambda$setupSync$1$android-window-SurfaceSyncer:128 android.window.SurfaceSyncer$$ExternalSyntheticLambda1.accept:8 android.window.SurfaceSyncer$SyncSet.checkIfSyncIsComplete:382
android.window.SurfaceSyncer$SyncSet.markSyncReady:359 android.window.SurfaceSyncer.markSyncReady:151 android.view.ViewRootImpl.performTraversals:4503 
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): reportDrawFinished seqId=0 mSyncId=-1 fn=1 mSurfaceChangedTransaction=0xb40000731a0a63f0
[  +15 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): MSG_WINDOW_FOCUS_CHANGED 1 0
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): mThreadedRenderer.initializeIfNeeded()#2 mSurface={isValid=true 0xb4000073ea0b9d70}
[        ] D/InputMethodManager(21288): startInputInner - Id : 0
[        ] I/InputMethodManager(21288): startInputInner - mService.startInputOrWindowGainedFocus
[   +3 ms] D/InputMethodManager(21288): startInputInner - Id : 0
[+5551 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 0
[  +41 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 1
[ +114 ms] W/OnBackInvokedCallback(21288): OnBackInvokedCallback is not enabled for the application.
[        ] W/OnBackInvokedCallback(21288): Set 'android:enableOnBackInvokedCallback="true"' in the application manifest.
[+11960 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 0
[  +26 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 1
[+10523 ms] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[   +1 ms] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[ +311 ms] I/[DWTE]  (21288): onPenEvent no editText
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 0
[        ] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[        ] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[ +543 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 1
[        ] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[        ] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[+19175 ms] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[   +1 ms] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[ +154 ms] I/[DWTE]  (21288): onPenEvent no editText
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 0
[        ] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[        ] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[+1380 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 1
[        ] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[        ] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[+13052 ms] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[        ] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[ +310 ms] I/[DWTE]  (21288): onPenEvent no editText
[        ] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 0
[        ] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[        ] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[ +690 ms] I/ViewRootImpl@4d5a037[MainActivity](21288): ViewPostIme pointer 1
[        ] I/ViewRootImpl(21288): updatePointerIcon pointerType = 20001, calling pid = 21288
[        ] D/InputManager(21288): setPointerIconType iconId = 20001, callingPid = 21288
[+18227 ms] Application finished.
[        ] DevFS: Deleting filesystem on the device (file:///data/user/0/dev.linwood.butterfly.nightly/code_cache/appKDPOEZ/app/)
[  +11 ms] the Dart compiler exited unexpectedly.
[        ] 
           #0      throwToolExit (package:flutter_tools/src/base/common.dart:10:3)
           #1      DefaultResidentCompiler._compile.<anonymous closure> (package:flutter_tools/src/compile.dart:874:9)
           #2      _rootRunUnary (dart:async/zone.dart:1407:47)
           #3      _CustomZone.runUnary (dart:async/zone.dart:1308:19)
           #4      Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:846:45)
           #5      Future._propagateToListeners (dart:async/future_impl.dart:875:13)
           #6      Future._completeWithValue (dart:async/future_impl.dart:647:5)
           #7      Future._asyncCompleteWithValue.<anonymous closure> (dart:async/future_impl.dart:721:7)
           #8      _rootRun (dart:async/zone.dart:1399:13)
           #9      _CustomZone.run (dart:async/zone.dart:1301:19)
           #10     _CustomZone.runGuarded (dart:async/zone.dart:1209:7)
           #11     _CustomZone.bindCallbackGuarded.<anonymous closure> (dart:async/zone.dart:1249:23)
           #12     _microtaskLoop (dart:async/schedule_microtask.dart:40:21)
           #13     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49:5)
           #14     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:118:13)
           #15     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:185:5)

[        ] DevFS: Deleted filesystem on the device (file:///data/user/0/dev.linwood.butterfly.nightly/code_cache/appKDPOEZ/app/)
[        ] "flutter run" took 91,000ms.
[  +25 ms] ensureAnalyticsSent: 39ms
[        ] Running 1 shutdown hook
[        ] ensureAnalyticsSent: 27ms
[        ] Running 1 shutdown hook
[        ] Shutdown hooks complete
[        ] Shutdown hooks complete
[        ] exiting with code 0

CodeDoctorDE commented 9 months ago

Sadly the log files doesn't help. Instead of breakpoints you could add print statements there:

...
print("DEBUG INPUT: ${event.kind}, ${event.buttons}");

This would help me a lot figuring out whats wrong

markwmuller commented 9 months ago

OK, added that just prior to _isScalingDisabled line. Below are three strokes:

21529 stylus stroke, button not touched
5189 stylus stroke, button pressed halfway through
5896 button pushed, then a stroke with button still pushed down.

Messing around, it seems that no output appears when I push button mid-stroke.

[+21529 ms] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[ +348 ms] I/[DWTE]  (23878): onPenEvent no editText
[        ] I/ViewRootImpl@39ea309[MainActivity](23878): ViewPostIme pointer 0
[        ] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[   +3 ms] I/flutter (23878): DEBUG INPUT: PointerDeviceKind.stylus, 1
[ +609 ms] I/ViewRootImpl@39ea309[MainActivity](23878): ViewPostIme pointer 1
[   +1 ms] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[+5189 ms] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[ +108 ms] I/[DWTE]  (23878): onPenEvent no editText
[        ] I/ViewRootImpl@39ea309[MainActivity](23878): ViewPostIme pointer 0
[        ] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[   +3 ms] I/flutter (23878): DEBUG INPUT: PointerDeviceKind.stylus, 1
[+1917 ms] I/ViewRootImpl@39ea309[MainActivity](23878): ViewPostIme pointer 1
[   +4 ms] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[+5896 ms] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[ +323 ms] I/[DWTE]  (23878): onPenEvent no editText
[        ] I/ViewRootImpl@39ea309[MainActivity](23878): ViewPostIme pointer 0
[        ] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[        ] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878
[ +879 ms] I/ViewRootImpl@39ea309[MainActivity](23878): ViewPostIme pointer 1
[   +1 ms] I/ViewRootImpl(23878): updatePointerIcon pointerType = 20001, calling pid = 23878
[   +2 ms] D/InputManager(23878): setPointerIconType iconId = 20001, callingPid = 23878

CodeDoctorDE commented 9 months ago

I'm currently confused. Our input should be first pushing the button and then drawing with the pen while holding the button. Is there really no output when doing this? In the pull request in saber, they use "invertedStylus" as kind but currently, I don't see this in the debug.

markwmuller commented 9 months ago

(Edit to add -- maybe the below helps? I'll be unable to add more for a bit, most likely, but maybe can test more tonight)

When I draw with my finger:

[   +4 ms] I/flutter (29768): DEBUG INPUT: PointerDeviceKind.touch, 1
[+1620 ms] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 1

Stylus, no button:

[+19362 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[  +59 ms] I/[DWTE]  (29768): onPenEvent no editText
[   +1 ms] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 0
[        ] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[   +1 ms] I/flutter (29768): DEBUG INPUT: PointerDeviceKind.stylus, 1
[+1531 ms] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 1
[        ] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768

Hovering pen over phone, no buttons, no stroke:

[+3721 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[   +1 ms] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768

Stylus, pushing button 1 second after starting stroke (nothing gets printed after pushing button, text only appears at start /end of stroke)

[+3425 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[   +1 ms] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[ +234 ms] I/[DWTE]  (29768): onPenEvent no editText
[        ] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 0
[        ] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[   +3 ms] I/flutter (29768): DEBUG INPUT: PointerDeviceKind.stylus, 1
[+1956 ms] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 1
[   +1 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768

Pressing button before starting stroke, releasing only after stroke ends:

[+4384 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[ +401 ms] I/[DWTE]  (29768): onPenEvent no editText
[        ] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 0
[        ] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[+1763 ms] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 1
[   +1 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768

Pressing button before starting stroke, releasing before stroke ends:

[+3617 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[ +123 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[ +118 ms] I/[DWTE]  (29768): onPenEvent no editText
[        ] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 0
[        ] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[        ] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768
[+1646 ms] I/ViewRootImpl@4d5a037[MainActivity](29768): ViewPostIme pointer 1
[   +4 ms] I/ViewRootImpl(29768): updatePointerIcon pointerType = 20001, calling pid = 29768
[   +1 ms] D/InputManager(29768): setPointerIconType iconId = 20001, callingPid = 29768

markwmuller commented 9 months ago

In summary, to add\ to above: no log text is generated when the button is pushed / released, but it does seem to register the stroke differently if button was pushed before pressing. Also, in case it matters: flutter --version Flutter 3.16.7 • channel stable • https://github.com/flutter/flutter.git Framework • revision ef1af02aea (5 days ago) • 2024-01-11 15:19:26 -0600 Engine • revision 4a585b7929 Tools • Dart 3.2.4 • DevTools 2.28.5

CodeDoctorDE commented 9 months ago

Thanks for testing it. I have really no idea how to fix this. This is the same code written in saber as here, we use the onPointerDown function from the listener. The only thing we could also try is adding these debug prints above https://github.com/LinwoodDev/Butterfly/blob/c7c2fd77e063ab12d5f9b237004f1a855158399f/app/lib/views/view.dart#L341C25-L341C25 to test if the kind and buttons changes while moving the pointer. Can you try this last idea out?

markwmuller commented 9 months ago

This does something! When I just use the stylus (no button) I get output DEBUG INPUT 2: PointerDeviceKind.stylus, 1

When I push the button it changes from 1 to 2. No debug input text appears if we push button before touching screen with stylus.

markwmuller commented 9 months ago

BTW, I accidentally got the debugger running, and hit the breakpoint. No idea how to get "event info", if you point me to a guide I can try to do later.

CodeDoctorDE commented 9 months ago

Really? This is the constant for stylus button: https://api.flutter.dev/flutter/gestures/kSecondaryButton-constant.html

Where did you get it? On the debugger you can hover on "event" and get all properties of it

markwmuller commented 9 months ago

I'm not sure I follow what you are asking, but I've tried to get that event data. Hopefully below screenshots provide enough context. I had to add the if statement to get the breakpoint when the button is down (otherwise it triggers always when stylus is down).

I've put a breakpoint specifically for when button 2 is down, see below:

When I enable a different breakpoint (2 lines up) and trigger without button 2 down I get below --

CodeDoctorDE commented 9 months ago

Hmm, and this button only shows up in the pointer move event? Currently the app only switches the tool when first interacting with the screen, the pointer down event. I'm currently interested when holding down the button and interacting with the screen after that

saipavanc commented 9 months ago

Hi, I have been following this discussion as well as I am looking for an opensource alternative to writing with pen on my android tab (samsung tab S7+). I have been using Write by StylusLabs since quite a while now. I really like the application, but sometimes I feel like the interface is not very smooth or it perhaps lacks some features.

I started using Butterfly, but the button not working in the app is a deal breaker for me, as it makes everything quite slow. I did see that the button actually does not work with some other opensource note taking apps as well. As you mentioned already, it has something to do with the sdk released by samsung for interfacing with the pen.

Though, I wanted to mention that the button works perfectly fine on Write, and I think that is the only free app I have seen it work. Since it is not open source, it is hard to say how they implemented it, but maybe it can give some idea? I am not sure. Also, I am not sure if I saw the discussion of using the spen remote sdk? Is it not used because it is not foss?

Also, I will be glad to help debug this issue. Though, I have to say I do not have any experience with coding for android apps, but I am okay with code in general on my linux machine. Let me know if there is anything I can do to generate debug messages.

CodeDoctorDE commented 9 months ago

Hello, I could test this app on a spen capable device and found out that onPointerDown, onPointerMove and onPointerUp does not fire. I don't really know how saber fixed it. When these events does not fire, I couldn't do anything about it. Since the issue https://github.com/flutter/flutter/issues/42846 has no workaround in it, I don't know what I can do

markwmuller commented 9 months ago

Isn't the logic the following?

User pushes s-pen down, app starts a pen drawing stroke
If, anywhere in that stroke, the user presses the eraser button, the app deletes the pen stroke, and now starts a path eraser stroke (note that my tests show that the app can see the button, as long as it is pushed after stroke begin)
- After stroke finishes, revert to pen tool.

CodeDoctorDE commented 9 months ago

Hmm, the problem here is: Every tool logic builds itself on having a "onPointerUp" call where you register this input. After that it edits your element with this pointerId. For this to work, it takes some time. But theoretically this could be the solution if the issue won't be solved in the near future. But on this logic we need to think about: If the user goes back to the pen tool, should it start drawing a stroke again?

markwmuller commented 9 months ago

But on this logic we need to think about: If the user goes back to the pen tool, should it start drawing a stroke again?

I don't think so. For me, it seems pretty intuitive that tapping the eraser button somewhere in a stroke means "this stroke is an eraser". Thus the eraser button is thought of as "make an eraser stroke now". I just checked, that's how Saber works.

Actually -- Saber turns the whole stroke (including the part that started as pen input) into eraser.

CodeDoctorDE commented 9 months ago

Okay, this sounds useful. Because this means a small refactor, I would start implementing it in the 2.1 update. It should be available ~ end of january

shayan15sa commented 8 months ago

Hey guys wanted let you know erasing with spen button works on Android 14 I just uptaded my tab s6 lite to android 14 now.It didn't used to work when I was on Android 13 but now it works fine.

CodeDoctorDE commented 8 months ago

I didn't change anything :D Are you sure that switching tools works with the button? Which version of the app are you using?

shayan15sa commented 8 months ago

I'm using version 2.0.0 and I haven't updated yet. My device is Samsung tab s6 lite 2022 edition.

https://github.com/LinwoodDev/Butterfly/assets/90975003/aa35c0d9-07de-4f59-a9dc-012de47e725b (Sorry for the quality, my internet connection is really bad)

Thank you very much for your attention.

CodeDoctorDE commented 8 months ago

Okay thanks. Maybe we could close this issue then?

shayan15sa commented 8 months ago

Sure And thank everybody for commenting on the issue

Laytongentleman commented 7 months ago

Please dont close the issue, my friends and I are using butterfly with S pen on old samsung tab (samsung tab s3) on android 9 with no possibility of updating to android 13 nor 14. It would still be a huge thing to correct the issue.

saipavanc commented 6 months ago

I am interested in this issue as well, I own a Samsung S7+ and it seems like I am not going to get any more software updates. So, maybe looking for an alternative for now, I just stumbled upon this: https://www.youtube.com/watch?v=PdCSMGYQGfw?

This is Tasker, and it seems like it can intervene and get the events from the spen and generate actions accordingly. Can this be used to register or generate events that Butterflu app can understand?

CodeDoctorDE commented 6 months ago

I don't know if this works. You can test it by going into the behavior settings and using the pointer test in the latest nightly

Dantee296 commented 2 months ago

@markwmuller HI

there is a particular log in your log file , do you know which app is printing these logs ?

Thanks

" 01-16 10:48:56.828 I/SLPASVC (24107): completed get profile info list [at com.samsung.euicc.service.EuiccServiceImpl.onGetEuiccProfileInfoList(EuiccServiceImpl.java:115)]"

markwmuller commented 2 months ago

@markwmuller HI

there is a particular log in your log file , do you know which app is printing these logs ?

Thanks

" 01-16 10:48:56.828 I/SLPASVC (24107): completed get profile info list [at com.samsung.euicc.service.EuiccServiceImpl.onGetEuiccProfileInfoList(EuiccServiceImpl.java:115)]"

Sorry, I don't know (and no longer have the toolchains to reproduce this)

LinwoodDev / Butterfly