Works on Mac :-) --- a few peanuts

[RaiMan]

macOS 10.15 - OpenJDK 14.0.2

Via the notifications on https://github.com/tulskiy/jkeymaster I got to your work here.

Congratulations - good stuff and very handy.

I am RaiMan from SikuliX (https://github.com/RaiMan/SikuliX1) and evaluating Ginj, about wether I can use it as screenshot tool (standalone/in parallel/as API).

I downloaded the sources and made my own project (IDEA, Git) for evaluation.

I noticed the following and made respective changes:

• the quit confirmation dialog buttons with star LEFT and BUTTOM are hidden by the star window solution: always display the dialog in the middle of the screen

• at first use I got noisy error messages about the settings.pref file missing (load and save) solution: the error at load should be a warning without stack trace, at first save of the defaults the missing folders should be created

• on start-up the star position should always be the middle of the stored edge (star.window.distance.from.corner not needed), for cases of screen changes or manual changes in the pref file

For the first two cases I can give you patch files if you want.

If I decide to continue with the evaluation and usage the next days, I will fork your repo and will then be able to create pull requests.

[vicnevicne]

Hi RaiMan. Thanks for testing and for the kind words !

It's great that it works - sort of ;-) . The glitches were more than expected as I guess you are the first one to have started Ginj on MacOS :-)

Regarding the changes, you are more than welcome to fork the project and propose PRs of course.

Your second point is clearly something I overlooked since I've not been in a "first use" case for weeks :-)

Regarding your first point, I'm not sure I understand how you get to the dialog being hidden. If you click the "more" button (gear icon), the More dialog should be centerd on screen, and then clinking on 'Exit Ginj' ('Quit Ginj' on Mac) should make the confirmation pop up centered on its parent Window (the More dialog). Isn't that what you are seeing ? : Now if you exit using the "Tray menu" (no idea how it looks on Mac), the confirmation should still be centered on its parent, which now is the star widget, and I guess that could cause issues. On Windows, it doesn't make a problem but I guess the layout is different on Mac: The easy way to make the confirmation dialog centered is to pass null as parent window to JOptionPane.showConfirmDialog(), but then the confirmation will not be modal anymore, which I find weird. On the other hand I could replace the default JOptionPane by a custom Dialog, but that's reinventing the wheel. Anyway, don't hesitate to propose a PR, and if the change is Mac-specific, please enclose it in a if (SystemUtils.IS_OS_MAC) { test.

As for the last point regarding position, I think it's important to remember where the user last positioned the star widget (on which screen, on which border and at what distance from the left or top edge of that display). For example, I'm regularly running Remote Desktop to access my computers, and the center of the top border is used by the Remote Desktop "handle", like this: https://www.mandsconsulting.com/wp-content/uploads/RemoteDesktop_FullScreen.png Normally, the positioning code should take care of constraining the values read from the config file to make sure the widget appears on a visible display, in the visible area, and not to close to any corner so the widget can "deploy"...

Please don't hesitate to comment, and maybe include a few screenshots. I'm really curious to know how Ginj looks on Mac :-).

PS: I'm going to have a look at SikuliX right now :-)

[RaiMan]

Thanks for your answer.

the More dialog should be centerd on screen ...

confirmed - no problem

Now if you exit using the "Tray menu" ...

yes, this leads to that depending on the star position the dialog is partly behind the star, so that at least at BOTTOM, the dialog has first to be moved, to get the buttons clickable.

The easy way to make the confirmation dialog centered is to pass null ...

I understand your point about modality. In SikuliX for the popups, I create a fake-frame (1x1) at a suitable place and use it as parent to keep the dialog popup modal.

As for the last point regarding position, I think it's important to remember ...

agreed - Of course it is ok and helpful to remember the screen edge, but I think the exact position does not really matter and might make problems in some cases. IMHO using the middle of the remembered edge is ok.

Some images about how it looks on my Mac: https://www.dropbox.com/s/qoi0dl3i1yjjv5l/GinjConfirmQuit.png?dl=0

All the best (will surely come back ;-)

BTW: the confirm dialog has Jing in its title ;-)

[vicnevicne]

Thanks for the pics. I get the issue more clearly now. In fact, I think the root cause is that on MacOS (but also on Ubuntu according to early feedback I got), the transparent window is just translucent (like 50% opacity or so). On Windows, as you can see on the screenshots above, there is no grey rectangle around the yellow circles. But it's not only about visibility: it also means that the transparent area on Windows is "click through", so even though buttons are just next to the circle, that does not prevent interacting with them. The "grey rectangle" may be due to one of the two following issues:

• either windows on Mac cannot be made transparent. That could be checked by testing the third example (shaped window) in that page of The Java Tutorial
• or windows can be made transparent, but drawing a partially transparent PNG on them (like I do, inspired by that code on StackOverflow ) is not supported

As I have no access to a Mac, I'd gladly exchange your proposal of writing Pull Requests for the results of those tests :-).

If the issue is with the transparent PNGs, the simple solution is to replace the PNG by code-drawn circles, which would be quite easy (it's already in my TODO list in any case). That would allow me to hit three birds with one stone (visual artifact on Mac, quit confirmation buttons clickability, and improved experience on Ubuntu). If the issue is with Windows transparency OTOH, I guess I'd be out of options for the visual issue, and the "intermediary 1px window" trick would prove handy :-) Clever !

Regarding the position on the border, I think the exact position does really matter because when the computer you remote-control has the yellow icon at the center, it cannot be accessed in full screen mode. Moving it to the left or to the right ensures it remains reachable even with the remote desktop bar on top. But I admit I don't understand the issue with remembering that position...

Oh, and did I type Jing - again ? Sometimes I'm getting a bit too close I guess :-P. That's fixed in the code now.

Thanks for all !

[RaiMan]

ok, I will now fork the repo and make some tests.

I think the exact position does really matter

Ok, understood and agreed. But then you should not store the absolute pixel location, but a relative value like pos-in-pixel/edge-length-in-pixel.

[vicnevicne]

Mmmh remembering a percentage makes sense indeed. Will do that. Thanks ! And regarding the tests, if you could just compile and run the the two sample codes linked above and tell me what they look like on Mac, that would be a giant step forward....

[vicnevicne]

Relative position is implemented. Will be in the next release.

[RaiMan]

ok, here you are with the Java examples on macOS:

TranslucentWindowDemo

had to add: setUndecorated(true);

GradientTranslucentWindowDemo

had to add: setUndecorated(true);

ShapedWindowDemo

hope it helps :-)

[RaiMan]

... and now the sample from StackExchange:

[vicnevicne]

Excellent, RaiMan. Thanks. What your captures prove is that the bug is in my code, not in Java (who doubted it ? :-)). Now I just have to find where :-). That's my next task.

[RaiMan]

feel free to come back for tests and/or evaluations

[vicnevicne]

Hmm, I didn't spot any real difference between the StackOverflow sample logic and the way I do. Then a friend on Mac reported the same issue to me and send me a video which shows something interesting: While the background is translucent grey when the mouse is away, it seems the background becomes opaque black when the widget "expands" because the mouse is over it. In fact, when the mouse is out, it's on purpose that the widget is made half transparent, and that it becomes opaque when hovered. But if the background becomes pitch black, I'm inclined to think the issue is with the PNG. So I checked more and observed the two widget PNGs (folded and expanded) are encoded as 8-bit PNG (by mistake) and not 32-bit: I'm starting to think there may be a bug in 8-bit transparent PNG handling on Mac and Linux. II quickly wrapped-up a Mac-only v0.3.10-pre . It should not be proposed as an auto-update, but you can download the installer and install it over the previous one. I'd be curious to know if it improves things... The other feedback I got is about the multi-desktop use of MacOS, where you swipe between apps on different Desktops, and Ginj does not detect it and remains on the original one instead of being visible on all desktops. I have to find out how this can be detected or handled in Java, but if you have hints about that, the're more than welcome :-).

[RaiMan]

ok, did it - no change :-(

mouse outside

mouse inside

BTW: For people like me, having Java available anyways, a jar-with-dependencies would be enough (download the small jar and double-click)

     <plugin>
        <artifactId>maven-assembly-plugin</artifactId>
        <version>3.1.1</version>
        <configuration>
          <archive>
            <manifest>
              <mainClass>info.ginj.Ginj</mainClass>
            </manifest>
          </archive>
          <descriptorRefs>
            <descriptorRef>jar-with-dependencies</descriptorRef>
          </descriptorRefs>
        </configuration>
        <executions>
          <execution>
            <id>make-assembly</id>
            <phase>package</phase>
            <goals>
              <goal>single</goal>
            </goals>
          </execution>
        </executions>
      </plugin>

.... and using <profile>you can make the creation optional

[RaiMan]

... and about multi-desktop - first impressions:

• the tray icon in the top menu bar is always visible

• the program icon in the dock also

• clicking the dock icon brings the star to the current desktop

• clicking the tray icon does not move the star, but opens menu

  • then the behavior of the menu items is different:
  • quit moves it and shows the confirm dialog
  • history does not, but opens the window in the desktop having the star (hence not visible :-(
  • using the capture key allows a capture, but only moves the star after finishing the capture and shows the capture window
  • ...

[vicnevicne]

ok, did it - no change :-(

Damn :-( I really wonder what the issue is... Clearly the transparent part of the PNG is rendered a pure black, and when it's not hovered, the 50% opacity makes it transparent grey (like the yellow circles are transparent yellow), very visible on your screenshots. I'll switch to drawing the circles using java calls instead of PNG. At least that should fix the issue when not hovered.

BTW: For people like me, having Java available anyways, a jar-with-dependencies would be enough (download the small jar and double-click)

Err, if I'm not mistaken, that's the Ginj.jar I usually provide, right ? I didn't do it for the 0.3.10-pre because it was not meant to be a release, but for all others, it's available. Or do you mean something else ?

... and about multi-desktop - first impressions:

Thanks for the detailed report. That's mostly what I had understood. At first I was thinking macOS used shifted coordinates for alternate desktop (like when you have multiple screens), but it's not the case. It seems when switching desktop macOS just makes all windows on the current desktop invisible, then makes the ones of the new desktop visible. When Ginj makes a capture, I first hide the widget with setVisible(false) (so it's not part of the captured image), then wait for the user to perform the selection, then I call setVisible(true), and it seems that call makes it reappear on the current Desktop. In other words, as soon as I detect a desktop change, I should just reissue a serVisible(true). But the question is: how do I detect a desktop change ? A quick search seems to indicate it's not possible without native calls via JNI or JNA, but as I'm already including JNA for global hotkey capture, maybe it's the way to go. Similar answer here. Seems the rococoa project was the way to go but java.net is dead. It was imported into Google Code but code.google.com is dead too. It was imported on github (for example here) but all the issues and discussions are lost forever (even absent from web.archive.org ). Jeez Internet is 30 years old and half of it has already disappeared... Sorry I digress.

Anyway, it doesn't look like an easy task, except maybe having a loop forcing "setVisible(true)" every second or so, but that's not something I can test remotely... I'll try to see if someone else has come up with a solution...

OK, back to the transparency issue for now.

[vicnevicne]

Hello. I just released a second "Mac-only" release (dmg or jar) with changes in 2 areas:

1. transparency around the widget First there's no 8-bit or 32-bit PNGs now. The yellow circles and the 8 rays (visible when dragging) are now drawn using drawing primitives (fillOval(), drawLine()) so PNG is ruled out (only the 3 "buttons" are still PNGs but they only appear when hovering but not dragging.

   • If the issue is fixed, great.
   • If not, I've added a config setting (*) called "debug.no.opacity.change". If set to true, the code won't fiddle with opacity. So the widget will not "fade to semi transparent" when not hovered, but hopefully that could fix the transparency issue
   • If still not good, I've added another config setting (*) called "debug.no.fake.transparency". If set to true, the code won't fiddle with transparency. So when hovered, the area around the 3 buttons will not keep them opened. It's a bit annoying for slow people because the buttons will sometimes close before they have the time to click on them, but that's the last display difference with the working "leaf" sample.

2. presence of the widget on the right desktop ("Space") For debugging, I have added a "recovery" method now called each time a menu entry (from the tray) is clicked, which does the following to the widget:

   setVisible(false);
   setVisible(true);
   toFront();
   requestFocus(); 

   So if Ginj runs but you are on a desktop without the widget, could you please open any tray menu entry (e.g "Exit" (then Cancel)) and check that the widget is now on the current desktop. If so, can you check that all other menus now behave correctly Then can you go to another desktop and do the same with e.g. History If not working, well I'm afraid that's all I can do. If it works, I'd like to determine which of the 4 instructions above are required to successfully bring the widget in full function, so I made them all optional. If you change the following config params (*) one by one, we should hopefully be able to determine what is required;

   debug.no.setvisible.false.in.recovery
   debug.no.setvisible.true.in.recovery
   debug.no.to.front.in.recovery
   debug.no.request.focus.in.recovery

   My hope would be that setVisible(true) is sufficient.

I understand this looks like I'm asking you to be my guinea pig, so please tell me if I'm bothering you, and already big thanks for your help so far. Maybe I can also ask my friend to share part of the tests. For example, maybe you can focus on opacity and I can ask him feedback about "widget recovery"...

KR. Vicne

(*) To change a setting:

• first exit Ginj
• open the "settings.properties" (in the user home dir's .Ginj subfolder)
• edit/add/remove the parameter and save
• restart Ginj

[RaiMan]

No problem - I will do what you want. Thanks for the detailed description.

... be sure, that I am not feeling like your guinea pig. With SikuliX I am also always happy, if someone else helps me with tests and evaluations (especially on Linux systems ;-)

[vicnevicne]

(fyi I made one more pre-release that should fix preference file and folder creation upon first use - with no stacktrace if not existing upon load)

[RaiMan]

Behaviour after switching desktop

when clicking a menu entry from tray icon, the star is moved to the current desktop, but the resulting window/dialog is not brought to front, but might be fully/partly hidden behind the frontmost app window, depending on its size and position. With capture, the behavior is not consistent at all:

• sometimes the star gets visible before the cross, sometimes not
• sometimes it stays visible while capturing
• sometimes I only get a black screen with the cross and the area around the star
• ESC to cancel the selection does not work sometimes
• after having done a first trial/recovery it might work normally

IMHO the best solution would be a hotkey, that moves the star to the current desktop/monitor and makes it the frontmost app. So the app has a consistent start point.

[vicnevicne]

Thanks for your observations. I'm leaning towards some kind of race condition, due to the async nature of Swing. E.g. my "setvisible false/true" might not come into effect before the action is launched. But adding a "sleep" is clearly not a clean solution. What I could do is have a background thread that tries to recover the widget every second. Of course, I would avoid making it invisible first (which would cause an annoying flash), and stealing focus (which would be absolutely horrible). To check that it could work, could you set the two following parameters in the config file: debug.no.setvisible.false.in.recovery=true debug.no.request.focus.in.recovery=true Then restart Ginj, switch to another Desktop, Use the tray menu to initiate "Quit" and cancel the box (after that, setVisible(true) and toFront() will have been called), and then try to see if the behaviour has improved (widget visible ? new windows opening on top ?) Or do you have another idea ?

[vicnevicne]

Hi, Just to let you know, I have stolen a iMac ( ;-) ) to make a few tests regarding opacity. The good news is that I have found the culprit for the black background: it's due to the Synth Look and Feel. [rant]After working with Synth for 2 months I think I can testify that it is both the best idea and the worst implementation for custom UIs. Almost no documentation, inconsistent naming (TEXT_BACKGROUND or BACKGROUND ?) and moreover an atrocious and over-complicated precedence logic where every attribute is (or is not, it depends) inherited from another state (or no state), or an alias style, or a default style, and also depends on the order of appearance of the declarations in the XML file.[/rant]. Anyway, I believe I could fix it but it's not the worst unfortunately: it seems on Mac, transparent Java windows are not "click-through". I just re-tested the "leaf" example on Windows, and the result is : https://www.dropbox.com/s/vxbuiut4mtisbkr/leaf.mp4?dl=0 . Can you test if it behaves the same on a "real" Mac (I'm using a VM) or if the transparent area captures the clicks ? If clicks in the transparent area are not forwarded to the background app, I'm afraid the whole concept is flawed :-( . And it's not only the widget, but I'm now also using this "transparent trick" to draw a rectangle around the video being captured, and if you cannot interact with the app while it is being recorded, that kinds of defeats the idea of recording the screen :-/

[RaiMan]

Could you give me the source code of your click-through-leaf-test ;-)

[vicnevicne]

That's just how the unmodified Stackexchange code you tested above in that post behaves on Windows... If it's not click-through on Mac, it would be interesting to test if shaped windows (the test just before) is click-through...

[RaiMan]

Uups, apparently too hot here these days ;-) Thanks.

[RaiMan]

ok, confirmed on Mac: The clicks are captured by the complete overlay frame and do not go through the transparent areas to the application behind.

[vicnevicne]

:-(. So probably I should change the size of the window to just include the circle when « folded ». Hmmm... not much time for anything these days unfortunately... Thanks for the test.

Other topic: do you have any idea how a Process launched from a Java app on Mac can be authorized to capture screen contents ? The case is ffmpeg of course. My first mistake is that the ffmpeg version installed with v0.4.0 does not have the executable bit set, but even after a ‘chmod 755’, ffmpeg cannot access the screen. When launching ffmpeg to capture the Desktop from Terminal, I’m getting a request to authorize Terminal to capure screen, but when lauched by Ginj I don’t know how to prompt the user and have ffmpeg authorized... Any idea ? KR. Vicne

[RaiMan]

You have to authorize the Java process:

The challenge: It cannot be evaluated, which line stands for what version. The ones with the terminal icon stand for a java started in Terminal and the others might be for the cases where a jar was double-clicked.

[RaiMan]

I am sure this can be supported by the macOS Api, but that means fiddling around with JNA/JNI.