ActivityWatch / activitywatch

The best free and open-source automated time tracker. Cross-platform, extensible, privacy-focused.
https://activitywatch.net/
Mozilla Public License 2.0
12.42k stars 558 forks source link

macOS Catalina complains repeatedly about an unverified developer #334

Closed tivvit closed 2 years ago

tivvit commented 4 years ago

Screenshot 2020-01-16 at 19 49 25

How can I overcome this?

rtpHarry commented 4 years ago

I've got stuck with this as well. I'm pretty new to macOS and looking for a good replacement from manictimer which I used on Windows.

I went through all of the executables in the main folder and all the dyn libs; by going to the context menu, clicking open, and open on the dialog to bypass gatekeeper.

But after that it complained about ./PyQt5/QtCore.so not being verified, which can't be done via the open method above.

I've tried to do it via the terminal instead, using spctl --add /Path/To/Application.app and I did it by dragging the file onto terminal to ensure that the path was correct.

It asked for my password which I supplied via touchid, but it didn't make any difference the next time I tried to run aw-qt.

I'm stuck!

RobertoMaurizzi commented 4 years ago

I had/have it running on Catalina but the dialogs I got were a bit different:

0) start ./aw-qt 1) It'll show a dialog saying something like "Sorry, no signature, I won't run this at all, move to bin" (there was no Open button, or it was greyed out) 2) After you get this go in "System Preferences / Security & Privacy / General": you'll see text close to the bottom of the window mentioning the offending file and a button to allow execution 2a) The first time only, before clicking the allow button, CLICK THE ICON TO BECOME ADMIN and type your password. If you don't the button will click, but it won't have any effect nor show any error (because Apple UX testing is sooo gooood :-p ) 3) Click the "Allow" button in the Security & Privacy window. Keep its window open. 4) Your only option now is to cancel the existing error dialog then run again ./aw-qt 5) This time around you'll get a dialog with a clickable "Open" button. Click it. After you click, you'll get another error dialog for a new binary with the only option to cancel or move to bin, just the same as in point 1). Press Cancel. 6) go back to 2) and repeat until you don't get any dialogs and it'll work. For me that was about 50 times...

If instead you're getting dialogs with an active "Open" button, I'd expect that you'll get around 50 of them (each asking for a different file) then aw-qt will start.

I do think that install everything in a user-created virtual environment should avoid all these problems, at least until Apple makes impossible to compile your own Python and/or use pyenv. At that point I'll be time for me to start again using a different OS.

tivvit commented 4 years ago

Maybe it would be better to run the server in docker?

Next, it would be nice to pack the monitoring script differently (to use system libraries). But I do not know if this is possible and if the maintainer is willing to do it.

I am willing to help, but I have to examine the project and why is it distributed like this.

johan-bjareholt commented 4 years ago

@tivvit Running the server in docker would eat up a lot of RAM and make the installation more complicated, if someone wants to do it themselves that's fine but having it as the recommended way to run ActivityWatch is out of the question. It's not even technically possible to run aw-qt or the watchers inside docker. Docker is not made for user applications and is a bad fit unless you only want to host aw-server.

It is not possible to use system libraries because we use python and qt. The built-in python version in macOS is over a decade old and qt is not available.

There are improvements to the packaging of ActivityWatch for macOS to make it into an app bundle, go to the aw-qt to see the progress on that. Not sure if it solves any of these permission issues though.

ErikBjare commented 4 years ago

We have a .app bundle almost ready here: https://github.com/ActivityWatch/activitywatch/pull/327

I also recently got an old MBP 2010 but I don't have Catalina running on it. However, the moment we have the PR merged I can give you a download to test out. If that's not enough, I will investigate registering with Apple to properly sign the .app.

RobertoMaurizzi commented 4 years ago

Just today (while fighting with Logitech's device manager app that apparently forgets to tell you to give Input Monitoring permissions) I noticed that in Security & Privacy under Privacy there's a Developer Tools entry (on my Mac containing only Terminal) that apparently "Allows the apps below to run software locally that does not meet the system's policy". That said, I can't wait for the offical .app, thanks @xylix :-D

xylix commented 4 years ago

@tivvit @rtpHarry @RobertoMaurizzi

Update: I have now updated my macOS to Catalina and (at least) the newest master works as expected after code signing with a self-made certificate. If you want to sign yourself, these instructions should be enough, expect change the codesign arguments to:

codesign --deep -s "your new cert" /Applications/ActivityWatch.app

If anyone is interested you can try my self signed version from here. It is not an official build, but I will help you debug macOS specific issues with this build or any self-built version, so we can get ActivityWatch on macOS working as smooth as possible :).

I'm working on getting the code signing to work in our CI, but it will take some time.

rtpHarry commented 4 years ago

I tried the self-signed version on your Google drive but that didn't work:

image

Will investigate the other option.

xylix commented 4 years ago

Sorry about the problems, but progress is being made. The problem with the .zip was my zipping tools messing up some metadata which made the signature invalid.

@rtpHarry A new signed dmg is here https://drive.google.com/open?id=1KPNPIND8jGFVq21fAoV7KICLadP3Nwg- .

Try it out if you can. CI support is still in progress.

rtpHarry commented 4 years ago

Aww yissss! :) It has installed and opened. It seems to be collecting data. I've given it the permissions it wants. I'll report back after a few days usage.

tivvit commented 4 years ago

works for me

xylix commented 4 years ago

We'll be releasing an official 0.9 very soon with a signed dmg. Thanks to everyone who helped me test the catalina signed versions :)

xylix commented 4 years ago

Have any of you running the .app had issues with window titles not being recorded? (issue #376)

Screenshot 2020-03-15 at 19 17 05

rtpHarry commented 4 years ago

Not had that problem, no

rtpHarry commented 4 years ago

is it possible to get this build to auto start on load? I can't find a setting or anything in the properties of the .app

rtpHarry commented 4 years ago

oops brain is not working, didn't google it first (https://www.idownloadblog.com/2015/03/24/apps-launch-system-startup-mac/)

vitorgalvao commented 4 years ago

I’m not a user of this software, but this is being submitted to Homebrew Cask and I was interested in the self-signing approach.

Self-signed certificates are not meant to be accepted by Gatekeeper, and sure enough this still doesn’t seem to fix the issue in the top post.

I’ve tried both the DMG on the 0.9.0 release and the one on Google Drive on a clean VM. They still report this as being from an unidentified developer and behave like any unsigned download.

xylix commented 4 years ago

@vitorgalvao Yes the .dmg needs the user to manually "open anyway" through the security menu. After that there should be no more popups on future openings of the application, which is the best result we could get through self signing.

vitorgalvao commented 4 years ago

which is the best result we could get through self signing.

Which is the same result you get from an app that isn’t signed. That’s the point I was making: self-signing does nothing in this context (except more work for you).

xylix commented 4 years ago

Oh. I'll test an unsigned .dmg then.

xylix commented 4 years ago

Did a new build and confirmed self-signing the .dmg doesn't make a difference.

@ErikBjare usage of the .app is possible on Catalina without signing, but the warnings will always happen if we don't get the official apple developer license.

This issue's original problem which was that all the script files contained in the /activitywatch folder needed to be given permission one at a time is fixed with the .dmg / .app packaging though.

rtpHarry commented 4 years ago

@xylix yes I am getting the no titles being recorded issue now. Still running the same custom build that you gave me.

One thing I changed was to make the app auto load on startup. Could it be that it gets some different permissions when loaded this way?

I have also installed the latest macos update that came out, but I suppose that's unlikely the culprit if you have been experiencing it before then.

rtpHarry commented 4 years ago

Hmm I just looked through the logs and it seems like the last title it recorded was me starting the latest catalina system update:

image

xylix commented 4 years ago

@rtpHarry Good info. As a possible fix, could you try removing ActivityWatch from your allowed accessibility permissions and re-adding it?

I'm also working on a clearer "missing permissions reporting" over at aw-watcher-window, but it'll take some time to be ready for use.

rtpHarry commented 4 years ago

Good call! That restores the titles, yes:

image

rtpHarry commented 4 years ago

fyi the same issue happened when I went to the site and downloaded the 0.9.2 dmg and installed it - no titles until I removed and re-added accessibility permissions

jankatins commented 4 years ago

I've installed activitywatch by downloading the the activitywatch-v0.9.2-macos-x86_64.dmg file, dragging the icon to application and then went to /Application, clicked right->open, answered the security question and got an icon in my tray/menu bar. I also went and added activitywatch to the accessibility security&privacy list.

Unfortunately, now my system has about 60% of an CPU running trustd, syspolicyd and my virus scanner.

I can trigger this by quitting activitywatch via the menu icon (all processes stop using CPU) and restarting it (all start up again). When I play with the started modules, the CPU load stops when I stop aw-watcher-window and starts again when I enable it. The rest (server and afk) make no problems.

Unfortunately, this 60%CPU is already enough to start the fan spinning, so it's quite annoying :-(

rtpHarry commented 4 years ago

@jankatins I had some heavy usage spike with it and thought it was going to be a big drain, but I think that's just the initial scan from your virus checker. Once it has read all the files and scanned them it should settle down. The power usage on my laptop now is negligible, and I've been using this for a month or two. Perhaps give it a bit of time to see if it settles down?

(btw if this is a real issue then you should probably make your own ticket so it can be tracked properly)

jankatins commented 4 years ago

Sorry, should have realized this: https://github.com/ActivityWatch/aw-watcher-window/issues/45

I started this comment when I was still thinking this was related to the signing problem because lot's of syspolicy google hits pointed to certificate checks.

grigorye commented 4 years ago

I'm not sure whether this is relevant, but I see "System Events.app" eating some noticeable amount of CPU (10%) with Activity Watch 0.9.2 and aw-watcher-window enabled.

xylix commented 4 years ago

@grigorye I think the CPU usage is related to https://github.com/ActivityWatch/aw-watcher-window/issues/45 . Can't confirm right now though.

seantms commented 3 years ago

I tried installing this recently, but macOS Catalina did not show the "Open Anyway" option.

Had to do a "sudo spctl --master-disable" which I believe disables Gatekeeper, so you should do a "sudo spctl --master-enable" once done with the installation and giving it the permissions it needs.

You can read up more here: https://en.wikipedia.org/wiki/Gatekeeper_(macOS)

ErikBjare commented 3 years ago

but macOS Catalina did not show the "Open Anyway" option.

@seantms I tried this today, and when first trying to open it I didn't get the option on the dialog, but when I tried again there was an "Open Anyway" button :man_shrugging:

I'm running the latest v0.10.0 .app on a clean Catalina VM, and apart from "unverified developer" warnings things are working fine (as well as the dialog we open to tell the user to give us Accessibility permissions getting drowned under other open windows). Didn't need to do sudo spctl --master-disable.

TheTimeWalker commented 3 years ago

I just want to note for Big Sur (and possibly Catalina), you don't have to fully disable Gatekeeper to get the Open Anyway option. When you move the app into the Applications folder, you can just open the Context menu (hold CTRL or right click) and then select Open. This actually is different from double-clicking and opens up a different pop up that adds in the option to "Open Anyway".

dr-mrsthemonarch commented 3 years ago

To comment on this thread, on a macbook pro m1 big sur 11.4, I had to allow every single script to get the program to run at all...this took a full hour of starting the /aw-qt and reclosing and accepting 'run anyways' to each single complaint until it worked without errors, the .app refused to work at all. So to those who want to use it, it does certainly work eventually.

may commented 2 years ago

Update on this thread for anyone who got here from the Downloads page.

To get ActivityWatch working on MacOS Monterey 12.2.1:

  1. Download v 11 (the latest at the time of this writing.)

  2. Open the .dmg file and drag the app into Applications as usual.

  3. Launch the app. Note that it doesn't launch/the permissions dialgoue you get doesn't give you an option to allow.

  4. Open System Preferences.

  5. Go to System Preferences -> Security and Privacy

  6. Ensure you are on the General tab, and you see something towards the bottom that says 'Activity Watch was blocked from use because it is not from an identified developer....' just click Open Anyway, then you get another scary popup, click Open on there.

    image image
  7. Activity Watch should now be running.

  8. Notice that your timeline is full of 'unknown application'

  9. Click the Activity Watch icon and select 'Quit Activity Watch'

The only reliable way I've found to ensure ActivityWatch has the correct permissions on MacOS is to run ActivityWatch via the command line. For those who'd like to do this but don't want to use the command line, I created a one-line script to do it for you, that can also be triggered on login:

  1. Open up Terminal and copy-paste this line and press ENTER:
  2. echo "/Applications/ActivityWatch.app/Contents/MacOS/aw-qt &" >> ~/Documents/aw.command
  3. Then copy-paste this line and press ENTER:
  4. chmod +x ~/Documents/aw.command

THEN, you need to ensure that:

  1. System Preferences -> Security & Privacy -> Privacy -> Automation -> Terminal
  2. Terminal is allowed System Events and ideally also Safari image
  3. Terminal AND Activity Watch are allowed allowed Accessibility permissions as well

Now you can launch ActivityWatcher with all the proper permissions (you may have to say yes to a few things, the 'Terminal' app will ask for permissions on behalf of Activity Watch).

To do so, go to your Documents folder and double-click on the aw.command file we just created.

For bonus points navigate System Preferences->Users->Login Items->plus under Login Items->find Documents then aw.command that we created. This will launch Activity Watch on login.

Now Activity Watch will work as expected without any additional fussing. Hopefully.

If you want that terminal window to disappear on its own, follow these instructions. The window will still flash up momentarily, but that's it.

The only downside I've noticed with this method is that if you quit the app and relaunch it (via the script) you won't be able to quit it a second time. You can either kill all the processes manually or restart. Otherwise, seems solid so far.

betty-white-it-works

Note that the latest commit in the main branch does work better in MacOS, but still doesn't work out of the box with the proper permissions.

I'm curious what @dr-mrsthemonarch did repeatedly to get so many prompts for permissions and get it working that way.

Below follows a bunch of other things I thought I got to work but couldn't reproduce successfully on another computer.

  1. Go back to System Preferences
  2. Navigate again to System Preferences -> Security & Privacy
  3. Go to the Privacy tab
  4. Select 'Accessibility' on the left. Activity Watch may or may not be listed under 'Allow the apps below to control your computer'. If it is, click the lock to unlock, authenticate, and click the checkmark next to Activity Watch.
  5. If Activity Watch isn't listed, click the plus icon and find it in the lookup (within the Applications folder).
  6. Then click the checkmark next to Activity Watch.

When you're done, it should look like this (note Activity Watch at the top of the greyed out list):

image

Finally, for the devs who may read this:

  1. Thank you. I think this will be better than RescueTime. I just need to look and see if there's a goals function.
  2. Please note that when I tried v0.10 it PROMPTED me for Accessibility permissions. Restoring that in v0.11 or v0.12 will probably help new users. That said, I know that's Apple changing things on you, but I wanted to mention it.
  3. There is a regression between v0.11 and v0.10 -- v0.10 correctly identifies when I'm in 'Safari' but v0.11 lists it as 'unknown'.

Edit: for #3, I was able to get window titles working by editing the config file for aw_window_watcher:

[aw-watcher-window]                                                     
#exclude_title = false                                                  
#poll_time = 1.0                                                        
#strategy_macos = "jxa"                                                 
strategy_macos = "applescript" 

To do so:

  1. Open Terminal
  2. nano ~/Library/Application\ Support/activitywatch/aw-watcher-window/aw-watcher-window.toml
  3. Press Ctrl-X, then type Y to save, then ENTER.
  4. Relaunch ActivityWatcher.

I tried a bunch of other solutions, including granting various permissions to ActivityWatcher (after removing and adding the accessibility permisionn) and that didn't work. However, when I run Activity Watcher from the command line (eg. Terminal, which already has full Accessibility permissions), everything works fine without editing the config file, which leads me to believe that it's some kind of permissions issue.

If anyone else wants to run it from the command line: open /Applications/ActivityWatch.app

Or to prove that the new method of getting window titles works with enough permissions:

while sleep 1; do /Applications/ActivityWatch.app/Contents/MacOS/aw_watcher_window/printAppStatus.jxa; done

Running the watcher directly from the command line is also enough to get around this:

/Applications/ActivityWatch.app/Contents/MacOS/aw-watcher-window

ErikBjare commented 2 years ago

@may That is very helpful, thanks!

may commented 2 years ago

Edit: This comment did not work, stick with what I outlined in the above.

I think System Preferences -> Security & Privacy -> Privacy -> Automation may also be needed, and I can't drag-and-drop nor manually add any part of ActivityWatch to that list. (Every other activity tracker I'm currently using needs that permission - RescueTime, Qbserve, and the terminal I run ActivityWatch from etc.). This also doesn't work if you tell activity-watcher-window to utilize the old AppleScript method.

An alternative approach I am trying is explicitly giving Accessibility permissions to all parts of ActivityWatch.

  1. Open System Preferences -> Security & Privacy -> Accessibility.
  2. Open Finder -> Applications -> right-click ActivityWatch -> Show Package Contents
  3. In Finder, within the ActivityWatch folder you just opened: navigate to Contents -> MacOS.
  4. Drag each of the following into the Security & Privacy window, giving them each accessibility permissions:
  • aw-qt
  • aw-server
  • aw-watcher-afk
  • aw-watcher-window

Perhaps we only need to give aw-watcher-window the permissions, but it doesn't hurt to be through.

I will test this and update this comment if it works consistently.

image

fcosanabria commented 2 years ago

Thanks @may I was able to replicate this in Monterey 12.2.1 Intel using these instructions. Everything looks good so far.

Just want to mention that the Terminal won't show up under Automation if you have not used it before for background process. As May mentioned, this step is only to double check, but if it does not appear, no worries, you can try to run the file.command, and it will show up. :)

ErikBjare commented 2 years ago

Can someone try this with the new v0.12.0b1 beta, and see if its an improvement? The .app is now properly codesigned: https://github.com/ActivityWatch/activitywatch/releases/tag/v0.12.0b1

It might not open when double-clicked (unrelated bug), but should open if run from terminal with open ./ActivityWatch.app

may commented 2 years ago

@ErikBjare Code signing improved, still permissions issues, but we can work around permissions issues. So technically I guess this bug could be closed assuming others replicate the 'unverified developer' problem is handled.

  1. Quit ActivityWatch.
  2. Download and open 0.12.0b1: https://github.com/ActivityWatch/activitywatch/releases/tag/v0.12.0b1
  3. Drag to Applications folder.
  4. System Preferences -> Privacy -> Accessibility -> Remove any existing Activity Watch entries.
  5. Click the plus and add the new Activity Watch from Applications.
  6. You may need to exit System Preferences between step 4 and 5.
  7. Launch Activity Watch from Applications folder. Notice that it is codesigned now and doesn't warn about scary unknown devs.
  8. Click the Activity Watch icon, click Open Dashboard.
  9. Notice that no data is being displayed from the watcher.
  10. Notice that Activity Watch, when launched, does not prompt for any permissions.

Error message for Activity Watch window watcher is as follows: File "aw_watcher_window/macos_jxa.py", line 64, in getInfo Exception: jxa error: Error: Error: An error occurred. 2022-03-12 15:18:45 [ERROR]: Exception thrown while trying to get active window (aw_watcher_window.main:75) Traceback (most recent call last): File "aw_watcher_window/main.py", line 72, in heartbeat_loop File "aw_watcher_window/lib.py", line 55, in get_current_window File "aw_watcher_window/lib.py", line 27, in get_current_window_macos File "aw_watcher_window/macos_jxa.py", line 64, in getInfo Exception: jxa error: Error: Error: An error occurred.

  1. Quit Activity Watch.
  2. Reboot to prevent issues.
  3. Use Terminal method described in https://github.com/ActivityWatch/activitywatch/issues/334#issuecomment-1052392579
stpn commented 2 years ago

@may Thanks for your suggestion - your method allows to launch ActivityWatch successfully, however it seems that it produces another issue - I cannot quit Chrome, as it automatically restarts. Any chance you'd have some suggestion on how to address that? Thank you

may commented 2 years ago

@stpn No idea why that'd be the case, but I'm glad to hear about it since I've been having problems with chrome not quitting too. All the script is doing is starting ActivityWatch from the command line, with proper permissions.

vishalkrv commented 2 years ago

@may Thank you for putting up the installation steps, I was able to install v0.12.0b2 on M1 Monterey 12.2.4 and everything seems to be working including firefox integration, vscode integration and window title.

JonThom commented 2 years ago

Hi guys, Gatekeeper blocks ActivityWatch on MacOS 12.2.1 (“ActivityWatch” cannot be opened because the developer cannot be verified.) Haven't gone through every comment above, but I suppose this is due to code signing, and most likely Qt. I myself had to struggle to get a Qt app bundle (using QtWebEngine via PySide6 in Python) codesigned and notarized, but managed in the end

ninadpchaudhari commented 2 years ago

In my experience, for v0.11 release and Intel macOS Monterey 12.3.1 These did not work. The app would hang up on launch and got an error about

[Exception thrown while trying to get active window: error compiling jxa script](https://forum.activitywatch.net/t/exception-thrown-while-trying-to-get-active-window-error-compiling-jxa-script/1399)

What worked was this: Adding AW to Complete File Control, Developer Tools ( Could not find it in Automation though)

dingwen07 commented 2 years ago

@may Just found another workaround of directly adding the permission: https://github.com/ActivityWatch/activitywatch/discussions/768

ErikBjare commented 2 years ago

Hi all,

I bought a MacBook Air M2 last week just so I can finally resolve all the macOS issues, and today I bring good news:

I've managed to merge https://github.com/ActivityWatch/aw-watcher-window/pull/68 which changes how window titles are tracked, and seems to be a lot more robust (solves a bunch of issues with the current method). I'm almost done with the finishing touches, and hopefully everything will "Just Work™️" once the release is out.

I know you all have strugged getting ActivityWatch to run on macOS, and hopefully this is the last thing needed to get it working without issue.

Thank you all for your patience and support. Stay tuned for a new release.

ErikBjare commented 2 years ago

Please try this new release and see if things have improved: https://github.com/ActivityWatch/activitywatch/releases/tag/v0.12.1 🤞

ErikBjare commented 2 years ago

From all I've heard from users, this issue has been resolved.

Thank you all for your comments!