Cache never finishes building

[jfthompson1]

Workflow version

2021.11

Alfred version

4.6.1 build 1274

macOS version

12.1

Debugger output

[15:19:56.258] Logging Started...
[15:20:04.735] Google Drive[Script Filter] Queuing argument 'j'
[15:20:04.886] Google Drive[Script Filter] Queuing argument 'ju'
[15:20:04.897] Google Drive[Script Filter] Script with argv 'j' finished
[15:20:04.900] Google Drive[Script Filter] {"items":[{"title":"Cache is being built…","subtitle":"Please be patient. Depending on how many files you have, it can take a while.","valid":false}]}
[15:20:04.963] Google Drive[Script Filter] Queuing argument 'jua'
[15:20:05.072] Google Drive[Script Filter] Queuing argument 'juan'
[15:20:05.100] Google Drive[Script Filter] Script with argv 'jua' finished
[15:20:05.102] Google Drive[Script Filter] {"items":[{"title":"Cache is being built…","subtitle":"Please be patient. Depending on how many files you have, it can take a while.","valid":false}]}
[15:20:05.234] Google Drive[Script Filter] Script with argv 'juan' finished
[15:20:05.238] Google Drive[Script Filter] {"items":[{"title":"Cache is being built…","subtitle":"Please be patient. Depending on how many files you have, it can take a while.","valid":false}]}

More details

Trying to use this workflow for the first time. Initiated cache build 3 days ago. When I enter a query, I get the message, "please be patient..". 3 days later I am still getting the same message. The google drive I am trying to index is approx. 980 GB. Any suggestions would be welcome.

[vitorgalvao]

approx. 980 GB

The size does not matter, only the number of paths. Ten 1MB files take ten times as long than one 10MB file. But three days sounds like something interrupted the process.

Restart your Mac, then run the following in a terminal:

osascript -e 'tell application id "com.runningwithcrayons.Alfred" to run trigger "googledrivelaunchd" in workflow "com.alfredapp.googledrive" with argument "uninstall"'
osascript -e 'tell application id "com.runningwithcrayons.Alfred" to run trigger "googledrivelaunchd" in workflow "com.alfredapp.googledrive" with argument "install"'

It will force a restart of the cache building. Be sure to have the debugger open when you do it.

[jfthompson1]

Ran two scripts as you directed. here is the output from the terminal: juan@Juans-MacBook-Pro ~ % osascript -e 'tell application id "com.runningwithcrayons.Alfred" to run trigger "googledrivelaunchd" in workflow "com.alfredapp.googledrive" with argument "uninstall"' juan@Juans-MacBook-Pro ~ % osascript -e 'tell application id "com.runningwithcrayons.Alfred" to run trigger "googledrivelaunchd" in workflow "com.alfredapp.googledrive" with argument "install"' juan@Juans-MacBook-Pro ~ %

Here is the output from the debugger:

[14:10:06.340] Logging Started... [14:10:42.646] Google Drive[External] Processing complete [14:10:42.650] Google Drive[External] Passing output 'uninstall' to Run Script [14:10:42.727] Google Drive[Run Script] Processing complete [14:10:42.732] Google Drive[Run Script] Passing output '' to Conditional [14:11:18.864] Google Drive[External] Processing complete [14:11:18.869] Google Drive[External] Passing output 'install' to Run Script [14:11:18.942] Google Drive[Run Script] Processing complete [14:11:18.946] Google Drive[Run Script] Passing output 'BUILD_CACHE' to Conditional [14:11:18.948] Google Drive[Conditional] Processing complete [14:11:18.949] Google Drive[Conditional] Passing output 'BUILD_CACHE' to Run Script [14:11:19.030] Google Drive[External] Processing complete [14:11:19.033] Google Drive[External] Passing output '' to Arg and Vars [14:11:19.034] Google Drive[Arg and Vars] Processing complete [14:11:19.035] Google Drive[Arg and Vars] Passing output '' to Run Script [14:11:19.158] Google Drive[Run Script] Processing complete [14:11:19.166] Google Drive[Run Script] Passing output '' to Run Script [14:11:19.168] Google Drive[Run Script] Passing output '' to Conditional [14:11:19.169] Google Drive[Conditional] Processing complete [14:11:19.169] Google Drive[Conditional] Passing output '' to Conditional [14:11:19.402] ERROR: Google Drive[Run Script] 'remote_info_plist' (https://raw.githubusercontent.com/alfredapp/google-drive-workflow/master/source/info.plist) appears to not be reachable.

The computer crashes about 1 minute after running the second script.

[jfthompson1]

Perhaps it is relevant that this is running on a 2021 Macbook Pro M1 with 16GB RAM. Here's the overview: Model Name: MacBook Pro Model Identifier: MacBookPro18,1 Chip: Apple M1 Pro Total Number of Cores: 10 (8 performance and 2 efficiency) Memory: 16 GB System Firmware Version: 7429.61.2 OS Loader Version: 7429.61.2 Activation Lock Status: Disabled

[jfthompson1]

First message I get in the Finder is :CommCenter wants to use the "login" keychain. If I enter it here, I get a long series of popup messages from different services asking the same question, then the computer crashes.

[vitorgalvao]

here is the output

Please take care to always use fenced code blocks when posting code.

Update to the latest version before continuing.

First message I get in the Finder is :CommCenter wants to use the "login" keychain. If I enter it here, I get a long series of popup messages from different services asking the same question, then the computer crashes.

That has nothing to do with the Workflow. Search for CommCenter online (example) and fix the crash before trying again with the instructions from the previous post.

[jfthompson1]

Updated to version 2021.12 of the workflow. Ran first script. Debugger output:

[08:13:13.193] Logging Started... [08:13:56.510] Google Drive[External] Processing complete [08:13:56.515] Google Drive[External] Passing output 'uninstall' to Run Script [08:13:56.593] Google Drive[Run Script] Processing complete [08:13:56.600] Google Drive[Run Script] Passing output '' to Conditional

posting this output now in case 2nd script causes my computer to crash again before I can post the output from the 2nd script.

[jfthompson1]

output from 2nd script: no crash, new error "no such file or directory".

[08:18:21.226] Google Drive[External] Processing complete [08:18:21.232] Google Drive[External] Passing output 'uninstall' to Run Script [08:18:21.238] STDERR: Google Drive[Run Script] Boot-out failed: 5: Input/output error Try re-running the command as root for richer errors. rm: /Users/juan/Library/LaunchAgents/com.alfredapp.launchd.googledrivecache.plist: No such file or directory [08:18:21.239] Google Drive[Run Script] Processing complete [08:18:21.240] Google Drive[Run Script] Passing output '' to Conditional

[vitorgalvao]

Please don’t post serially. This is not a chat, and having multiple replies in a row makes it really hard to follow. Plus, your last output was not from the second command, it was from the first.

You can’t do it one step at a time, you have to restart your Mac then run both commands and wait. The cache takes a while to build, so when your machine crashes everything was wasted. Until you fix the other problem with your macOS installation, there’s no point in continuing to try this.

[jfthompson1]

Hi Vítor, Ok, I'm not familiar with the use of GitHub for posting issues. I will consolidate my response in one comment. The commCenter messages (and following crash) only happen after I run the scripts.  So there is no issue to fix separate from these scripts.  I believe I have followed the instructions correctly. On the 24th, I restarted the Mac, started the Alfred  debugger, and ran both commands from the terminal. That is when the keychain messages began popping up, followed by a crash a minute or two later. I repeated the process, with the same result.  Today, I updated the workflow per your instructions, rebooted, and ran both scripts, one after the other. There were no error messages in the log this time, but after about a minute I got a pop-up, this time starting with CallHistoryPluginHelper. "Callhistorypluginhelper wants to use..." I did nothing, did not even click cancel. After another minute or so the computer shut down. I did not put the debit log results in GitHub yet because the computer crashed. I did copy the output to a note. Hopefully after restarting I can log in and retrieve the log and post it. Sincerely, Juan

Sent from Yahoo Mail on Android

On Mon, Dec 27, 2021 at 9:57 AM, Vítor @.***> wrote:

Please don’t reply serially. This is not a chat, and having multiple posts in a row makes it really hard to follow. Plus, your last output was not from the second command, it was from the first.

You can’t do it one step at a time, you have to restart your Mac then run both commands and wait. Those commands are resetting the cache rebuild. As I said before you need to fix your crash issue before doing it. The cache takes a while to build, so when your machine crashes, everything was wasted. Until you fix the other problem with your macOS installation, there’s no point in continuing to try this.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: @.***>

[jfthompson1]

Ok, restarted the computer, started Alfred workflow debugging, and ran both commands from terminal, one after the other. Below is the log from Alfred:

first script:

[10:57:34.369] Logging Started...
[10:57:58.735] Google Drive[External] Processing complete
[10:57:58.739] Google Drive[External] Passing output 'uninstall' to Run Script
[10:57:58.812] Google Drive[Run Script] Processing complete
[10:57:58.817] Google Drive[Run Script] Passing output '' to Conditional

2nd script:

[10:58:03.169] Google Drive[External] Processing complete
[10:58:03.172] Google Drive[External] Passing output 'install' to Run Script
[10:58:03.246] Google Drive[Run Script] Processing complete
[10:58:03.249] Google Drive[Run Script] Passing output 'BUILD_CACHE' to Conditional
[10:58:03.250] Google Drive[Conditional] Processing complete
[10:58:03.251] Google Drive[Conditional] Passing output 'BUILD_CACHE' to Run Script
[10:58:03.334] Google Drive[External] Processing complete
[10:58:03.336] Google Drive[External] Passing output '' to Arg and Vars
[10:58:03.337] Google Drive[Arg and Vars] Processing complete
[10:58:03.337] Google Drive[Arg and Vars] Passing output '' to Run Script
[10:58:03.461] Google Drive[Run Script] Processing complete
[10:58:03.473] Google Drive[Run Script] Passing output '' to Run Script
[10:58:03.475] Google Drive[Run Script] Passing output '' to Conditional
[10:58:03.477] Google Drive[Conditional] Processing complete
[10:58:03.479] Google Drive[Conditional] Passing output '' to Conditional

I waited. I did not take any action on the computer. After about a minute, got a pop-up that "CallHistoryPluginHelper wants access to your keychain". I did nothing, not even to click 'cancel' on the pop-up. After about another minute, the computer shut down.

Because the pop-ups and the following shutdown only appear after running these scripts, I do not see any way to troubleshoot this separately. I can contact Apple Support, but I expect they will say the problem lies with the script. I am not a developer, but what stands out to me in the report is this line: CORE 0 is the one that panicked I can provide the crash report if that would be helpful. How do you suggest proceeding?

[vitorgalvao]

got a pop-up that "CallHistoryPluginHelper wants access to your keychain"

That is yet again something different. All those commands are doing is setting up a launchd agent and running it. It has nothing to do with the KeyChain. Furthermore, it doesn’t make sense the crash would happen a minute after running the commands. Only thing that comes to mind is that something in your launchd is broken.

I can provide the crash report if that would be helpful.

It might.

How do you suggest proceeding?

What is the output of launchctl list | grep googledrivecache?

[jfthompson1]

Hi Vítor, I will run that grep in the next hour and will let you know the results. Two things occured to me: first, roughly how much disk space do I need to have free in order to build a cache for a Google Drive of about 1 TB and tens of thousands of files? I have 200 GB free now. Should that be sufficient? Not sure what is being cached. Second, my Google drive app is in ' streaming ' mode,  rather than ' offline ' mode, so that all the files are in the cloud only, and only the files I am using are downloaded on demand. Will your workflow handle that configuration? Sincerely, Juan

Sent from Yahoo Mail on Android

On Mon, Dec 27, 2021 at 11:51 AM, Vítor @.***> wrote:

got a pop-up that "CallHistoryPluginHelper wants access to your keychain"

That is yet again something different. All those commands are doing is setting up a launchd agent and running it. It has nothing to do with the KeyChain. Furthermore, it doesn’t make sense the crash would happen a minute after running the commands. Only thing that comes to mind is that something in your launchd is broken.

I can provide the crash report if that would be helpful.

It might.

How do you suggest proceeding?

What is the output of launchctl list | grep googledrivecache?

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: @.***>

[vitorgalvao]

I have 200 GB free now

More than enough.

Not sure what is being cached.

The paths themselves, as text, plus little associated information (e.g. if it’s a file or directory).

my Google drive app is in ' streaming ' mode, rather than ' offline ' mode, so that all the files are in the cloud only

The Workflow specifically addresses that configuration.

[jfthompson1]

FWIW, the only non-standard thing I can think of that I have installed on my Mac OS configuration is Homebrew, with which I installed 'coreutils'. I mention that in case it might affect the operation of your script.

here's the output from the command: launchctl list | grep googledrivecache

output: - 0 com.alfredapp.launchd.googledrivecache

below is the crash report generated when the computer crashes:

Crash Report

Debugger message: panic
Memory ID: 0x1
OS release type: User
OS version: 21C52
Kernel version: Darwin Kernel Version 21.2.0: Sun Nov 28 20:28:41 PST 2021; root:xnu-8019.61.5~1/RELEASE_ARM64_T6000
Fileset Kernelcache UUID: 88208EF24C07AA8B02CDA1369E036EBD
Kernel UUID: 2F4E22DC-64BE-3369-B574-0909F0571FA6
iBoot version: iBoot-7429.61.2
secure boot?: YES
Paniclog version: 13
KernelCache slide: 0x000000000fe10000
KernelCache base:  0xfffffe0016e14000
Kernel slide:      0x000000001063c000
Kernel text base:  0xfffffe0017640000
Kernel text exec slide: 0x0000000010724000
Kernel text exec base:  0xfffffe0017728000
mach_absolute_time: 0x3717b44fd
Epoch Time:        sec       usec
  Boot    : 0x61c9e158 0x0009d408
  Sleep   : 0x00000000 0x00000000
  Wake    : 0x00000000 0x00000000
  Calendar: 0x61c9e3ba 0x000d14ce

Zone info:
Foreign   : 0xfffffe00202fc000 - 0xfffffe0020310000
Native    : 0xfffffe100060c000 - 0xfffffe300060c000
Readonly  : 0xfffffe14cd2d8000 - 0xfffffe1666c6c000
Metadata  : 0xfffffe74811b0000 - 0xfffffe748d134000
Bitmaps   : 0xfffffe748d134000 - 0xfffffe7492b6c000
Largest zones:
Zone Name                      Cur Size  Free Size
kext.   kalloc.32768              5256M         0K
        vnodes                      60M        11K
        APFS_INODES                 44M     11546K
        namecache                   24M        38K
kext.   kalloc.16384                 4M         0K
CORE 0 PVH locks held: None
CORE 1 PVH locks held: None
CORE 2 PVH locks held: None
CORE 3 PVH locks held: None
CORE 4 PVH locks held: None
CORE 5 PVH locks held: None
CORE 6 PVH locks held: None
CORE 7 PVH locks held: None
CORE 8 PVH locks held: None
CORE 9 PVH locks held: None
CORE 0 is the one that panicked. Check the full backtrace for details.
CORE 1: PC=0xfffffe00178af27c, LR=0xfffffe00178af278, FP=0xfffffe6fee42be80
CORE 2: PC=0xfffffe00177aef18, LR=0xfffffe00177aef18, FP=0xfffffe6fee01bef0
CORE 3: PC=0xfffffe00177aef1c, LR=0xfffffe00177aef18, FP=0xfffffe6feb81bef0
CORE 4: PC=0xfffffe00177aef18, LR=0xfffffe00177aef18, FP=0xfffffe6feb86bef0
CORE 5: PC=0xfffffe00177aef18, LR=0xfffffe00177aef18, FP=0xfffffe6fe9f2bef0
CORE 6: PC=0xfffffe00177aef18, LR=0xfffffe00177aef18, FP=0xfffffe60a594bef0
CORE 7: PC=0xfffffe00177aef18, LR=0xfffffe00177aef18, FP=0xfffffe6fec50bef0
CORE 8: PC=0xfffffe00177aef18, LR=0xfffffe00177aef18, FP=0xfffffe6feb7cbef0
CORE 9: PC=0xfffffe00177aef18, LR=0xfffffe00177aef18, FP=0xfffffe6fee2ebef0
Total cpu_usage: 12618441
Thread task pri cpu_usage
0xfffffe2999e15040 kernel_task 0 5322470
0xfffffe2999778100 kernel_task 0 5770876
0xfffffe2999779140 kernel_task 0 3
0xfffffe299977b1c0 kernel_task 0 8
0xfffffe2999775860 kernel_task 0 15

Panicked task 0xfffffe2999e8c678: 0 pages, 517 threads: pid 0: kernel_task
Panicked thread: 0xfffffe2999e15040, backtrace: 0xfffffe6017af7830, tid: 102
          lr: 0xfffffe001777a4e4  fp: 0xfffffe6017af78a0
          lr: 0xfffffe001777a1b4  fp: 0xfffffe6017af7910
          lr: 0xfffffe00178ba070  fp: 0xfffffe6017af7930
          lr: 0xfffffe00178ac47c  fp: 0xfffffe6017af79b0
          lr: 0xfffffe00178a9ecc  fp: 0xfffffe6017af7a70
          lr: 0xfffffe001772f7f8  fp: 0xfffffe6017af7a80
          lr: 0xfffffe0017779e28  fp: 0xfffffe6017af7e20
          lr: 0xfffffe0017779e28  fp: 0xfffffe6017af7e90
          lr: 0xfffffe0017f8faf8  fp: 0xfffffe6017af7eb0
          lr: 0xfffffe001815f84c  fp: 0xfffffe6017af7ee0
          lr: 0xfffffe001815ec34  fp: 0xfffffe6017af7f20
          lr: 0xfffffe001815c6a8  fp: 0xfffffe6017af7f40
          lr: 0xfffffe0018951e24  fp: 0xfffffe6017af7fd0
          lr: 0xfffffe00178acfd0  fp: 0xfffffe6017af7fe0
          lr: 0xfffffe001772f86c  fp: 0xfffffe6017af7ff0
          lr: 0xfffffe00177aef18  fp: 0xfffffe6feb563ef0
          lr: 0xfffffe00177af130  fp: 0xfffffe6feb563f20
          lr: 0xfffffe0017738e78  fp: 0x0000000000000000
      Kernel Extensions in backtrace:
         com.apple.driver.AppleInterruptControllerV2(1.0d1)[D3C6594C-832C-3488-837F-28F5AC4E754C]@0xfffffe00189503f0->0xfffffe0018952d27
            dependency: com.apple.driver.AppleARMPlatform(1.0.2)[1C04FFC7-DD27-3161-A460-F936CDBDBA3E]@0xfffffe0018112df0->0xfffffe001815b58f
         com.apple.driver.AppleARMWatchdogTimer(1.0)[C9B89662-4FED-3B89-88C4-3BD5B8AD87CF]@0xfffffe001815b590->0xfffffe001815fd1b
            dependency: com.apple.driver.AppleARMPlatform(1.0.2)[1C04FFC7-DD27-3161-A460-F936CDBDBA3E]@0xfffffe0018112df0->0xfffffe001815b58f

last started kext at 924009706: com.apple.filesystems.smbfs 4.0 (addr 0xfffffe0017626d60, size 64483)
loaded kexts:
com.apple.filesystems.smbfs 4.0
com.apple.filesystems.autofs    3.0
com.apple.fileutil  20.036.15
com.apple.driver.AppleBiometricServices 1
com.apple.driver.CoreKDL    1
com.apple.driver.AppleTopCaseHIDEventDriver 5020.1
com.apple.driver.SEPHibernation 1
com.apple.driver.BCMWLANFirmware4387.Hashstore  1
com.apple.driver.DiskImages.ReadWriteDiskImage  493.0.0
com.apple.driver.DiskImages.UDIFDiskImage   493.0.0
com.apple.driver.DiskImages.RAMBackingStore 493.0.0
com.apple.driver.DiskImages.FileBackingStore    493.0.0
com.apple.filesystems.apfs  1933.61.1
com.apple.driver.AppleUSBDeviceNCM  5.0.0
com.apple.driver.AppleThunderboltIP 4.0.3
com.apple.driver.AppleFileSystemDriver  3.0.1
com.apple.nke.l2tp  1.9
com.apple.filesystems.tmpfs 1
com.apple.filesystems.lifs  1
com.apple.IOTextEncryptionFamily    1.0.0
com.apple.filesystems.hfs.kext  582.60.2
com.apple.security.BootPolicy   1
com.apple.BootCache 40
com.apple.AppleFSCompression.AppleFSCompressionTypeZlib 1.0.0
com.apple.AppleEmbeddedSimpleSPINORFlasher  1
com.apple.AppleFSCompression.AppleFSCompressionTypeDataless 1.0.0d1
com.apple.driver.AppleCS42L84Audio  520.5
com.apple.driver.ApplePMP   1
com.apple.driver.AppleSN012776Amp   520.5
com.apple.driver.AppleSmartIO2  1
com.apple.driver.AppleT6000SOCTuner 1
com.apple.driver.AppleT6000CLPCv3   1
com.apple.driver.AppleSmartBatteryManager   161.0.0
com.apple.driver.AppleALSColorSensor    1.0.0d1
com.apple.driver.AppleAOPVoiceTrigger   100.1
com.apple.driver.AppleDPDisplayTCON 1
com.apple.driver.ApplePMPFirmware   1
com.apple.driver.usb.AppleSynopsysUSB40XHCI 1
com.apple.driver.AppleSDXC  3.1.1
com.apple.AGXG13X   187.59
com.apple.driver.AppleSamsungSerial 1.0.0d1
com.apple.driver.AppleMCDP29XXUpdateSupport 1
com.apple.driver.AppleSerialShim    1
com.apple.driver.AppleS5L8960XNCO   1
com.apple.driver.AppleAVD   416
com.apple.driver.AppleAVE2  520.21.1
com.apple.driver.AppleEventLogHandler   1
com.apple.driver.AppleProResHW  126.2.0
com.apple.driver.AppleM68Buttons    1.0.0d1
com.apple.driver.AppleJPEGDriver    4.7.9
com.apple.driver.AppleSPMIPMU   1.0.1
com.apple.driver.AppleMobileDispT600X-DCP   140.0
com.apple.driver.AppleT6000PMGR 1
com.apple.driver.AppleS8000AES  1
com.apple.driver.AppleS8000DWI  1.0.0d1
com.apple.driver.AppleInterruptControllerV2 1.0.0d1
com.apple.driver.AppleT8110DART 1
com.apple.driver.AppleBluetoothModule   1
com.apple.driver.AppleBCMWLANBusInterfacePCIe   1
com.apple.driver.AppleS5L8920XPWM   1.0.0d1
com.apple.driver.AudioDMAController-T600x   100.51
com.apple.driver.AppleT6000DART 1
com.apple.driver.AppleSPIMC 1
com.apple.driver.AppleS5L8940XI2C   1.0.0d2
com.apple.driver.AppleT6000 1
com.apple.iokit.IOUserEthernet  1.0.1
com.apple.driver.usb.AppleUSBUserHCI    1
com.apple.iokit.IOKitRegistryCompatibility  1
com.apple.iokit.EndpointSecurity    1
com.apple.driver.AppleDiskImages2   126.60.3
com.apple.AppleSystemPolicy 2.0.0
com.apple.nke.applicationfirewall   402
com.apple.kec.InvalidateHmac    1
com.apple.kec.AppleEncryptedArchive 1
com.apple.driver.driverkit.serial   6.0.0
com.apple.kext.triggers 1.0
com.apple.iokit.IOAVBFamily 1010.2
com.apple.plugin.IOgPTPPlugin   1000.11
com.apple.iokit.IOEthernetAVBController 1.1.0
com.apple.driver.AppleMesaSEPDriver 100.99
com.apple.iokit.IOBiometricFamily   1
com.apple.driver.AppleHSBluetoothDriver 5020.1
com.apple.driver.IOBluetoothHIDDriver   9.0.0
com.apple.driver.AppleActuatorDriver    5430.1
com.apple.driver.AppleMultitouchDriver  5430.1
com.apple.driver.AppleHIDKeyboard   228
com.apple.driver.AppleTrustedAccessory  1
com.apple.iokit.AppleSEPGenericTransfer 1
com.apple.driver.AppleSEPHDCPManager    1.0.1
com.apple.driver.DiskImages.KernelBacked    493.0.0
com.apple.driver.AppleXsanScheme    3
com.apple.driver.usb.networking 5.0.0
com.apple.driver.AppleThunderboltUSBDownAdapter 1.0.4
com.apple.driver.AppleThunderboltDPInAdapter    8.5.1
com.apple.driver.AppleThunderboltDPAdapterFamily    8.5.1
com.apple.driver.AppleThunderboltPCIDownAdapter 4.1.1
com.apple.driver.AppleHIDTransportSPI   5400.30
com.apple.driver.AppleHIDTransport  5400.30
com.apple.driver.AppleInputDeviceSupport    5400.30
com.apple.nke.ppp   1.9
com.apple.driver.AppleDCPDPTXProxy  1.0.0
com.apple.driver.AppleConvergedIPCOLYBTControl  1
com.apple.driver.AppleConvergedPCI  1
com.apple.driver.AppleBluetoothDebug    1
com.apple.driver.AppleBSDKextStarter    3
com.apple.driver.AppleBTM   1.0.1
com.apple.driver.DCPDPFamilyProxy   1
com.apple.filesystems.hfs.encodings.kext    1
com.apple.driver.AppleDiagnosticDataAccessReadOnly  1.0.0
com.apple.driver.AppleCSEmbeddedAudio   520.5
com.apple.driver.ApplePassthroughPPM    3.0
com.apple.driver.AppleAOPAudio  102.2
com.apple.driver.AppleEmbeddedAudio 520.5
com.apple.iokit.AppleARMIISAudio    100.1
com.apple.driver.AppleSPU   1
com.apple.driver.DCPAVFamilyProxy   1
com.apple.driver.AppleUSBXDCIARM    1.0
com.apple.driver.AppleUSBXDCI   1.0
com.apple.iokit.IOUSBDeviceFamily   2.0.0
com.apple.driver.usb.AppleSynopsysUSBXHCI   1
com.apple.driver.usb.AppleUSBXHCI   1.2
com.apple.driver.AppleEmbeddedUSBHost   1
com.apple.driver.usb.AppleUSBHub    1.2
com.apple.driver.usb.AppleUSBHostCompositeDevice    1.2
com.apple.AGXFirmwareKextG13XRTBuddy    187.59
com.apple.AGXFirmwareKextRTBuddy64  187.59
com.apple.iokit.IONVMeFamily    2.1.0
com.apple.driver.AppleNANDConfigAccess  1.0.0
com.apple.driver.AppleSART  1
com.apple.iokit.IOGPUFamily 35.11
com.apple.driver.AppleHPM   3.4.4
com.apple.driver.AppleT6000TypeCPhy 1
com.apple.driver.AppleT8103TypeCPhy 1
com.apple.driver.AppleDialogPMU 1.0.1
com.apple.driver.AppleStockholmControl  1.0.0
com.apple.driver.AppleSPMI  1.0.1
com.apple.iokit.IOMobileGraphicsFamily-DCP  343.0.0
com.apple.driver.AppleDCP   1
com.apple.driver.AppleFirmwareKit   1
com.apple.iokit.IOMobileGraphicsFamily  343.0.0
com.apple.driver.ApplePMGR  1
com.apple.driver.usb.AppleUSBHostPacketFilter   1.0
com.apple.driver.AppleARMWatchdogTimer  1
com.apple.driver.AppleDisplayCrossbar   1.0.0
com.apple.iokit.IODisplayPortFamily 1.0.0
com.apple.driver.AppleTypeCPhy  1
com.apple.driver.AppleThunderboltNHI    7.2.8
com.apple.driver.AppleT6000PCIeC    1
com.apple.iokit.IOThunderboltFamily 9.3.3
com.apple.driver.ApplePIODMA    1
com.apple.driver.AppleT600xPCIe 1
com.apple.driver.AppleMultiFunctionManager  1
com.apple.driver.AppleBluetoothDebugService 1
com.apple.driver.AppleBCMWLANCore   1.0.0
com.apple.iokit.IO80211Family   1200.12.2b1
com.apple.driver.IOImageLoader  1.0.0
com.apple.driver.AppleOLYHAL    1
com.apple.driver.corecapture    1.0.4
com.apple.driver.AppleEmbeddedPCIE  1
com.apple.driver.AppleMCA2-T600x    600.95
com.apple.driver.AppleEmbeddedAudioLibs 100.9.1
com.apple.driver.AppleFirmwareUpdateKext    1
com.apple.driver.AppleH13CameraInterface    4.87.0
com.apple.driver.AppleH10PearlCameraInterface   17.0.3
com.apple.driver.AppleGPIOICController  1.0.2
com.apple.driver.AppleFireStormErrorHandler 1
com.apple.driver.AppleMobileApNonce 1
com.apple.iokit.IOTimeSyncFamily    1000.11
com.apple.driver.DiskImages 493.0.0
com.apple.iokit.IOGraphicsFamily    593
com.apple.iokit.IOBluetoothSerialManager    9.0.0
com.apple.iokit.IOBluetoothHostControllerUSBTransport   9.0.0
com.apple.iokit.IOBluetoothHostControllerUARTTransport  9.0.0
com.apple.iokit.IOBluetoothHostControllerTransport  9.0.0
com.apple.driver.IOBluetoothHostControllerPCIeTransport 9.0.0
com.apple.iokit.IOBluetoothFamily   9.0.0
com.apple.driver.FairPlayIOKit  68.13.1
com.apple.iokit.CSRBluetoothHostControllerUSBTransport  9.0.0
com.apple.iokit.BroadcomBluetoothHostControllerUSBTransport 9.0.0
com.apple.driver.AppleSSE   1.0
com.apple.driver.AppleSEPKeyStore   2
com.apple.driver.AppleUSBTDM    532.40.7
com.apple.iokit.IOUSBMassStorageDriver  209.40.6
com.apple.iokit.IOPCIFamily 2.9
com.apple.iokit.IOSCSIBlockCommandsDevice   452.60.2
com.apple.iokit.IOSCSIArchitectureModelFamily   452.60.2
com.apple.driver.AppleIPAppender    1.0
com.apple.driver.AppleFDEKeyStore   28.30
com.apple.driver.AppleEffaceableStorage 1.0
com.apple.driver.AppleCredentialManager 1.0
com.apple.driver.KernelRelayHost    1
com.apple.iokit.IOUSBHostFamily 1.2
com.apple.driver.AppleUSBHostMergeProperties    1.2
com.apple.driver.usb.AppleUSBCommon 1.0
com.apple.driver.AppleSMC   3.1.9
com.apple.driver.RTBuddy    1.0.0
com.apple.driver.AppleEmbeddedTempSensor    1.0.0
com.apple.driver.AppleARMPMU    1.0
com.apple.iokit.IOAccessoryManager  1.0.0
com.apple.driver.AppleOnboardSerial 1.0
com.apple.iokit.IOSkywalkFamily 1.0
com.apple.driver.mDNSOffloadUserClient  1.0.1b8
com.apple.iokit.IONetworkingFamily  3.4
com.apple.iokit.IOSerialFamily  11
com.apple.driver.AppleSEPManager    1.0.1
com.apple.driver.AppleA7IOP 1.0.2
com.apple.driver.IOSlaveProcessor   1
com.apple.driver.AppleBiometricSensor   2
com.apple.iokit.IOHIDFamily 2.0.0
com.apple.iokit.CoreAnalyticsFamily 1
com.apple.driver.AppleANELoadBalancer   5.34.2
com.apple.driver.AppleH11ANEInterface   5.34.0
com.apple.AUC   1.0
com.apple.iokit.IOAVFamily  1.0.0
com.apple.iokit.IOHDCPFamily    1.0.0
com.apple.iokit.IOCECFamily 1
com.apple.iokit.IOAudio2Family  1.0
com.apple.driver.AppleIISController 100.1
com.apple.driver.AppleAudioClockLibs    100.9.1
com.apple.driver.AppleM2ScalerCSCDriver 265.0.0
com.apple.iokit.IOSurface   302.9
com.apple.driver.IODARTFamily   1
com.apple.security.quarantine   4
com.apple.security.sandbox  300.0
com.apple.kext.AppleMatch   1.0.0d1
com.apple.driver.AppleMobileFileIntegrity   1.0.5
com.apple.security.AppleImage4  4.2.0
com.apple.kext.CoreTrust    1
com.apple.iokit.IOCryptoAcceleratorFamily   1.0.1
com.apple.driver.AppleARMPlatform   1.0.2
com.apple.iokit.IOStorageFamily 2.1
com.apple.iokit.IOSlowAdaptiveClockingFamily    1.0.0
com.apple.iokit.IOReportFamily  47
com.apple.kec.pthread   1
com.apple.kec.Libm  1
com.apple.kec.corecrypto    12.0

** Stackshot Succeeded ** Bytes Traced 47622 (Uncompressed 170848) **```

[vitorgalvao]

I don’t see anything in the crash that looks relevant.

here's the output from the command

In that case, run:

rm "${HOME}/Library/Caches/com.runningwithcrayons.Alfred/Workflow Data/com.alfredapp.googledrive/tmp.db"

If it succeeds, reboot and go about your day. About one or two hours in, do gd in Alfred and see if it works.

[jfthompson1]

Ok, the command executed without any errors. Rebooting and I will let it run overnight and test it in the morning. On Monday, December 27, 2021, 08:26:18 PM EST, Vítor Galvão @.***> wrote:

I don’t see anything in the crash that looks relevant.

here's the output from the command

In that case, run: rm "${HOME}/Library/Caches/com.runningwithcrayons.Alfred/Workflow Data/com.alfredapp.googledrive/tmp.db" If it succeeds, reboot and go about your day. About one or two hours in, do gd in Alfred and see if it works.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: @.***>

[jfthompson1]

Opened Alfred, typed: gd Juan (because I know there are files and directories that include that name) Alfred returned the message: No cache found!

Last night, after executing the command and rebooting the computer per your instructions, I checked the folder, com.alfredapp.googledrive. It was empty.

[vitorgalvao]

Indeed, this only reinforces the issue is somewhere in your macOS setup/installation and not the Workflow. Though we already knew that because from the Workflow’s many users you’re the only experiencing the problem. I’m still not sure where exactly your setup is bad but it seems to be somewhere around launchd. The rest you’ll have to figure out yourself, since the issue isn’t related to either Alfred or the Workflow and I cannot reproduce.

You can still try to use the Workflow by manually generating the cache via :gdrebuildcache.

[jfthompson1]

Hmmm. I see what you mean, and yet I only have this issue when I run these scripts. It seems to be an interaction between your scripts my OS. The options I can think of are:1. Do a clean reinstall of the MacOS (time-consuming to reinstall everything)2. Contact Apple Support (I am doubtful that they will be able or willing to troubleshoot this issue)3. Choose another cloud provider that spotlight does integrate with in the 'optimized' mode, rather than with all files stored locally.  It sounds like Dropbox is one such provider. Limited research indicates that even iCloud does not support indexing if the files are not mirrored on the local drive.  Any thoughts? Juan On Tuesday, December 28, 2021, 08:45:13 AM EST, Vítor Galvão @.***> wrote:

Indeed, this only reinforces the issue is somewhere in your macOS setup/installation and not the Workflow. Though we already knew that because from the Workflow’s many users you’re the only experiencing the problem. I’m still not sure where exactly your setup is bad but it seems to be somewhere around launchd. The rest you’ll have to figure out yourself, since the issue isn’t related to either Alfred or the Workflow and I cannot reproduce.

You can still try to use the Workflow by manually generating the cache via :gdrebuildcache.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: @.***>

[vitorgalvao]

It seems to be an interaction between your scripts my OS.

Not necessarily. The script calls regular macOS tools, so presumably if you did that on your own in another capacity, the issue would still happen. It’s unlikely the script is causing the issue, but that launchd (which the script interacts with) is the source of the problem.

Think of it like having a car (the scripts) which you drive over a specific road (launchd) and always get a flat tire. Your reply assumes a combination of the car and road is causing the flat tire, when the likeliest answer is only the road is to blame; you just never tried going over that particular road with a different car.

Any thoughts?

I agree with your assessment in the three points, including the caveat in 2.

Limited research indicates that even iCloud does not support indexing if the files are not mirrored on the local drive.

I can’t say one way or another, as I never tried it. I would find it odd for that to be the case, though, since they’re both controlled by Apple.

Worth noting that on the Alfred Forums, the only regular complaint of missing indexing is about Google Drive.

[jfthompson1]

Thanks for the feedback. Perhaps it is worth opening a ticket with Apple, and perhaps also doing a search for the terms launchd, osascript and system crashes. I would hate to reinstall the OS, only to find that the problem still occurs because it is a code issue with launchd. Are there other standard (ideally Apple) scripts that I could run to see if I have this issue with other scripts? That would be very helpful. Juan On Tuesday, December 28, 2021, 10:59:29 AM EST, Vítor Galvão @.***> wrote:

It seems to be an interaction between your scripts my OS.

Not necessarily. The script calls regular macOS tools, so presumably if you did that on your own in another capacity, the issue would still happen. It’s unlikely the script is causing the issue, but that launchd (which the script interacts with) is the source of the problem.

Think of it like having a car (the scripts) which you drive over a specific road (launchd) and always get a flat tire. Your reply assumes a combination of the car and road is causing the flat tire, when the likeliest answer is only the road is to blame; you just never tried going over that particular road with a different car.

Any thoughts?

I agree with your assessment in the three points, including the caveat in 2.

Limited research indicates that even iCloud does not support indexing if the files are not mirrored on the local drive.

I can’t say one way or another, as I never tried it. I would find it odd for that to be the case, though, since they’re both controlled by Apple.

Worth noting that on the Alfred Forums, the only regular complaint of missing indexing is about Google Drive.

— Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android. You are receiving this because you authored the thread.Message ID: @.***>

[MWYang]

I've been having the same problem as @jfthompson1, including the weird "CommCenter" keychain errors. I'm also running a M1 MacBook and have a largish number of paths (find "/Volumes/GoogleDrive/My Drive" | wc -l gives me ~250K).

I believe the problem is due to Ruby glob trying to load all of its paths inefficiently into memory. I replaced glob by calling the builtin system find command instead.

I also had to add a default access time (Jan 1, 1970) instead of skipping a file completely, since path.atime.to_i seemed to fail for every path in my Google Drive.

I will add a PR for these changes shortly.

[vitorgalvao]

I believe the problem is due to Ruby glob trying to load all of its paths inefficiently into memory.

I wouldn’t expect it to manifest how it has, though.

I replaced glob by calling the builtin system find command instead.

Try the Find module first. Let’s keep it Ruby where we can rather than shelling out and having to interpret that.

I also had to add a default access time (Jan 1, 1970) instead of skipping a file completely, since path.atime.to_i seemed to fail for every path in my Google Drive.

That seems like another issue caused by the interaction between macOS and Google Drive. But this Workflow is meant to address that, so sure.

[vitorgalvao]

@jfthompson1 @MWYang The attached version changes few things but should work by using the Find module instead of glob. Could you both try it out and see if it works for you?

Google Drive.alfredworkflow.zip

[MWYang]

Thanks for fast response. Unfortunately, this new version still seems to create the same or similar issue as before — just as with the current release, my computer hangs and crashes shortly after running the script.

I wouldn’t expect it to manifest how it has, though.

I share your skepticism – 250K paths shouldn't be too much for modern computers. But IMO the evidence still points to a problem when trying to do all of the path scanning purely in Ruby (plus interaction with large number of paths and/or M1). I'd want to look into this further, but I'm not sure of the best way forward.

This post suggests using IO.popen in combination with find. That would avoid the creation of a temporary text file with the find results.

EDIT: Clarification

[vitorgalvao]

But IMO the evidence still points to a problem when trying to do all of the path scanning purely in Ruby (plus interaction with large number of paths and/or M1).

I’m not convinced yet. I’m also on an M1 and I can run the Workflow on the whole drive (/) which is over 4 million files and there is no crash. That’s with both the previous and current solutions. Plus, we can easily test that using find does indeed return things as it finds them rather than building the list first. Just compare:

require 'pathname'
Pathname.new(ENV['HOME']).glob('**/*').each { |a| puts a }

to

require 'pathname'
Pathname.new(ENV['HOME']).find { |a| puts a }

There is a noticeable delay before the first one begins to show files, while the second one is instant.

I'd want to look into this further, but I'm not sure of the best way forward.

Because I cannot reproduce, it’s near impossible to work on next steps. I’d need a reproducible case or know what is particular about both of your setups.

I recommended to @jfthompson1 using :gdrebuildcache but got no reply if it causes the same issue or not. Does doing that crash it for you?

[MWYang]

Yes, I did try only using :gdrebuildcache and got the same behavior. And then I narrowed the cause down to the glob and database loop by running a modified version of the :gdrebuildcache script on its own outside of Alfred. I'm very positive the issue is with running that specific block of code, but I'm also still not sure what exactly the mechanism is.

What led me to further think this is some kind of memory issue is that I saw ruby posting multiple GB (I think close to 10 GB one time before my computer froze) on the Disk section of Activity Monitor. That struck me as very weird, since the finished cache DB is only <50 MB.

But RAM usage also looked reasonable up until crash. So I'm not too sure either.

I'm happy to help you debug further, since I get that you can't do much without being able to reproduce the issue. But I also don't know what to try next.

[vitorgalvao]

What about this one?

[MWYang]

Ah, unfortunately still doesn't work with the same error. I should've mentioned earlier than I only have 8 GB of RAM, so even though I have many fewer files, my ceiling is lower.

When I wrote PR #11, I double-checked that Ruby reads the written-to-disk list of files one line at a time (ofc there were other flaws with this PR). Maybe the way Ruby handles file reads immediately discards previous lines in a way that the .find solution doesn't?

[vitorgalvao]

I have high hopes for this one.

But if that doesn’t work, here’s a followup which should be considerably slower.

And here’s a question for both of you: how many free disk space do you have?

[MWYang]

My hard drive has ~125 GB free out of 256 GB.

The first one using just .lazy still led to the same problem. The second one manually starting the garbage collection did finish building the cache successfully (i.e. no forced restart) but still caused my computer to be frozen and inoperable (i.e. multiple keychain login windows popped up, Mac menu bar gone, trackpad inoperable). It also ran much slower, as you thought it would.

I tried the second one again but changed the script to only call garbage collection every 1000 loops. Doing that still resulted in a bunch of keychain login windows, but the script did finish successfully, and after manually completing each of the login windows, my computer seemed to be fully operational again. Still, it's not something I would like to do regularly.

[vitorgalvao]

This still seems to be something related to your particular setups. I’m planning on making another release soon, though it won’t fix this. I don’t want to follow the solution in the PR because the true problem isn’t clear and I don’t want to make the code possibly worse for everyone else to fix an uncertain issue faced by a single person.

So much the better if we can find the definitive cause and solution in a way that is idiomatic and doesn’t affect anyone else. Until then, I did offer a working solution that works for you despite being slower and you also have the code from your PR. There’s no problem in changing those for your particular setup.

Thank you for the investigation so far.