Error while trying to execute list-targets or list-apps on macOS after updating

habiiev commented 3 years ago

Description

Updated to new macOS (11.6) and new Xcode (13.0) and started getting this error when trying to execute list-targets or list-apps commands:

2021-09-27 17:04:37,780 [ERROR] - root - Exception thrown in main
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion.py", line 71, in parse_json_line
    return json.loads(decoded_line)
  File "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 156, in spawn_notifier
    await self._read_notifier_output(stream=none_throws(process.stdout))
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 169, in _read_notifier_output
    update = parse_json_line(line)
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion.py", line 73, in parse_json_line
    raise IdbJsonException(f"Failed to parse json from: {decoded_line}")
idb.common.companion.IdbJsonException: Failed to parse json from:

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/idb/cli/main.py", line 310, in gen_main
    await root_command.run(args)
  File "/usr/local/lib/python3.9/site-packages/idb/common/command.py", line 90, in run
    return await self.resolve_command_from_args(args).run(args)
  File "/usr/local/lib/python3.9/site-packages/idb/cli/__init__.py", line 107, in run
    await self._run_impl(args)
  File "/usr/local/lib/python3.9/site-packages/idb/cli/__init__.py", line 127, in _run_impl
    async with _get_client(args=args, logger=self.logger) as client:
  File "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 181, in __aenter__
    return await self.gen.__anext__()
  File "/usr/local/lib/python3.9/site-packages/idb/cli/__init__.py", line 64, in _get_client
    async with ClientManager(
  File "/usr/local/Cellar/python@3.9/3.9.7/Frameworks/Python.framework/Versions/3.9/lib/python3.9/contextlib.py", line 181, in __aenter__
    return await self.gen.__anext__()
  File "/usr/local/lib/python3.9/site-packages/idb/grpc/management.py", line 103, in from_udid
    await self._spawn_notifier()
  File "/usr/local/lib/python3.9/site-packages/idb/grpc/management.py", line 57, in _spawn_notifier
    await companion_spawner.spawn_notifier()
  File "/usr/local/lib/python3.9/site-packages/idb/common/companion_spawner.py", line 159, in spawn_notifier
    raise CompanionSpawnerException(
idb.common.companion_spawner.CompanionSpawnerException: Failed to spawn the idb notifier. Stderr: 2021-09-27 17:04:22.519+0300 IDB Companion Built at Jun  2 2021 13:45:45
2021-09-27 17:04:22.520+0300 Invoked with args=[/usr/local/bin/idb_companion, --notify, /tmp/idb/targets] env={ long list of env's in here... }
2021-09-27 17:04:22.625+0300 Notifying /tmp/idb/targets
2021-09-27 17:04:22.625+0300 Providing targets across Simulator and Device sets.
2021-09-27 17:04:22.732+0300 CoreSimulator: Already loaded, skipping
2021-09-27 17:04:22.733+0300 CoreSimulator: SimDevice has correct path of /Library/Developer
2021-09-27 17:04:22.733+0300 AccessibilityPlatformTranslation: Already loaded, skipping
2021-09-27 17:04:22.734+0300 AccessibilityPlatformTranslation: AXPTranslationObject has correct path of /System/Library
2021-09-27 17:04:22.734+0300 Loaded All Private Frameworks [CoreSimulator, AccessibilityPlatformTranslation]
2021-09-27 17:04:37.540+0300 IDB Companion Built at Jun  2 2021 13:45:45
2021-09-27 17:04:37.541+0300 Invoked with args=[/usr/local/bin/idb_companion, --notify, /tmp/idb/targets] env={ long list of env's in here... }
2021-09-27 17:04:37.647+0300 Notifying /tmp/idb/targets
2021-09-27 17:04:37.647+0300 Providing targets across Simulator and Device sets.
2021-09-27 17:04:37.752+0300 CoreSimulator: Already loaded, skipping
2021-09-27 17:04:37.753+0300 CoreSimulator: SimDevice has correct path of /Library/Developer
2021-09-27 17:04:37.753+0300 AccessibilityPlatformTranslation: Already loaded, skipping
2021-09-27 17:04:37.754+0300 AccessibilityPlatformTranslation: AXPTranslationObject has correct path of /System/Library
2021-09-27 17:04:37.754+0300 Loaded All Private Frameworks [CoreSimulator, AccessibilityPlatformTranslation]

Reproduction

Install idb-companion using brew

brew tap facebook/fb
brew install idb-companion

Install idb cli using pip

# Tried both of these
sudo pip3.9 install fb-idb
pip3.9 install --user fb-idb

Run any of these commands:
```
idb list-targets
idb list-apps
```

Solution

...

Additional Information

idb-companion 1.1.3 (installed using brew)
idb cli 1.1.5 (installed using pip3.9)
macOS Big Sur 11.6

senosoft2009 commented 3 years ago

I'm experiencing the same problem & no solution yet

sjors98 commented 3 years ago

+1

LucianGalea commented 3 years ago

+1

hadnet commented 3 years ago

Same thing using Xcode 13.1

computerjazz commented 3 years ago

TLDR: Version 1.1.3 of idb-companion seems to be the issue, upgrading to 1.1.5 fixes it. Here's what worked for me:

brew tap --repair
brew update
brew upgrade

I finally figured out what was causing this error for me: brew was invisibly failing to upgrade my version of idb-companion because facebook changed their main branch name from master to main.

I realized when I called brew install idb-companion I was getting version 1.1.3, however, the current version is 1.1.5. I tried several times to reinstall, to brew tap facebook/fb, but nothing worked. Finally, I tried brew update and saw this:

fatal: couldn't find remote ref refs/heads/master
Error: Fetching /usr/local/Homebrew/Library/Taps/facebook/homebrew-fb failed!
Error: Some taps failed to update!
The following taps can not read their remote branches:
  facebook/fb
This is happening because the remote branch was renamed or deleted.
Reset taps to point to the correct remote branches by running `brew tap --repair`

So I ran brew tap --repair and saw:

==> facebook/fb: changed default branch name from master to main!

Once I did that I called brew update and saw:

Updated 1 tap (facebook/fb).
==> Updated Formulae
facebook/fb/idb-companion ✔

You have 2 outdated formulae installed.
You can upgrade them with brew upgrade
or list them with brew outdated.

And then finally called brew upgrade:

==> Upgrading 2 outdated packages:
facebook/fb/idb-companion 1.1.3 -> 1.1.5

After that, I was able to call idb list-targets and everything worked (including Flipper, which was what led me here in the first place).

bitcrumb commented 2 years ago

@computerjazz Thank you! This was indeed the fix. Looked around for a long while before stumbling upon your remarks. Thanks for writing this down! 🥵

lwmouneyrac commented 2 years ago

@computerjazz same as @bitcrumb here, thanks a lot!

MANTENN commented 2 years ago

On the new IDB cmd: "/usr/local/bin/idb_companion list-targets --json" is not a command...so it breaks flipper

elinnert commented 2 years ago

Hmm, I'm getting "No available targets" on my M1, running monterrey and xcode 13.. any idea what might be happening? Works fine on my mac mini, on Big Sur...

machadogj commented 2 years ago

Apart from what @computerjazz said, I had to also run the following command:

pip3 install --force-reinstall fb-idb

In my case the error message that I started getting after @computerjazz fix was:

➜  frames git:(feature/sign-up) ✗ idb list-targets
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.9/bin/idb", line 5, in <module>
    from idb.cli.main import main
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/idb/cli/__init__.py", line 29, in <module>
    from idb.grpc.client import Client as GrpcClient
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/idb/grpc/client.py", line 83, in <module>
    from idb.grpc.crash import (
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/idb/grpc/crash.py", line 10, in <module>
    from idb.grpc.idb_pb2 import (
  File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/idb/grpc/idb_pb2.py", line 20, in <module>
    _SETTING = DESCRIPTOR.enum_types_by_name['Setting']
AttributeError: 'NoneType' object has no attribute 'enum_types_by_name'

CoryWritesCode commented 2 years ago

Don't know if anyone else was running into this still but I was.

I ran idb list-targets --log DEBUG and found this error AMDeviceNotificationSubscribe failed with -402653085. After searching around I found the solution on microsoft's website which was to just run sudo killall -QUIT usbmuxd.

kylanhurt commented 2 months ago

This worked for me: sudo ln -s /opt/homebrew/bin/idb_companion /usr/local/bin

facebook / idb