User specified Python environment is not prioritized in Python Path

[mahaloz]

Version and Platform (required):

• Binary Ninja Version: 3.4.4133
• OS: Ubuntu
• OS Version: 22.04
• CPU Architecture: x64

Bug Description: When a user manually sets the "virtualenv" of their Binja's plugin, they expect that Python to have priority in package lookup against the builtin Binja Python; however, it is not prioritized.

Steps To Reproduce:

1. Open Binja and set up your Python to be something like this:
2. Restart Binja
3. Go to the Python interpreter in Binja and type import sys; sys.path, this should result in something like:

   ['/home/mahaloz/.binaryninja/repositories/official/plugins', '/home/mahaloz/.binaryninja/repositories/community/plugins', '/home/mahaloz/.binaryninja/plugins', '/home/mahaloz/.binaryninja/python310/site-packages', ...]

   Notice that the last entry in the path is /home/mahaloz/.binaryninja/python310/site-packages, which means when a user has a library installed on the Binja python, that one will always be chosen. Whenever a user sets their Python env manually, it's expected that their packages would be chosen when this happens.

Expected Behavior: Whenever a user sets their Python env manually, it's expected that their packages would be chosen when this happens. After they do this change, they should see their site packages before Binjas in the Path.

Screenshots: Show above.

Additional Information: Linking BinSync issue so both can close when this is fixed: https://github.com/binsync/binsync/issues/241

[fuzyll]

We had an internal discussion about this today. Just wanted to add a note here for clarification.

There are actually two problems at play here:

1. When the user changes their interpreter, if no python.virtualenv is set, we still default to our path.
2. Even when python.virtualenv is set, we still have our path set as well.

There's also an additional problem: Python ABIs are not stable across certain platform changes, which means our current system of versioning will potentially break in cases above where a user swaps from our bundled Python 3.10 to a system Python 3.10 install.

From a user perspective, the workflow we should move to should look something like:

• User goes to settings
• User selects new Python interpreter
• User is prompted to restart Binary Ninja to use the new interpreter
• On restart, we ensure our paths are not present and that we are using the paths from the new interpreter
• On init, prompt the user and let them know their plugin dependencies are likely no longer installed because their Python environment has changed
• Give the user a choice: "Would you like to re-install all plugin dependencies? Yes, No + Don't Ask Again checkbox"
• If they select "yes", they need to be prompted for what dependencies are about to be installed where and given a choice just like the existing workflow
• If something fails, the user needs to be notified so they can correct it themselves

We also need to probably add a hash to our Python target directory so that we can keep different Python 3.10 interpreters separate to address the above point about ABI mismatches. The proposed way of doing this is to introduce a hash of the Python interpreter path.

[brandon-fryslie]

This is a critical issue for people who are very experienced in Python and expect to be able to use standard python workflows (or custom workflows, or anything beyond installing everything directly from the plugin manager). For example, I want to have a Pipefile (using pipenv) that contains lists the packages for my Binja python installation. There is no reason Binja should need to be aware of or care about every python tool and workflow, and attempting to do so would be completely impractical. I figured that was why the fields on the settings page (seem to) allow you configure Python yourself.

The current behavior is unpredictable and basically non-functional. I've spent days diving deep into the details of site-packages and PYTHONPATH (again) to try to understand why I cannot properly configure Binja's python interpreter. I've tried a dozen ways or more, using system python, pyenv python, pipenv python, and multiple variations of each. I've tried working backwards, attempting to use using Binja's bundled python directly as my general interpreter. That's a terrible option, but I figured at least I wouldn't have to touch Binja's settings.

If I'm going to be writing Python software with Binja (plugins, snippets, etc) I need to be able to use Python from the command line (pip install, etc. not headless binja). I've spent too many years writing Python to go back to copying and pasting my code into a tiny window or managing all packages through Binja.

Binja is awesome and I love it, so please don't take my criticism the wrong way. It's a beautiful tool and way ahead of the competition imo (also the only one I can afford 😅 since RE is purely a hobby for me). I think Sidekick is great and I believe ML/LLMs have immense potential for RE (even if it hasn't reached its peak potential). Ultimately it's the reason I purchased Binary Ninja along with a year subscription to Sidekick. But that power is terribly hobbled by the very poor experience of actually writing code for Binary Ninja for someone who is an experienced software engineer but new to Binary Ninja. It's like trying to write a web application entirely in the Internet Explorer 6 developer tools (no offense).

This long winded post is really just me avoiding saying "me too". I'm also saying "this is more than a moderate impact" and that I personally don't see this as a "feature enhancement". If the intent is to let users configure their own python interpreter in a way that is compatible with Python then it's totally broken. It will run your interpreter, throw an inscrutable error (my favorite is ModuleNotFoundError: No module named 'encodings'), and then you get to try to figure out why Binja ignores it's own settings.

Anyway, I'm not going to stop using Binja or anything. I'll hack around it eventually. But it would be super nice if it worked out of the box. Here are my goals/plans, in case I'm off the rails somewhere:

• Currently, mostly trying to explore the API. For this I want to use a Jupyter notebook because the console is impossible to write any significant code in, while the snippet editor a) has no real way to view or print out values, and b) seems to stop working after running my code 2-3x
• Be able to interactively explore the data structures provided by Binja, which is functionality a Jupyter notebook provides (like a debugger that lets you step around arbitrarily and run any set of instructions at any time)
• Writing snippets to do things like "remove dead store" for all functions in a binary
• Eventually writing plugins (I'm new to RE but something like recursively inlining functions that only return the value of a function call seems to be something easier to accomplish in code than by hand)
• Use standard Python workflows for my code (Binja shouldn't really care but I use pipenv)
• Ideally writing some tests would be nice, if I'm going to release anything. This is going to require running cli commands, meaning a custom python interpreter

Here are some suggestions (since this is on your radar hopefully this will suffice rather than filing a separate tickets for related issues:

• Implement the bullet points in @fuzyll 's comment above. Whether that was the consensus of the team or their own interpretation, I believe that hits the nail on the head. They clearly know what they're talking about and I believe that would largely unshackle python, unblock a bunch of people who probably just gave up and never said anything, and get us to stop complaining for a while. 🙂 I believe addressing the bullet points above would make most/all of the following irrelevant
• Show the default values in or near the boxes so users have a more clear idea of what a valid value looks like
• Consider some validation on startup for those fields. It's quite easy to brick binja with a typo. A plugin complained about a value (path to interpreter), I changed it to something wrong, the plugin seemed to work just fine. then I quite and binary ninja refused to start until I manually edited my settings.json
• Consider explaining in which situations the fields are respected or ignored by Binja. For example, setting only the site-packages directory seems to be ignored by Binja, but NOT ignored by at least some plugins. This gets confusing
• Provide a few examples of using Binja's bundled python3 from the terminal (if this is even possible). That way if someone has given up on the custom interpreter idea, they can still run python commands w/o another wall of errors and debugging
• Consider letting users define a python script that will run once at startup, after Binja has initialized. That would allow a developer to fix any perceived problems themselves without the potential impact of changing how all of your python is loaded
• Explain in the settings page that setting site-packages requires setting a custom interpreter, and will be ignored if one isn't set. It's not clear that leaving one set to the default will cause a different setting to be ignored
• This is tougher to solve on your end, but it would be great if plugins didn't directly query settings for the python settings. Since Binja ignores them in some cases, it's easy to get into a situation where a plugin wants a value to be set, but it's tough to figure out which value will work with both Binja and the plugin

For now, I think I can work around the current blocker by adding a .pth file directly to the site-packages directory in the Application bundle (and re-signing the bundle). Crossing my fingers that maybe this work gets to the top of a backlog someday soon. Thanks for the great software, very much appreciated.

[brandon-fryslie]

Here's a screenshot of binary ninja ignoring my defined site-packages:

~Sadly adding a .pth file to the (previously non-existent) site-packages directory in the application bundle did not work. I also realized that I have no idea if there is actually a bundled python binary included in the application at all. I couldn't find anything. ~Would that mean it's using the system python?~ Can't be, that's 3.9.6... Is it just using whatever binary is on my path with the bundled 3.10 frameworks? That doesn't sound feasible from a support perspective. I'm on MacOS btw~

Whoops, I was conflating the "python path" (multiple directories) with "site-packages" (single directory). The .pth in the app bundle did work. Take care