Unsafe Python Environment

[XVicarious]

Jedi by default seems to require the python executable to owned by the root user for safety reasons. It appears without setting the safe parameter in create_environment to False, you will be unable to use a virtual environment for your projects.

The relevant line is https://github.com/CoatiSoftware/SourcetrailPythonIndexer/blob/8b1dc111cdd3297cf162f08c9b239295d56c049c/indexer.py#L28

CoatiSoftware/Sourcetrail/issues/697

[mlangkabel]

Thank you for looking into it! So I think this is the relevant comment in the Jedi code:

    # The interpreter needs to be owned by root. This means that it wasn't
    # written by a user and therefore attacking Jedi is not as simple.
    # The attack could look like the following:
    # 1. A user clones a repository.
    # 2. The repository has an innocent looking folder called foobar. jedi
    #    searches for the folder and executes foobar/bin/python --version if
    #    there's also a foobar/bin/activate.
    # 3. The bin/python is obviously not a python script but a bash script or
    #    whatever the attacker wants.

[XVicarious]

That is true, however its common practice to use a virtual environment with something like pyenv for individual projects to keep everything just right, and they're owned by the user.

Perhaps you can use get_default_environment. For reference: jedi-vim

[mlangkabel]

So if I understand it correctly, the Jedi comment is talking about a user that check out a repository that already contains a virtual python environment (which may be a safety risk)?

[XVicarious]

That seems to be the reasoning to me. I can confirm it works with my virtual environment when I'm using the jedi-vim plugin.

[mlangkabel]

Ok, then I would suggest that we use only safe environments if the user doesn't specify a specific one, but if the user explicitly provide one (via the Sourcetrail project settings), also "unsafe" environments shall be allowed.

[XVicarious]

Sounds like a fair compromise to me.

[mlangkabel]

fixed with merge of pull request #60

[robinst]

This doesn't seem to be released yet. Is there a nightly build available to try out?

[mlangkabel]

That's right. It has been fixed in the code. Now we need to create a new SourcetrailPythonIndexer release. You could download and extract that release and replace the data/python folder of your Sourcetrail package. Getting it into a new Sourcetrail Release will take longer.

I will schedule the SourcetrailPythonIndexer release now, and let you know once it is available.

[mlangkabel]

Done, it's ready for download: https://github.com/CoatiSoftware/SourcetrailPythonIndexer/releases/tag/v1_db25_p1

[reberac]

Hello,

I do have the same issue (python binary potentially unsafe) with a conda environment on OS X / Sourcetrail 2019.4.61.
I've downloaded the new SourcetrailPythonIndexer release and extract that release and replace the data/python folder of my Sourcetrail package but the issue remains. Thanks for any support

[mlangkabel]

Hm, that is really strange. I tested the fix on Ubuntu and it was then accepting an environment (anaconda based) that was before marked as unsafe.

We were planning to release a new version of Sourcetrail next week and were really looking forward to having this issue fixed.

If anyone who still has this issue wants to debug it:

• down our source from this repo
• pip install "jedi==0.15.0"
• python run.py check-environment --environment-path "path/to/your/env"

[reberac]

I did the following :

• Check out this repository
• Install jedi by running conda install jedi (v0.15 installed)
• Download the SourcetrailDB Python bindings for Python 3.6 on macOS and extract both the _sourcetraildb.so and the sourcetraildb.py files to the root of the checked out repository
• run python run.py check-environment --environment-path "path/to/your/env" (for me python run.py check-environment --environment-path "Applications/Anaconda3/envs/py36")

And I had the following ImportError

Traceback (most recent call last):
  File "run.py", line 2, in <module>
    import indexer
  File "/Applications/SourcetrailPythonIndexer-master/indexer.py", line 7, in <module>
    import sourcetraildb as srctrl
  File "/Applications/SourcetrailPythonIndexer-master/sourcetraildb.py", line 17, in <module>
    _sourcetraildb = swig_import_helper()
  File "/Applications/SourcetrailPythonIndexer-master/sourcetraildb.py", line 16, in swig_import_helper
    return importlib.import_module('_sourcetraildb')

 File "/Applications/anaconda3/envs/py36/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
ImportError: dlopen(/Applications/SourcetrailPythonIndexer-master/_sourcetraildb.so, 2): Library not loaded: /usr/local/opt/python/Frameworks/Python.framework/Versions/3.6/Python
  Referenced from: /Applications/SourcetrailPythonIndexer-master/_sourcetraildb.so
  Reason: image not found

An idea to go further ? Thanks

[reberac]

Hmm, I've just made another test.
If I run Sourcetrail using the Sourcetrail.app launcher it failed.
If I run Sourcetrail by double-clicking on the executable file /Applications/Sourcetrail/Contents/MacOs/Sourcetrail, it works !

[mlangkabel]

What do you mean? Starting Sourcetrail fails or works? Or using the python environment fails or works?

[reberac]

When I run Sourcetrail using the Sourcetrail.app launcher I have the "python binary potentially unsafe" issue, while when I run Sourcetrail by double-clicking on the /Applications/Sourcetrail/Contents/MacOs/Sourcetrail executable file, the issue is solved, the python environment works

[mlangkabel]

Ok, thank you for the update. @egraether: what do you think about this?