Support bazel repositories by documenting environment file format

[jlisee]

Is your feature request related to a problem? Please describe.

When trying to run scip-python on my Bazel based repository with a non-standard requirements.txt location and no use of pip the program locks up in the Evaluating python environment dependencies stage.

Describe the solution you'd like

I would like to be able to generate the dependency map myself since our repository installs dependencies through Bazel, hence files are in non-standard locations and the requirements.txt file has a non-standard location and path.

We have enough automation in place that will be straight forward to maintain, and way easier than trying to tool to understand our build system.

Additional context

Pyright itself works fine with just using [tool.pyright] extraPaths in our pyproject.toml to find these locations, a sample simple configuration:

[tool.pyright]
# Pylance plugin does not have an option to exclude paths so we do it here
exclude = ["bazel-av", "bazel-out", "bazel-bin", "bazel-external", "bazel-testlogs", "external", "avos-build", ".git", "**/node_modules", "**/__pycache__"]
extraPaths = [
    # For each library
    "./external/internal_pip_dependency_click/pypi__click",
    "./external/internal_pip_dependency_gitpython/pypi__gitpython",
    "./external/internal_pip_dependency_more_itertools/pypi__more_itertools",
    "./external/internal_pip_dependency_pytest/pypi__pytest",
    "./external/internal_pip_dependency_pyyaml/pypi__pyyaml",
    "./external/internal_pip_dependency_tabulate/pypi__tabulate",
    # For generated code like protobufs
    "./bazel-bin",
]

[jlisee]

Those folders are unpacked wheels that contain a click-8.0.4.dist-info folder:

$ find external/internal_pip_dependency_click/pypi__click/
external/internal_pip_dependency_click/pypi__click/
external/internal_pip_dependency_click/pypi__click/BUILD
external/internal_pip_dependency_click/pypi__click/click-8.0.4.dist-info
external/internal_pip_dependency_click/pypi__click/click-8.0.4.dist-info/WHEEL
external/internal_pip_dependency_click/pypi__click/click-8.0.4.dist-info/METADATA
external/internal_pip_dependency_click/pypi__click/click-8.0.4.dist-info/RECORD
external/internal_pip_dependency_click/pypi__click/click-8.0.4.dist-info/top_level.txt
external/internal_pip_dependency_click/pypi__click/click-8.0.4.dist-info/LICENSE.rst
external/internal_pip_dependency_click/pypi__click/click
external/internal_pip_dependency_click/pypi__click/click/parser.py
external/internal_pip_dependency_click/pypi__click/click/utils.py
external/internal_pip_dependency_click/pypi__click/click/testing.py
external/internal_pip_dependency_click/pypi__click/click/_unicodefun.py
external/internal_pip_dependency_click/pypi__click/click/termui.py
external/internal_pip_dependency_click/pypi__click/click/exceptions.py
external/internal_pip_dependency_click/pypi__click/click/_winconsole.py
external/internal_pip_dependency_click/pypi__click/click/types.py
external/internal_pip_dependency_click/pypi__click/click/shell_completion.py
external/internal_pip_dependency_click/pypi__click/click/__init__.py
external/internal_pip_dependency_click/pypi__click/click/_compat.py
external/internal_pip_dependency_click/pypi__click/click/py.typed
external/internal_pip_dependency_click/pypi__click/click/_textwrap.py
external/internal_pip_dependency_click/pypi__click/click/globals.py
external/internal_pip_dependency_click/pypi__click/click/formatting.py
external/internal_pip_dependency_click/pypi__click/click/core.py
external/internal_pip_dependency_click/pypi__click/click/_termui_impl.py
external/internal_pip_dependency_click/pypi__click/click/decorators.py

[varungandhi-src]

Hi @jlisee, trying to catch up here since TJ is out on leave. Could you clarify the following for me:

1. Did you have a separate convo in Slack/Zoom with TJ about this after filing this issue -- is there something else that is needed from your side?
2. Are you entirely blocked from being able to use scip-python or have you been able to figure out a workaround?

[jlisee]

Blocked? No, I am not blocked. I created a workaround by making stub pip binary that returns empty results. This enables indexing to complete successfully and be usable in a private Sourcegraph instance. Since we have a monorepository, and third party code is not indexed by our instance, this package information missing does not hurt us.

Fixed? No. I would like to be able to:

• Disable the querying of this information with something like a CLI flag
• Supply the information directly in a well documented format like a JSON file

[tjdevries]

With the latest configuration changes from #77, I think this should be possible now (and should also re-use any existing pyright configuration you have).

Does the current README give you enough info to do what you're looking for?