Gathering environment information fails consistently

[affan-hussain]

Hey folks! I'm just trying out SCIP for the first time. I wanted to try running it on my python repo and used the command scip-python index . and it is consistenly failing. Running pip3 show -f on the CLI works fine.

(13:48:15) Evaluating python environment dependencies
(13:48:17)   Gathering environment information from `pip`

Experienced Fatal Error While Indexing:
Please create an issue at github.com/sourcegraph/scip-python: <ref *1> Error: spawnSync /bin/sh ENOBUFS
    at Object.spawnSync (node:internal/child_process:1124:20)
    at spawnSync (node:child_process:876:24)
    at Object.execSync (node:child_process:957:15)
    at /opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:104453
    at t.withStatus (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:77273)
    at t.default (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:104241)
    at M.index (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:26081)
    at f (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:38106)
    at I.<anonymous> (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/scip-python.js:1:3948)
    at I._actionHandler (/opt/homebrew/lib/node_modules/@sourcegraph/scip-python/dist/vendor.js:2:1059335) {
  errno: -55,
  code: 'ENOBUFS',
  syscall: 'spawnSync /bin/sh',
  path: '/bin/sh',
  spawnargs: [
    '-c',
    'pip3 show -f aiohttp aiosignal aiostream annotated-types anthropic anyio asgiref attrs beautifulsoup4 black bleach blinker boltons boto3 boto3-stubs botocore botocore-stubs bracex certifi cffi cfgv charset-normalizer click click-option-group cloudpickle colorama coverage cryptography dataclasses-json defusedxml Deprecated distlib distro dnspython docker exa-py face fastapi fastjsonschema filelock flake8 Flask frozenlist fsspec gitdb GitPython glom grpclib h11 h2 hpack httpcore httpx huggingface-hub hyperframe identify idna importlib-metadata iniconfig isort itsdangerous Jinja2 jira jmespath jsonpatch jsonpointer jsonschema jsonschema-specifications jupyter_client jupyter_core jupyterlab_pygments jwt langchain langchain-community langchain-core langchain-openai langchain-pinecone langsmith loguru markdown-it-py marko MarkupSafe marshmallow mccabe mdurl mistune modal multidict mypy mypy-extensions nbclient nbconvert nbformat networkx networkx-stubs nodeenv numpy oauthlib openai packaging pandas pandas-stubs pandocfilters pathspec peewee pillow pinecone-client pip platformdirs pluggy pre-commit protobuf pycodestyle pycparser pydantic pydantic_core pyflakes PyGithub Pygments PyJWT PyNaCl pytest pytest-json-report pytest-metadata pytest-mock python-dateutil python-dotenv pytz PyYAML pyzmq redis referencing regex requests requests-oauthlib requests-toolbelt rich rpds-py ruamel.yaml ruamel.yaml.clib ruff s3transfer semgrep setuptools sigtools six slack-bolt slack_sdk smmap sniffio soupsieve SQLAlchemy starlette synchronicity tblib tenacity tiktoken tinycss2 tokenizers toml tomli tornado tqdm traitlets tree-sitter tree-sitter-languages typer types-awscrt types-certifi types-pytz types-PyYAML types-requests types-s3transfer types-setuptools types-toml types-tree-sitter types-tree-sitter-languages types-urllib3 typing_extensions typing-inspect tzdata unidiff urllib3 uvicorn virtualenv watchfiles watchtower wcmatch webencodings Werkzeug wrapt yarl zipp'
  ],
  error: [Circular *1],
  status: null,
  signal: 'SIGTERM',
  output: [
    null,
    <Buffer 4e 61 6d 65 3a 20 61 69 6f 68 74 74 70 0a 56 65 72 73 69 6f 6e 3a 20 33 2e 39 2e 31 0a 53 75 6d 6d 61 72 79 3a 20 41 73 79 6e 63 20 68 74 74 70 20 63 ... 1048618 more bytes>,
    <Buffer >
  ],
  pid: 74461,
  stdout: <Buffer 4e 61 6d 65 3a 20 61 69 6f 68 74 74 70 0a 56 65 72 73 69 6f 6e 3a 20 33 2e 39 2e 31 0a 53 75 6d 6d 61 72 79 3a 20 41 73 79 6e 63 20 68 74 74 70 20 63 ... 1048618 more bytes>,
  stderr: <Buffer >
}

[zygi]

Looks like it's because pip's output is too big to put in the buffer. A proper fix would involve rewriting this with child_process.spawn() (https://stackoverflow.com/questions/63796633/spawnsync-bin-sh-enobufs) but a workaround is to just give a larger buffer to exec:

diff --git a/packages/pyright-scip/src/virtualenv/environment.ts b/packages/pyright-scip/src/virtualenv/environment.ts
index 802ba9f8d..1abd0178b 100644
--- a/packages/pyright-scip/src/virtualenv/environment.ts
+++ b/packages/pyright-scip/src/virtualenv/environment.ts
@@ -29,13 +29,13 @@ let getPipCommand = () => {
 };

 function pipList(): PipInformation[] {
-    return JSON.parse(child_process.execSync(`${getPipCommand()} list --format=json`).toString()) as PipInformation[];
+    return JSON.parse(child_process.execSync(`${getPipCommand()} list --format=json`, { maxBuffer: 1024*1024*5 }).toString()) as PipInformation[];
 }

 function pipBulkShow(names: string[]): string[] {
     // TODO: This probably breaks with enough names. Should batch them into 512 or whatever the max for bash would be
     return child_process
-        .execSync(`${getPipCommand()} show -f ${names.join(' ')}`)
+        .execSync(`${getPipCommand()} show -f ${names.join(' ')}`, { maxBuffer: 1024*1024*5 })
         .toString()
         .split('\n---');
 }

[drob]

I'm running into this as well. Any workarounds? I'm wiping my venv but let me know if there's a better way.

[RheagalFire]

I'm facing the same issue , any updates on this?