pixi environments loaded, but not selectable, in PyCharm 2024.3

[rprechelt]

Description

My pixi project has multiple environments: only the dev environment has pixi-pycharm. When trying to add the environment to PyCharm, the path to the conda replacement is recognized and the list of environments is loaded and shown in a drop-down menu in PyCharm, but none of the environments are selectable, e.g. even if the dev environment is selected, I cannot click OK to add it to PyCharm.

Versions

❯ pixi --version
pixi 0.35.0
❯ pixi list | grep pixi-pycharm
Environment: dev
pixi-pycharm            0.0.8       unix_1234567_0      8.7 KiB     conda  pixi-pycharm-0.0.8-unix_1234567_0.conda

PyCharm version: 2024.3

Platform: osx-arm64

Logs

Attempting to run: ['info', '--envs', '--json']

PyCharm logs:

2024-11-10 20:00:59,355 [80497691]   INFO - #c.j.p.p.m.PythonPackageManager - Exit code 1
2024-11-10 20:00:59,355 [80497691]   INFO - #c.j.p.p.m.PythonPackageManager - Traceback (most recent call last):
  File "<path-to-repo>/.pixi/envs/dev/libexec/conda", line 185, in <module>
    sys.exit(main(args))
             ^^^^^^^^^^
  File "<path-to-repo>/.pixi/envs/dev/libexec/conda", line 168, in main
    raise NotImplementedError(str(args))
NotImplementedError: ['/Applications/PyCharm 2024.2.app/Contents/plugins/python-ce/helpers/packaging_tool.py', 'list']

2024-11-10 20:00:59,368 [80497704]   INFO - #c.i.u.i.p.ProjectIndexableFilesFilterHealthCheck - Following files are NOT indexable but they were found in filter. Errors count: 21831. Examples:
file id=34312 path=<path-to-repo>/.pixi/envs/dev
file id=34314 path=<path-to-repo>/.pixi/envs/dev/lib
file id=34315 path=<path-to-repo>/.pixi/envs/dev/libexec
file id=34316 path=<path-to-repo>.pixi/envs/dev/conda-meta
file id=34317 path=<path-to-repo>/.pixi/envs/dev/share

[pavelzw]

could you show me the contents of '/Applications/PyCharm 2024.2.app/Contents/plugins/python-ce/helpers/packaging_tool.py' on your system?

[rprechelt]

import traceback
import getopt
import os

ERROR_WRONG_USAGE = 1
ERROR_NO_PIP = 2
ERROR_NO_SETUPTOOLS = 3
ERROR_EXCEPTION = 4

os.putenv("PIP_REQUIRE_VIRTUALENV", "false")

def exit(retcode):
    major, minor, micro, release, serial = sys.version_info
    version = major * 10 + minor
    if version < 25:
        import os
        os._exit(retcode)
    else:
        sys.exit(retcode)

def usage():
    sys.stderr.write('Usage: packaging_tool.py <list|install|uninstall|pyvenv>\n')
    sys.stderr.flush()
    exit(ERROR_WRONG_USAGE)

def error(message, retcode):
    sys.stderr.write('Error: %s\n' % message)
    sys.stderr.flush()
    exit(retcode)

def error_no_pip():
    type, value, tb = sys.exc_info()
    if tb is not None and tb.tb_next is None:
        error("Python packaging tool 'pip' not found", ERROR_NO_PIP)
    else:
        error(traceback.format_exc(), ERROR_EXCEPTION)

def do_list():
    if sys.version_info < (3, 10):
        try:
            import pkg_resources
        except ImportError:
            error("Python packaging tool 'setuptools' not found", ERROR_NO_SETUPTOOLS)
        for pkg in pkg_resources.working_set:
            try:
                requirements = pkg.requires()
            except Exception:
                requirements = []
            requires = ':'.join([str(x) for x in requirements])
            sys.stdout.write('\t'.join([pkg.project_name, pkg.version, pkg.location, requires])+chr(10))
    else:
        import importlib.metadata
        for pkg in importlib.metadata.distributions():
            try:
                requirements = [] if (pkg.requires is None) else pkg.requires
            except Exception:
                requirements = []
            requires = ':'.join([str(x) for x in requirements])
            if pkg.name is None or pkg.version is None or pkg._path is None:
                continue
            sys.stdout.write('\t'.join([pkg.name, pkg.version, str(pkg._path.parent), requires])+chr(10))
    sys.stdout.flush()

def do_install(pkgs):
    run_pip(['install'] + pkgs)

def do_uninstall(pkgs):
    run_pip(['uninstall', '-y'] + pkgs)

def run_pip(args):
    import runpy
    sys.argv[1:] = args
    # pip.__main__ has been around since 2010 but support for executing it automatically
    # was added in runpy.run_module only in Python 2.7/3.1
    module_name = 'pip.__main__' if sys.version_info < (2, 7) else 'pip'
    try:
        runpy.run_module(module_name, run_name='__main__', alter_sys=True)
    except ImportError:
        error_no_pip()

def do_pyvenv(args):
    import runpy
    try:
        import ensurepip
        sys.argv[1:] = args
    except ImportError:
        sys.argv[1:] = ['--without-pip'] + args

    try:
        runpy.run_module('venv', run_name='__main__', alter_sys=True)
    except ImportError:
        error("Standard Python 'venv' module not found", ERROR_EXCEPTION)

def main():
    try:
        # As a workaround for #885 in setuptools, don't expose other helpers
        # in sys.path so as not no confuse it with possible combination of
        # namespace/ordinary packages
        sys.path.remove(os.path.dirname(__file__))
    except ValueError:
        pass

    try:
        if len(sys.argv) < 2:
            usage()
        cmd = sys.argv[1]
        if cmd == 'list':
            if len(sys.argv) != 2:
                usage()
            do_list()
        elif cmd == 'install':
            if len(sys.argv) < 2:
                usage()

            pkgs = sys.argv[2:]
            do_install(pkgs)

        elif cmd == 'uninstall':
            if len(sys.argv) < 2:
                usage()
            pkgs = sys.argv[2:]
            do_uninstall(pkgs)
        elif cmd == 'pyvenv':
            opts, args = getopt.getopt(sys.argv[2:], '', ['system-site-packages'])
            if len(args) != 1:
                usage()
            do_pyvenv(sys.argv[2:])
        else:
            usage()
    except Exception:
        traceback.print_exc()
        exit(ERROR_EXCEPTION)

if __name__ == '__main__':
    main()

[ricardoV94]

I have the same visual problem on a flatpak installation on Fedora 41.

Can't find any useful logs from pycharm though.

[ricardoV94]

Seems to be an issue with PyCharm 2024.2.3, I can use it if I install 2024.2.2. Same thing happens with micromamba-pycharm

[pavelzw]

hmm, i just tried it out in both pycharm professional 2024.2.3 and 2024.3 on macOS arm and it seemed to work in both cases... i'll try out the community edition as well

[ricardoV94]

hmm, i just tried it out in both pycharm professional 2024.2.3 and 2024.3 on macOS arm and it seemed to work in both cases... i'll try out the community edition as well

I installed as a flatpak, maybe that matters?

[pavelzw]

yes, with the community edition, i can reproduce it. wondering whether this is because i'm not using a "clean" pycharm professional edition or whether it's actually a difference between professional and community edition

although my pixi-pycharm logs only look as follows:

Attempting to run: ['info', '--envs', '--json']

[ricardoV94]

Same for me I can't see any log for errors on the pycharm side

[pavelzw]

PyCharm version: 2024.2.1

@rprechelt i'm assuming you mistyped as your screenshot looks like the new selection screen from 2024.3

[pavelzw]

@ricardoV94 are you using the community or professional edition?

[ricardoV94]

professional

[pavelzw]

alright, i'll try it in a clean installation again

[ricardoV94]

I suspect the macos version uses a python script that shows those more interesting logs, wheres the linux is all in the java executable?

I couldn't find a script with the same name anywhere

[pavelzw]

in remy's logs we can see PyCharm 2024.2.app 👀 also, when i try to reproduce the issue, i get different line numbers in the stacktrace 🤔 the logs might be from some time before?

❯ .pixi/envs/default/libexec/conda "/Applications/PyCharm 2024.2.app/Contents/plugins/python-ce/helpers/packaging_tool.py" list
Traceback (most recent call last):
  File ".../.pixi/envs/default/libexec/conda", line 182, in <module>
    sys.exit(main(args))
             ^^^^^^^^^^
  File ".../.pixi/envs/default/libexec/conda", line 165, in main
    raise NotImplementedError(str(args))
NotImplementedError: ['/Applications/PyCharm 2024.2.app/Contents/plugins/python-ce/helpers/packaging_tool.py', 'list']

also, the NotImplementedError should have been written to ~/.cache/pixi-pycharm.log as well

[pavelzw]

to be fair, with conda 4.14.0, you get the same issue

[pavelzw]

even with the latest conda version 24.9.2, pycharm still outputs No conda environment selected...

@ricardoV94 can you verify my findings?

[pavelzw]

https://github.com/JetBrains/intellij-community/blob/bf4f92203e12482cc8a5afe8e06c8c61f3dfb424/python/src/com/jetbrains/python/sdk/add/v2/CondaExistingEnvironmentSelector.kt#L52 seems to be where we get the error

[ricardoV94]

even with the latest conda version 24.9.2, pycharm still outputs No conda environment selected...

@ricardoV94 can you verify my findings?

You mean it fails to recognize conda directly?

[pavelzw]

yes