`isort` does not pickup configuration when it is not in current working directory

achen187 commented 1 year ago

Hi, it seems since updated to VSCode 1.73, the import sort order that gets autosaved has changed.

This is what my import sort used to be:

Now, when I hit save, this is what it changes to:

Here is our settings.json:

{
  "[typescriptreact]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  },
  "editor.formatOnSave": true,
  "files.exclude": {
    "**/__pycache__": true
  },
  "files.eol": "\n",
  "explorer.compactFolders": false,
  "[python]": {
    "editor.codeActionsOnSave": {
      "source.organizeImports": true
    }
  },
  "python.formatting.provider": "black",
  "python.testing.pytestEnabled": true,
  "python.testing.unittestArgs": ["-v", "-s", ".", "-p"],
  "python.testing.cwd": "pipeline",
  "jupyter.kernels.filter": [

  ],
  "window.zoomLevel": 1,
}

pyproject.toml:

[tool.poetry]
name = "pipeline"
version = "0.1.0"
description = ""
authors = ["achen187 <chen.andy1993@gmail.com>"]

[tool.poetry.dependencies]
python = "^3.9"
dagster = "0.13.8"
dagit = "0.13.8"
pandas = "^1.3.5"
pyproj = "^3.3.0"
pyshp = "^2.1.3"
networkx = "^2.6.3"
Shapely = "1.7.1"
pandera = "^0.8.1"
openpyxl = "^3.0.9"
xlwt = "^1.3.0"
xlrd = "^2.0.1"

[tool.poetry.dev-dependencies]
pytest = "^6.2.5"
pytest-mockito = "^0.0.4"
black = "^21.12b0"
flake8 = "^4.0.1"
isort = "^5.10.1"
autoflake = "^1.4"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.pytest.ini_options]
minversion = "6.0"
testpaths = [
    "pipeline_tests",
]
addopts = "-ra --basetemp=pipeline_tests/tmp_test_data"

[tool.isort]
profile = "black"

When I run /Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python -m isort ./ It goes back to the original sort order I'm expecting.

So it seems like the new Isort mechanism isn't actually using the same command as my manual call?

I then tried to change the setting: "isort.importStrategy": "fromEnvironment" but then that actually crashed the iSort server.

TypeError: stop_all_processes() missing 1 required positional argument: 'self'
[Info  - 4:04:54 PM] Connection to server got closed. Server will restart.
Traceback (most recent call last):
  File "/Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/tool/server.py", line 41, in <module>
    from pygls import lsp, protocol, server, uris, workspace
  File "/Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/libs/pygls/lsp/__init__.py", line 23, in <module>
    from pygls.lsp.types import *
  File "/Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/libs/pygls/lsp/types/__init__.py", line 2, in <module>
    from pygls.lsp.types.basic_structures import *
  File "/Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/libs/pygls/lsp/types/basic_structures.py", line 30, in <module>
    from pydantic import BaseModel, root_validator
  File "/Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/libs/pydantic/__init__.py", line 2, in <module>
    from . import dataclasses
  File "/Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/libs/pydantic/dataclasses.py", line 52, in <module>
    from typing_extensions import dataclass_transform
ImportError: cannot import name 'dataclass_transform' from 'typing_extensions' (/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/lib/python3.9/site-packages/typing_extensions.py)
Error in atexit._run_exitfuncs:
TypeError: stop_all_processes() missing 1 required positional argument: 'self'
[Error - 4:04:54 PM] Connection to server got closed. Server will not be restarted.

achen187 commented 1 year ago

In the meantime, we're going to turn off "source.organizeImports" and call isort manually from the command line.

This was super painful though, and I burned an afternoon tracking this down.

achen187 commented 1 year ago

What's interesting though is that when I look in the output, it does theoretically say that its using the same isort executable?

CWD Server: /Users/andychen/dev/nira-data-pipeline
sys.path used to run Server:
   /Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/libs
   /Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/tool
   /Users/andychen/.pyenv/versions/3.9.6/lib/python39.zip
   /Users/andychen/.pyenv/versions/3.9.6/lib/python3.9
   /Users/andychen/.pyenv/versions/3.9.6/lib/python3.9/lib-dynload
   /Users/andychen/dev/nira-data-pipeline/pipeline/.venv/lib/python3.9/site-packages
   /Users/andychen/dev/nira-data-pipeline/pipeline
Settings used to run Server:
[
    {
        "check": false,
        "workspace": "file:///Users/andychen/dev/nira-data-pipeline",
        "logLevel": "error",
        "args": [],
        "severity": {
            "W": "Warning",
            "E": "Error"
        },
        "path": [],
        "interpreter": [
            "/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python"
        ],
        "importStrategy": "useBundled",
        "showNotifications": "off"
    }
]

/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python -m isort --version-number
CWD Linter: /Users/andychen/dev/nira-data-pipeline
Version info for isort running for /Users/andychen/dev/nira-data-pipeline:
5.10.1

SUPPORTED isort>=5.10.1
FOUND isort==5.10.1

karthiknadig commented 1 year ago

@achen187 When you run using organize imports what is the command that you see in the logs?

karthiknadig commented 1 year ago

And can you set loglevel for isort to debug before getting the logs?

achen187 commented 1 year ago

You mean these logs right?

achen187 commented 1 year ago

{
  "[typescriptreact]": {
    "editor.defaultFormatter": "esbenp.prettier-vscode"
  },
  "editor.formatOnSave": true,
  "files.exclude": {
    "**/__pycache__": true
  },
  "files.eol": "\n",
  "explorer.compactFolders": false,
  "[python]": {
    "editor.codeActionsOnSave": {
      "source.organizeImports": true
    }
  },
  "python.formatting.provider": "black",
  "python.testing.pytestEnabled": true,
  "python.testing.unittestArgs": ["-v", "-s", ".", "-p"],
  "python.testing.cwd": "pipeline",
  "jupyter.kernels.filter": [

  ],
  "window.zoomLevel": 1,
  "isort.logLevel": "debug"
}

achen187 commented 1 year ago

startup logs:

[INFO 2022-10-3 16:18:32.506]: Server: Stop requested
[DEBUG 2022-10-3 16:18:32.506]: Server State: Stopped
[INFO 2022-10-3 16:18:32.514]: Server run command: /Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python /Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/tool/server.py
[INFO 2022-10-3 16:18:32.515]: Server: Start requested.
[DEBUG 2022-10-3 16:18:32.515]: Server State: Starting
CWD Server: /Users/andychen/dev/nira-data-pipeline
sys.path used to run Server:
   /Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/libs
   /Users/andychen/.vscode/extensions/ms-python.isort-2022.4.0/bundled/tool
   /Users/andychen/.pyenv/versions/3.9.6/lib/python39.zip
   /Users/andychen/.pyenv/versions/3.9.6/lib/python3.9
   /Users/andychen/.pyenv/versions/3.9.6/lib/python3.9/lib-dynload
   /Users/andychen/dev/nira-data-pipeline/pipeline/.venv/lib/python3.9/site-packages
   /Users/andychen/dev/nira-data-pipeline/pipeline
Settings used to run Server:
[
    {
        "check": false,
        "workspace": "file:///Users/andychen/dev/nira-data-pipeline",
        "logLevel": "debug",
        "args": [],
        "severity": {
            "W": "Warning",
            "E": "Error"
        },
        "path": [],
        "interpreter": [
            "/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python"
        ],
        "importStrategy": "useBundled",
        "showNotifications": "off"
    }
]

/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python -m isort --version-number
CWD Linter: /Users/andychen/dev/nira-data-pipeline
Version info for isort running for /Users/andychen/dev/nira-data-pipeline:
5.10.1

SUPPORTED isort>=5.10.1
FOUND isort==5.10.1

/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python -m isort --show-config
CWD Linter: /Users/andychen/dev/nira-data-pipeline
Config details for isort running for /Users/andychen/dev/nira-data-pipeline:
{
    "_known_patterns": null,
    "_section_comments": null,
    "_section_comments_end": null,
    "_skips": null,
    "_skip_globs": null,
    "_sorting_function": null,
    "py_version": "py3",
    "force_to_top": [],
    "skip": [
        ".mypy_cache",
        "node_modules",
        ".venv",
        "venv",
        ".nox",
        ".pants.d",
        ".bzr",
        ".hg",
        "dist",
        "_build",
        ".svn",
        ".eggs",
        ".tox",
        "build",
        "__pypackages__",
        ".git",
        "buck-out",
        ".direnv"
    ],
    "extend_skip": [],
    "skip_glob": [],
    "extend_skip_glob": [],
    "skip_gitignore": false,
    "line_length": 79,
    "wrap_length": 0,
    "line_ending": "",
    "sections": [
        "FUTURE",
        "STDLIB",
        "THIRDPARTY",
        "FIRSTPARTY",
        "LOCALFOLDER"
    ],
    "no_sections": false,
    "known_future_library": [
        "__future__"
    ],
    "known_third_party": [],
    "known_first_party": [],
    "known_local_folder": [],
    "known_standard_library": [
        "datetime",
        "gc",
        "tempfile",
        "queue",
        "parser",
        "time",
        "typing",
        "sre_compile",
        "pickle",
        "heapq",
        "tabnanny",
        "contextlib",
        "decimal",
        "ctypes",
        "codeop",
        "traceback",
        "copy",
        "importlib",
        "colorsys",
        "imghdr",
        "netrc",
        "msvcrt",
        "dataclasses",
        "binascii",
        "glob",
        "grp",
        "pydoc",
        "dummy_threading",
        "asyncio",
        "platform",
        "syslog",
        "pwd",
        "pickletools",
        "random",
        "smtplib",
        "xmlrpc",
        "pdb",
        "linecache",
        "configparser",
        "ast",
        "codecs",
        "statistics",
        "_ast",
        "getopt",
        "gettext",
        "fnmatch",
        "quopri",
        "socketserver",
        "imaplib",
        "pipes",
        "_dummy_thread",
        "shelve",
        "turtledemo",
        "encodings",
        "wsgiref",
        "readline",
        "sndhdr",
        "fpectl",
        "types",
        "bz2",
        "http",
        "base64",
        "msilib",
        "tty",
        "selectors",
        "graphlib",
        "xml",
        "contextvars",
        "tokenize",
        "keyword",
        "socket",
        "ossaudiodev",
        "rlcompleter",
        "chunk",
        "tarfile",
        "test",
        "code",
        "ftplib",
        "macpath",
        "cgi",
        "distutils",
        "pathlib",
        "formatter",
        "zlib",
        "sre",
        "symtable",
        "sre_parse",
        "difflib",
        "ensurepip",
        "math",
        "builtins",
        "crypt",
        "dbm",
        "xdrlib",
        "calendar",
        "winsound",
        "pty",
        "telnetlib",
        "lib2to3",
        "getpass",
        "collections",
        "nis",
        "timeit",
        "plistlib",
        "mmap",
        "csv",
        "uuid",
        "hmac",
        "urllib",
        "threading",
        "cProfile",
        "re",
        "subprocess",
        "poplib",
        "audioop",
        "fcntl",
        "smtpd",
        "ssl",
        "weakref",
        "select",
        "spwd",
        "operator",
        "token",
        "faulthandler",
        "wave",
        "sysconfig",
        "symbol",
        "cmath",
        "locale",
        "enum",
        "cmd",
        "shlex",
        "stringprep",
        "ntpath",
        "curses",
        "trace",
        "unicodedata",
        "string",
        "filecmp",
        "io",
        "zipimport",
        "email",
        "compileall",
        "marshal",
        "gzip",
        "posix",
        "secrets",
        "zipapp",
        "posixpath",
        "asyncore",
        "cgitb",
        "json",
        "venv",
        "fractions",
        "ipaddress",
        "logging",
        "tkinter",
        "functools",
        "textwrap",
        "pkgutil",
        "optparse",
        "nntplib",
        "stat",
        "aifc",
        "struct",
        "itertools",
        "imp",
        "hashlib",
        "warnings",
        "zoneinfo",
        "copyreg",
        "resource",
        "turtle",
        "errno",
        "argparse",
        "dis",
        "html",
        "bdb",
        "sre_constants",
        "inspect",
        "binhex",
        "profile",
        "uu",
        "mailbox",
        "webbrowser",
        "reprlib",
        "tracemalloc",
        "bisect",
        "termios",
        "_thread",
        "sqlite3",
        "zipfile",
        "array",
        "modulefinder",
        "lzma",
        "asynchat",
        "pprint",
        "shutil",
        "pyclbr",
        "abc",
        "multiprocessing",
        "fileinput",
        "py_compile",
        "concurrent",
        "atexit",
        "doctest",
        "mailcap",
        "numbers",
        "signal",
        "os",
        "pstats",
        "site",
        "mimetypes",
        "winreg",
        "sched",
        "sunau",
        "unittest",
        "runpy",
        "sys"
    ],
    "extra_standard_library": [],
    "known_other": {},
    "multi_line_output": "GRID",
    "forced_separate": [],
    "indent": "    ",
    "comment_prefix": "  #",
    "length_sort": false,
    "length_sort_straight": false,
    "length_sort_sections": [],
    "add_imports": [],
    "remove_imports": [],
    "append_only": false,
    "reverse_relative": false,
    "force_single_line": false,
    "single_line_exclusions": [],
    "default_section": "THIRDPARTY",
    "import_headings": {},
    "import_footers": {},
    "balanced_wrapping": false,
    "use_parentheses": false,
    "order_by_type": true,
    "atomic": false,
    "lines_before_imports": -1,
    "lines_after_imports": -1,
    "lines_between_sections": 1,
    "lines_between_types": 0,
    "combine_as_imports": false,
    "combine_star": false,
    "include_trailing_comma": false,
    "from_first": false,
    "verbose": false,
    "quiet": false,
    "force_adds": false,
    "force_alphabetical_sort_within_sections": false,
    "force_alphabetical_sort": false,
    "force_grid_wrap": 0,
    "force_sort_within_sections": false,
    "lexicographical": false,
    "group_by_package": false,
    "ignore_whitespace": false,
    "no_lines_before": [],
    "no_inline_sort": false,
    "ignore_comments": false,
    "case_sensitive": false,
    "sources": [
        {
            "py_version": "py3",
            "force_to_top": [],
            "skip": [
                ".mypy_cache",
                "node_modules",
                ".venv",
                "venv",
                ".nox",
                ".pants.d",
                ".bzr",
                ".hg",
                "dist",
                "_build",
                ".svn",
                ".eggs",
                ".tox",
                "build",
                "__pypackages__",
                ".git",
                "buck-out",
                ".direnv"
            ],
            "extend_skip": [],
            "skip_glob": [],
            "extend_skip_glob": [],
            "skip_gitignore": false,
            "line_length": 79,
            "wrap_length": 0,
            "line_ending": "",
            "sections": [
                "FUTURE",
                "STDLIB",
                "THIRDPARTY",
                "FIRSTPARTY",
                "LOCALFOLDER"
            ],
            "no_sections": false,
            "known_future_library": [
                "__future__"
            ],
            "known_third_party": [],
            "known_first_party": [],
            "known_local_folder": [],
            "known_standard_library": [
                "datetime",
                "gc",
                "tempfile",
                "queue",
                "parser",
                "time",
                "typing",
                "sre_compile",
                "pickle",
                "heapq",
                "tabnanny",
                "contextlib",
                "decimal",
                "ctypes",
                "codeop",
                "traceback",
                "copy",
                "importlib",
                "colorsys",
                "imghdr",
                "netrc",
                "msvcrt",
                "dataclasses",
                "binascii",
                "glob",
                "grp",
                "pydoc",
                "dummy_threading",
                "asyncio",
                "platform",
                "syslog",
                "pwd",
                "pickletools",
                "random",
                "smtplib",
                "xmlrpc",
                "pdb",
                "linecache",
                "configparser",
                "ast",
                "codecs",
                "statistics",
                "_ast",
                "getopt",
                "gettext",
                "fnmatch",
                "quopri",
                "socketserver",
                "imaplib",
                "pipes",
                "_dummy_thread",
                "shelve",
                "turtledemo",
                "encodings",
                "wsgiref",
                "readline",
                "sndhdr",
                "fpectl",
                "types",
                "bz2",
                "http",
                "base64",
                "msilib",
                "tty",
                "selectors",
                "graphlib",
                "xml",
                "contextvars",
                "tokenize",
                "keyword",
                "socket",
                "ossaudiodev",
                "rlcompleter",
                "chunk",
                "tarfile",
                "test",
                "code",
                "ftplib",
                "macpath",
                "cgi",
                "distutils",
                "pathlib",
                "formatter",
                "zlib",
                "sre",
                "symtable",
                "sre_parse",
                "difflib",
                "ensurepip",
                "math",
                "builtins",
                "crypt",
                "dbm",
                "xdrlib",
                "calendar",
                "winsound",
                "pty",
                "telnetlib",
                "lib2to3",
                "getpass",
                "collections",
                "nis",
                "timeit",
                "plistlib",
                "mmap",
                "csv",
                "uuid",
                "hmac",
                "urllib",
                "threading",
                "cProfile",
                "re",
                "subprocess",
                "poplib",
                "audioop",
                "fcntl",
                "smtpd",
                "ssl",
                "weakref",
                "select",
                "spwd",
                "operator",
                "token",
                "faulthandler",
                "wave",
                "sysconfig",
                "symbol",
                "cmath",
                "locale",
                "enum",
                "cmd",
                "shlex",
                "stringprep",
                "ntpath",
                "curses",
                "trace",
                "unicodedata",
                "string",
                "filecmp",
                "io",
                "zipimport",
                "email",
                "compileall",
                "marshal",
                "gzip",
                "posix",
                "secrets",
                "zipapp",
                "posixpath",
                "asyncore",
                "cgitb",
                "json",
                "venv",
                "fractions",
                "ipaddress",
                "logging",
                "tkinter",
                "functools",
                "textwrap",
                "pkgutil",
                "optparse",
                "nntplib",
                "stat",
                "aifc",
                "struct",
                "itertools",
                "imp",
                "hashlib",
                "warnings",
                "zoneinfo",
                "copyreg",
                "resource",
                "turtle",
                "errno",
                "argparse",
                "dis",
                "html",
                "bdb",
                "sre_constants",
                "inspect",
                "binhex",
                "profile",
                "uu",
                "mailbox",
                "webbrowser",
                "reprlib",
                "tracemalloc",
                "bisect",
                "termios",
                "_thread",
                "sqlite3",
                "zipfile",
                "array",
                "modulefinder",
                "lzma",
                "asynchat",
                "pprint",
                "shutil",
                "pyclbr",
                "abc",
                "multiprocessing",
                "fileinput",
                "py_compile",
                "concurrent",
                "atexit",
                "doctest",
                "mailcap",
                "numbers",
                "signal",
                "os",
                "pstats",
                "site",
                "mimetypes",
                "winreg",
                "sched",
                "sunau",
                "unittest",
                "runpy",
                "sys"
            ],
            "extra_standard_library": [],
            "known_other": {},
            "multi_line_output": "GRID",
            "forced_separate": [],
            "indent": "    ",
            "comment_prefix": "  #",
            "length_sort": false,
            "length_sort_straight": false,
            "length_sort_sections": [],
            "add_imports": [],
            "remove_imports": [],
            "append_only": false,
            "reverse_relative": false,
            "force_single_line": false,
            "single_line_exclusions": [],
            "default_section": "THIRDPARTY",
            "import_headings": {},
            "import_footers": {},
            "balanced_wrapping": false,
            "use_parentheses": false,
            "order_by_type": true,
            "atomic": false,
            "lines_before_imports": -1,
            "lines_after_imports": -1,
            "lines_between_sections": 1,
            "lines_between_types": 0,
            "combine_as_imports": false,
            "combine_star": false,
            "include_trailing_comma": false,
            "from_first": false,
            "verbose": false,
            "quiet": false,
            "force_adds": false,
            "force_alphabetical_sort_within_sections": false,
            "force_alphabetical_sort": false,
            "force_grid_wrap": 0,
            "force_sort_within_sections": false,
            "lexicographical": false,
            "group_by_package": false,
            "ignore_whitespace": false,
            "no_lines_before": [],
            "no_inline_sort": false,
            "ignore_comments": false,
            "case_sensitive": false,
            "sources": [],
            "virtual_env": "",
            "conda_env": "",
            "ensure_newline_before_comments": false,
            "directory": "",
            "profile": "",
            "honor_noqa": false,
            "src_paths": [],
            "old_finders": false,
            "remove_redundant_aliases": false,
            "float_to_top": false,
            "filter_files": false,
            "formatter": "",
            "formatting_function": null,
            "color_output": false,
            "treat_comments_as_code": [],
            "treat_all_comments_as_code": false,
            "supported_extensions": [
                "pyx",
                "pxd",
                "py",
                "pyi"
            ],
            "blocked_extensions": [
                "pex"
            ],
            "constants": [],
            "classes": [],
            "variables": [],
            "dedup_headings": false,
            "only_sections": false,
            "only_modified": false,
            "combine_straight_imports": false,
            "auto_identify_namespace_packages": true,
            "namespace_packages": [],
            "follow_links": true,
            "indented_import_headings": true,
            "honor_case_in_force_sorted_sections": false,
            "sort_relative_in_force_sorted_sections": false,
            "overwrite_in_place": false,
            "reverse_sort": false,
            "star_first": false,
            "git_ignore": {},
            "format_error": "{error}: {message}",
            "format_success": "{success}: {message}",
            "sort_order": "natural",
            "source": "defaults"
        }
    ],
    "virtual_env": "",
    "conda_env": "",
    "ensure_newline_before_comments": false,
    "directory": "/Users/andychen/dev/nira-data-pipeline",
    "profile": "",
    "honor_noqa": false,
    "src_paths": [
        "/Users/andychen/dev/nira-data-pipeline/src",
        "/Users/andychen/dev/nira-data-pipeline"
    ],
    "old_finders": false,
    "remove_redundant_aliases": false,
    "float_to_top": false,
    "filter_files": false,
    "formatter": "",
    "formatting_function": null,
    "color_output": false,
    "treat_comments_as_code": [],
    "treat_all_comments_as_code": false,
    "supported_extensions": [
        "pyx",
        "pxd",
        "py",
        "pyi"
    ],
    "blocked_extensions": [
        "pex"
    ],
    "constants": [],
    "classes": [],
    "variables": [],
    "dedup_headings": false,
    "only_sections": false,
    "only_modified": false,
    "combine_straight_imports": false,
    "auto_identify_namespace_packages": true,
    "namespace_packages": [],
    "follow_links": true,
    "indented_import_headings": true,
    "honor_case_in_force_sorted_sections": false,
    "sort_relative_in_force_sorted_sections": false,
    "overwrite_in_place": false,
    "reverse_sort": false,
    "star_first": false,
    "git_ignore": {},
    "format_error": "{error}: {message}",
    "format_success": "{success}: {message}",
    "sort_order": "natural"
}

[DEBUG 2022-10-3 16:18:32.861]: Server State: Running

achen187 commented 1 year ago

on every save;

/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python -m isort - --filename /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py
CWD Linter: /Users/andychen/dev/nira-data-pipeline
/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python -m isort - --filename /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py
CWD Linter: /Users/andychen/dev/nira-data-pipeline

karthiknadig commented 1 year ago

The way extension does import sorting on save is equivalent to cat <file-path> | python -m isort - --filename <file-path>. This is because, the buffer is not yet saved, so we have to send contents of the editor to isort as text.

Can you run python -m isort --show-config for the one from your command line, i.e., with the isort that works correctly. This way we can compare what the extension version of isort is seeing vs the one that is installed in the environment.

achen187 commented 1 year ago

{
    "_known_patterns": null,
    "_section_comments": null,
    "_section_comments_end": null,
    "_skips": null,
    "_skip_globs": null,
    "_sorting_function": null,
    "py_version": "py3",
    "force_to_top": [],
    "skip": [
        ".direnv",
        ".svn",
        ".hg",
        ".bzr",
        ".mypy_cache",
        ".tox",
        "_build",
        ".git",
        ".pants.d",
        "node_modules",
        ".nox",
        ".venv",
        "dist",
        "buck-out",
        "venv",
        ".eggs",
        "__pypackages__",
        "build"
    ],
    "extend_skip": [],
    "skip_glob": [],
    "extend_skip_glob": [],
    "skip_gitignore": false,
    "line_length": 88,
    "wrap_length": 0,
    "line_ending": "",
    "sections": [
        "FUTURE",
        "STDLIB",
        "THIRDPARTY",
        "FIRSTPARTY",
        "LOCALFOLDER"
    ],
    "no_sections": false,
    "known_future_library": [
        "__future__"
    ],
    "known_third_party": [],
    "known_first_party": [],
    "known_local_folder": [],
    "known_standard_library": [
        "codeop",
        "xml",
        "keyword",
        "posixpath",
        "dummy_threading",
        "sysconfig",
        "timeit",
        "mailcap",
        "sre_constants",
        "fnmatch",
        "ntpath",
        "symtable",
        "getopt",
        "typing",
        "pickle",
        "pdb",
        "pickletools",
        "_dummy_thread",
        "reprlib",
        "ssl",
        "dis",
        "bz2",
        "audioop",
        "modulefinder",
        "bisect",
        "fileinput",
        "pkgutil",
        "site",
        "tempfile",
        "sndhdr",
        "turtledemo",
        "winreg",
        "msilib",
        "re",
        "enum",
        "codecs",
        "contextvars",
        "shelve",
        "tarfile",
        "lzma",
        "grp",
        "configparser",
        "inspect",
        "binascii",
        "shutil",
        "socketserver",
        "pwd",
        "colorsys",
        "posix",
        "weakref",
        "pprint",
        "collections",
        "gc",
        "binhex",
        "atexit",
        "optparse",
        "bdb",
        "locale",
        "numbers",
        "imghdr",
        "xmlrpc",
        "email",
        "chunk",
        "compileall",
        "selectors",
        "quopri",
        "curses",
        "encodings",
        "fractions",
        "cgitb",
        "mmap",
        "hashlib",
        "importlib",
        "gzip",
        "calendar",
        "string",
        "marshal",
        "webbrowser",
        "linecache",
        "sched",
        "ossaudiodev",
        "test",
        "rlcompleter",
        "datetime",
        "stringprep",
        "formatter",
        "getpass",
        "sre_parse",
        "traceback",
        "smtpd",
        "py_compile",
        "pydoc",
        "sre_compile",
        "sqlite3",
        "pathlib",
        "io",
        "distutils",
        "http",
        "warnings",
        "fcntl",
        "imaplib",
        "heapq",
        "abc",
        "tty",
        "errno",
        "shlex",
        "dataclasses",
        "graphlib",
        "html",
        "profile",
        "venv",
        "_ast",
        "decimal",
        "uuid",
        "csv",
        "cmath",
        "faulthandler",
        "pyclbr",
        "symbol",
        "tracemalloc",
        "ipaddress",
        "winsound",
        "ast",
        "zipapp",
        "glob",
        "difflib",
        "tkinter",
        "array",
        "time",
        "imp",
        "os",
        "subprocess",
        "zipfile",
        "stat",
        "concurrent",
        "xdrlib",
        "platform",
        "token",
        "doctest",
        "parser",
        "struct",
        "tabnanny",
        "asyncore",
        "runpy",
        "itertools",
        "msvcrt",
        "statistics",
        "resource",
        "cmd",
        "smtplib",
        "operator",
        "sys",
        "socket",
        "poplib",
        "pty",
        "asyncio",
        "cgi",
        "unittest",
        "types",
        "sre",
        "macpath",
        "wave",
        "builtins",
        "turtle",
        "termios",
        "zlib",
        "copyreg",
        "cProfile",
        "base64",
        "dbm",
        "secrets",
        "ftplib",
        "filecmp",
        "netrc",
        "tokenize",
        "ctypes",
        "pstats",
        "random",
        "mimetypes",
        "aifc",
        "lib2to3",
        "gettext",
        "nis",
        "_thread",
        "ensurepip",
        "multiprocessing",
        "textwrap",
        "wsgiref",
        "contextlib",
        "asynchat",
        "uu",
        "json",
        "threading",
        "syslog",
        "trace",
        "mailbox",
        "copy",
        "math",
        "pipes",
        "telnetlib",
        "select",
        "sunau",
        "unicodedata",
        "argparse",
        "zoneinfo",
        "zipimport",
        "hmac",
        "plistlib",
        "functools",
        "nntplib",
        "spwd",
        "readline",
        "queue",
        "signal",
        "urllib",
        "fpectl",
        "crypt",
        "logging",
        "code"
    ],
    "extra_standard_library": [],
    "known_other": {},
    "multi_line_output": "VERTICAL_HANGING_INDENT",
    "forced_separate": [],
    "indent": "    ",
    "comment_prefix": "  #",
    "length_sort": false,
    "length_sort_straight": false,
    "length_sort_sections": [],
    "add_imports": [],
    "remove_imports": [],
    "append_only": false,
    "reverse_relative": false,
    "force_single_line": false,
    "single_line_exclusions": [],
    "default_section": "THIRDPARTY",
    "import_headings": {},
    "import_footers": {},
    "balanced_wrapping": false,
    "use_parentheses": true,
    "order_by_type": true,
    "atomic": false,
    "lines_before_imports": -1,
    "lines_after_imports": -1,
    "lines_between_sections": 1,
    "lines_between_types": 0,
    "combine_as_imports": false,
    "combine_star": false,
    "include_trailing_comma": true,
    "from_first": false,
    "verbose": false,
    "quiet": false,
    "force_adds": false,
    "force_alphabetical_sort_within_sections": false,
    "force_alphabetical_sort": false,
    "force_grid_wrap": 0,
    "force_sort_within_sections": false,
    "lexicographical": false,
    "group_by_package": false,
    "ignore_whitespace": false,
    "no_lines_before": [],
    "no_inline_sort": false,
    "ignore_comments": false,
    "case_sensitive": false,
    "sources": [
        {
            "py_version": "py3",
            "force_to_top": [],
            "skip": [
                ".direnv",
                ".svn",
                ".hg",
                ".bzr",
                ".mypy_cache",
                ".tox",
                "_build",
                ".git",
                ".pants.d",
                "node_modules",
                ".nox",
                ".venv",
                "dist",
                "buck-out",
                "venv",
                ".eggs",
                "__pypackages__",
                "build"
            ],
            "extend_skip": [],
            "skip_glob": [],
            "extend_skip_glob": [],
            "skip_gitignore": false,
            "line_length": 79,
            "wrap_length": 0,
            "line_ending": "",
            "sections": [
                "FUTURE",
                "STDLIB",
                "THIRDPARTY",
                "FIRSTPARTY",
                "LOCALFOLDER"
            ],
            "no_sections": false,
            "known_future_library": [
                "__future__"
            ],
            "known_third_party": [],
            "known_first_party": [],
            "known_local_folder": [],
            "known_standard_library": [
                "codeop",
                "xml",
                "keyword",
                "posixpath",
                "dummy_threading",
                "sysconfig",
                "timeit",
                "mailcap",
                "sre_constants",
                "fnmatch",
                "ntpath",
                "symtable",
                "getopt",
                "typing",
                "pickle",
                "pdb",
                "pickletools",
                "_dummy_thread",
                "reprlib",
                "ssl",
                "dis",
                "bz2",
                "audioop",
                "modulefinder",
                "bisect",
                "fileinput",
                "pkgutil",
                "site",
                "tempfile",
                "sndhdr",
                "turtledemo",
                "winreg",
                "msilib",
                "re",
                "enum",
                "codecs",
                "contextvars",
                "shelve",
                "tarfile",
                "lzma",
                "grp",
                "configparser",
                "inspect",
                "binascii",
                "shutil",
                "socketserver",
                "pwd",
                "colorsys",
                "posix",
                "weakref",
                "pprint",
                "collections",
                "gc",
                "binhex",
                "atexit",
                "optparse",
                "bdb",
                "locale",
                "numbers",
                "imghdr",
                "xmlrpc",
                "email",
                "chunk",
                "compileall",
                "selectors",
                "quopri",
                "curses",
                "encodings",
                "fractions",
                "cgitb",
                "mmap",
                "hashlib",
                "importlib",
                "gzip",
                "calendar",
                "string",
                "marshal",
                "webbrowser",
                "linecache",
                "sched",
                "ossaudiodev",
                "test",
                "rlcompleter",
                "datetime",
                "stringprep",
                "formatter",
                "getpass",
                "sre_parse",
                "traceback",
                "smtpd",
                "py_compile",
                "pydoc",
                "sre_compile",
                "sqlite3",
                "pathlib",
                "io",
                "distutils",
                "http",
                "warnings",
                "fcntl",
                "imaplib",
                "heapq",
                "abc",
                "tty",
                "errno",
                "shlex",
                "dataclasses",
                "graphlib",
                "html",
                "profile",
                "venv",
                "_ast",
                "decimal",
                "uuid",
                "csv",
                "cmath",
                "faulthandler",
                "pyclbr",
                "symbol",
                "tracemalloc",
                "ipaddress",
                "winsound",
                "ast",
                "zipapp",
                "glob",
                "difflib",
                "tkinter",
                "array",
                "time",
                "imp",
                "os",
                "subprocess",
                "zipfile",
                "stat",
                "concurrent",
                "xdrlib",
                "platform",
                "token",
                "doctest",
                "parser",
                "struct",
                "tabnanny",
                "asyncore",
                "runpy",
                "itertools",
                "msvcrt",
                "statistics",
                "resource",
                "cmd",
                "smtplib",
                "operator",
                "sys",
                "socket",
                "poplib",
                "pty",
                "asyncio",
                "cgi",
                "unittest",
                "types",
                "sre",
                "macpath",
                "wave",
                "builtins",
                "turtle",
                "termios",
                "zlib",
                "copyreg",
                "cProfile",
                "base64",
                "dbm",
                "secrets",
                "ftplib",
                "filecmp",
                "netrc",
                "tokenize",
                "ctypes",
                "pstats",
                "random",
                "mimetypes",
                "aifc",
                "lib2to3",
                "gettext",
                "nis",
                "_thread",
                "ensurepip",
                "multiprocessing",
                "textwrap",
                "wsgiref",
                "contextlib",
                "asynchat",
                "uu",
                "json",
                "threading",
                "syslog",
                "trace",
                "mailbox",
                "copy",
                "math",
                "pipes",
                "telnetlib",
                "select",
                "sunau",
                "unicodedata",
                "argparse",
                "zoneinfo",
                "zipimport",
                "hmac",
                "plistlib",
                "functools",
                "nntplib",
                "spwd",
                "readline",
                "queue",
                "signal",
                "urllib",
                "fpectl",
                "crypt",
                "logging",
                "code"
            ],
            "extra_standard_library": [],
            "known_other": {},
            "multi_line_output": "GRID",
            "forced_separate": [],
            "indent": "    ",
            "comment_prefix": "  #",
            "length_sort": false,
            "length_sort_straight": false,
            "length_sort_sections": [],
            "add_imports": [],
            "remove_imports": [],
            "append_only": false,
            "reverse_relative": false,
            "force_single_line": false,
            "single_line_exclusions": [],
            "default_section": "THIRDPARTY",
            "import_headings": {},
            "import_footers": {},
            "balanced_wrapping": false,
            "use_parentheses": false,
            "order_by_type": true,
            "atomic": false,
            "lines_before_imports": -1,
            "lines_after_imports": -1,
            "lines_between_sections": 1,
            "lines_between_types": 0,
            "combine_as_imports": false,
            "combine_star": false,
            "include_trailing_comma": false,
            "from_first": false,
            "verbose": false,
            "quiet": false,
            "force_adds": false,
            "force_alphabetical_sort_within_sections": false,
            "force_alphabetical_sort": false,
            "force_grid_wrap": 0,
            "force_sort_within_sections": false,
            "lexicographical": false,
            "group_by_package": false,
            "ignore_whitespace": false,
            "no_lines_before": [],
            "no_inline_sort": false,
            "ignore_comments": false,
            "case_sensitive": false,
            "sources": [],
            "virtual_env": "",
            "conda_env": "",
            "ensure_newline_before_comments": false,
            "directory": "",
            "profile": "",
            "honor_noqa": false,
            "src_paths": [],
            "old_finders": false,
            "remove_redundant_aliases": false,
            "float_to_top": false,
            "filter_files": false,
            "formatter": "",
            "formatting_function": null,
            "color_output": false,
            "treat_comments_as_code": [],
            "treat_all_comments_as_code": false,
            "supported_extensions": [
                "pxd",
                "pyx",
                "pyi",
                "py"
            ],
            "blocked_extensions": [
                "pex"
            ],
            "constants": [],
            "classes": [],
            "variables": [],
            "dedup_headings": false,
            "only_sections": false,
            "only_modified": false,
            "combine_straight_imports": false,
            "auto_identify_namespace_packages": true,
            "namespace_packages": [],
            "follow_links": true,
            "indented_import_headings": true,
            "honor_case_in_force_sorted_sections": false,
            "sort_relative_in_force_sorted_sections": false,
            "overwrite_in_place": false,
            "reverse_sort": false,
            "star_first": false,
            "git_ignore": {},
            "format_error": "{error}: {message}",
            "format_success": "{success}: {message}",
            "sort_order": "natural",
            "source": "defaults"
        },
        {
            "multi_line_output": 3,
            "include_trailing_comma": true,
            "force_grid_wrap": 0,
            "use_parentheses": true,
            "ensure_newline_before_comments": true,
            "line_length": 88,
            "source": "black profile"
        },
        {
            "profile": "black",
            "source": "/Users/andychen/dev/nira-data-pipeline/pipeline/pyproject.toml"
        }
    ],
    "virtual_env": "",
    "conda_env": "",
    "ensure_newline_before_comments": true,
    "directory": "/Users/andychen/dev/nira-data-pipeline/pipeline",
    "profile": "black",
    "honor_noqa": false,
    "src_paths": [
        "/Users/andychen/dev/nira-data-pipeline/pipeline/src",
        "/Users/andychen/dev/nira-data-pipeline/pipeline"
    ],
    "old_finders": false,
    "remove_redundant_aliases": false,
    "float_to_top": false,
    "filter_files": false,
    "formatter": "",
    "formatting_function": null,
    "color_output": false,
    "treat_comments_as_code": [],
    "treat_all_comments_as_code": false,
    "supported_extensions": [
        "pxd",
        "pyx",
        "pyi",
        "py"
    ],
    "blocked_extensions": [
        "pex"
    ],
    "constants": [],
    "classes": [],
    "variables": [],
    "dedup_headings": false,
    "only_sections": false,
    "only_modified": false,
    "combine_straight_imports": false,
    "auto_identify_namespace_packages": true,
    "namespace_packages": [],
    "follow_links": true,
    "indented_import_headings": true,
    "honor_case_in_force_sorted_sections": false,
    "sort_relative_in_force_sorted_sections": false,
    "overwrite_in_place": false,
    "reverse_sort": false,
    "star_first": false,
    "git_ignore": {},
    "format_error": "{error}: {message}",
    "format_success": "{success}: {message}",
    "sort_order": "natural"
}

karthiknadig commented 1 year ago

@achen187 One thing noticed is that when you ran it from terminal the directory is /Users/andychen/dev/nira-data-pipeline/pipeline, whereas when the extension is running with /Users/andychen/dev/nira-data-pipeline.

Temporary work around is to use this argument in the settings "isort.args": ["--settings-path", "${workspaceFolder}/pipeline/pyproject.toml"].

Would it be possible to share your project so I can repro this locally and see why there is difference in working directory.

achen187 commented 1 year ago

that fixes it! Yeah i suspected the toml file wasn't getting used anymore.

Suer, I just gave you read access. Yeah would be good to remove that setting from being required though. Seems like isort should know where to pick up the toml file correctly

karthiknadig commented 1 year ago

This might be a bug in isort itself. I will have to check with legal before I access your repo since it is not a public repo.

My suspicion is that since we are dealing with plain string. if you run isort from command like using this command you will run into the problem.

cd /Users/andychen/dev/nira-data-pipeline
cat /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py | python -m isort - --filename /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py

Even this might run into the same issue:

cd /Users/andychen/dev/nira-data-pipeline
python -m isort /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py

I think that the directory location is what is causing this issue.

achen187 commented 1 year ago

ah ok, I just cancelled your invite. Hm does the command change only because we're using this arg? "isort.args": ["--settings-path", "${workspaceFolder}/pipeline/pyproject.toml"]"

I feel like that argument shouldnt affect whether its plain string or not?

achen187 commented 1 year ago

This is the new command in the logs:

/Users/andychen/dev/nira-data-pipeline/pipeline/.venv/bin/python -m isort - --settings-path /Users/andychen/dev/nira-data-pipeline/pipeline/pyproject.toml --filename /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/get_tuples_caiso_model_data_for_all_models_helper/test_get_tuples_caiso_model_data_for_all_models_helper.py

achen187 commented 1 year ago

When I do the second option, it works:

andychen@Andys-MBP nira-data-pipeline % pwd
/Users/andychen/dev/nira-data-pipeline
andychen@Andys-MBP nira-data-pipeline % ./pipeline/.venv/bin/python -m isort /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py

achen187 commented 1 year ago

When I do the first option, it doesn't work, it returns what VScode was doing

andychen@Andys-MBP nira-data-pipeline % cat /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py | ./pipeline/.venv/bin/python -m isort - --filename /Users/andychen/dev/nira-data-pipeline/pipeline/pipeline_tests/caiso/sections/model_data/ops/remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1/test_remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.py

karthiknadig commented 1 year ago

The working directory is the problem here. isort starts its search for config from current working directory, and goes up. If you run isort from /Users/andychen/dev/nira-data-pipeline/pipeline it works because that is where the pyproject.toml is also located. so, from the terminal where it worked for you, it looked like the current working directory was /Users/andychen/dev/nira-data-pipeline/pipeline. The plain text or file is not really important here.

When I do the second option, it works

That tells me this is a bug with isort since the first method is what we need to do to sort before saving. I will file a bug on isort.

achen187 commented 1 year ago

from dagster import build_op_context
from pipeline_tests.test_helpers.file_helpers import (create_test_file,
                                                      nira_file_contents_equal)

from pipeline.caiso.sections.model_data.ops.remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1.remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1 import (
    _COMBINED_CONTINGENCIES_REMOVED_UNWANTED_TEXT_FILE_NAME,
    remove_unwanted_text_from_combined_contingency_file_caiso_cluster_14_phase_1)
from pipeline.helpers.file_helpers import (PARENT_DIR_CONFIG_KEY,
                                           nira_joinpath, nira_mkdir,
                                           nira_path_relative_to_cwd)

karthiknadig commented 1 year ago

Thanks for all the details. I think I have enough info to create a bug on isort with minimal repro instructions. I will file a bug and link it here.

achen187 commented 1 year ago

hm no, I dont think I'm following.

I tried both the first and second method from /Users/andychen/dev/nira-data-pipeline

The first method did not work, the second did. So that tells me it has to do with the text vs file invocation of isort. It doesn't have to do with the working directory @karthiknadig

karthiknadig commented 1 year ago

It has to do with, isort not searching properly when using the plain text method and when the directory is the parent of where the config is located.

achen187 commented 1 year ago

ah got it, so this bug only exists when using plain text method. Ok, sounds good.

mthanded commented 1 year ago

Wanted to also chime in here. I have spent the last day tracking down whats going on here. Having issues committing to any of my CI pipelines without manually calling isort. I'm also not seeing "Organize Imports" in the command menu.

karthiknadig commented 1 year ago

@mthanded As identified in the comment https://github.com/microsoft/vscode-isort/issues/138#issuecomment-1302662018 this is an issue with isort itself. IT should get resolved there.

As for the missing "Organize Imports" follow this issue: https://github.com/microsoft/vscode-isort/issues/139

fmunirdev commented 1 year ago

Temporary work around is to use this argument in the settings "isort.args": ["--settings-path", "${workspaceFolder}/pipeline/pyproject.toml"].

This works for me. But I still see the following error in Output > Log (Remote Extension Host):

2022-11-09 01:57:10.304 [info] ExtensionService#_doActivateExtension ms-python.isort, startup: false, activationEvent: 'onLanguage:python'
2022-11-09 01:57:11.805 [error] [ms-python.isort] provider FAILED
2022-11-09 01:57:11.805 [error] TypeError: Cannot read properties of null (reading 'map')
    at g (/home/fahad/.vscode-server/extensions/ms-python.isort-2022.4.0/dist/extension.js:1:214870)
    at Object.asCodeAction (/home/fahad/.vscode-server/extensions/ms-python.isort-2022.4.0/dist/extension.js:1:225810)
    at /home/fahad/.vscode-server/extensions/ms-python.isort-2022.4.0/dist/extension.js:1:154524
    at async x.provideCodeActions (/home/fahad/.vscode-server/bin/8fa188b2b301d36553cbc9ce1b0a146ccb93351f/out/vs/workbench/api/node/extensionHostProcess.js:94:43007)

karthiknadig commented 1 year ago

@fmunirdev the Cannot read properties of null (reading 'map') issue is this one, it should be fixed in the pre-release version for tomorrow https://github.com/microsoft/vscode-isort/issues/77

karthiknadig commented 1 year ago

Files an issue ion isort for this please upvote that: https://github.com/PyCQA/isort/issues/1989

karthiknadig commented 1 year ago

Duplicate of: https://github.com/microsoft/vscode-isort/issues/53

microsoft / vscode-isort

`isort` does not pickup configuration when it is not in current working directory #138