Detect isort config files and restart server on change.

[pfmoore]

My project has a .isort.cfg file specifying profile=black. The extension doesn't take account of this file, so I need to add the profile to my extension settings as well (and keep the two in line).

The extension should respect any of the config locations that isort supports.

[karthiknadig]

@pfmoore We don't do anything special in the extension that causes it to not find isort. This might be a bug in isort itself where running it via runpy.run_module causes it to skip config file detection.

I will file a bug on isort. Can you provide details on the relative location of .isort.cfg compared to the workspace directory? We run isort with current working directory set to the workspace root.

[pfmoore]

My setup was .isort.cfg in the project root, alongside pyproject.toml. If I run isort from the command line, via pre-commit, it successfully reformats my source code. But then, when I open the file in VS Code, it shows errors against all of the import lines, indicating that the isort extension thinks they are formatted incorrectly. Adding a config to the isort extension setting args --profile=black causes the highlighting to go away.

Let me know if you need more than this, or if there's any way I can do any diagnosis for you.

[pfmoore]

For what it's worth, I can confirm that isort does pick up the config file when run with runpy:

>>> import runpy
>>> import sys
>>> sys.argv=["isort", "--check", "src"]
>>> runpy.run_module("isort", run_name="__main__")
ERROR: C:\Work\Projects\pkg_metadata\src\pkg_metadata\__init__.py Imports are incorrectly sorted and/or formatted.

(This is using my .isort.cfg, but with a file formatted using the default config).

[alkatar21]

I use a pyproject.toml in the root directory and it seems to be used by isort. At least I just added "isort.args": ["--show-config"], to my settings and there under "sources" the pyproject.toml setting appears. However, after a change in pyproject.toml I have to call isort Formatter: Restart Server once before settings changes are applied by isort.

[pfmoore]

I can try (tomorrow) putting my isort config into pyproject.toml rather than .isort.cfg to see if that makes a difference. But that would only be a workaround - I want to keep my config out of pyproject.toml if possible (for reasons that aren't particularly relevant here).

[alkatar21]

@pfmoore Can you add "isort.args": ["--show-config"], to your settings and see if your config is under "sources" included as one of the last. For pyproject.toml I can confirm that the settings are used correctly. Also could you check if the CWD in the Output window -> isort formatter is the one where your config is?

[alkatar21]

@pfmoore I just tried it myself and it also works with a .isort.cfg:

• Add cfg to root folder:

  [settings]
  profile=black
  line_length=110

• Add "isort.args": ["--show-config"], to the settings
• Call isort Formatter: Restart Server
• Use a testfile and format imports
• In the output under "sources":

      {
          "profile": "black",
          "line_length": 110,
          "source": "<CWD>/.isort.cfg"
      }

So it doesn't seem to be a bug in isort or the extension. It works as expected.

[pfmoore]

Hmm, when I did that all I got in the isort output was

isort - --show-config --check --filename c:\Work\Projects\pkg_metadata\src\pkg_metadata\__init__.py
CWD Formatter: c:\Work\Projects\pkg_metadata

But if I re-save my settings, I get the following in the isort output:

isort - --check --filename c:\Work\Projects\pkg_metadata\src\pkg_metadata\__init__.py
CWD Formatter: c:\Work\Projects\pkg_metadata
CWD Format Server: c:\Work\Projects\pkg_metadata
sys.path used to run Formatter:
    c:\Users\Gustav\.vscode\extensions\ms-python.isort-2022.3.11611002\bundled\formatter
    C:\Users\Gustav\AppData\Local\Programs\Python\Python310\python310.zip
    C:\Users\Gustav\AppData\Local\Programs\Python\Python310\DLLs
    C:\Users\Gustav\AppData\Local\Programs\Python\Python310\lib
    C:\Users\Gustav\AppData\Local\Programs\Python\Python310
    c:\Work\Projects\pkg_metadata\.venv
    c:\Work\Projects\pkg_metadata\.venv\lib\site-packages
    c:\Users\Gustav\.vscode\extensions\ms-python.isort-2022.3.11611002\bundled\libs

Settings used to run Formatter:
[
    {
        "workspace": "file:///c%3A/Work/Projects/pkg_metadata",
        "trace": "error",
        "args": [
            "--show-config"
        ],
        "severity": {},
        "path": [],
        "interpreter": [
            "c:\\Work\\Projects\\pkg_metadata\\.venv\\Scripts\\python.exe"
        ]
    }
]

isort - --show-config --check --filename c:\Work\Projects\pkg_metadata\metadata_class.py
CWD Formatter: c:\Work\Projects\pkg_metadata
isort - --show-config --check --filename c:\Work\Projects\pkg_metadata\src\pkg_metadata\__init__.py
CWD Formatter: c:\Work\Projects\pkg_metadata

I see nothing in there about which config files are used, though...

Moving my config between pyproject.toml and .isort.cfg, and even using an invalid profile profile="bloop" doesn't seem to make any consistent difference...

[brettcannon]

FYI the code that runs isort can be found at:

https://github.com/microsoft/vscode-isort/blob/a01f5995d00dafb5f0c5c109db7b23d94664a889/bundled/formatter/format_server.py#L139-L146

[brettcannon]

@pfmoore are you using the isort we ship in the extension or your own copy?

[alkatar21]

You must call isort Formatter: Restart Server after each change to a config.

And the output of --show-config appears in the python file itself and is around 700 lines long (And i use the bundled isort.).

[pfmoore]

@brettcannon as far as I know the one shipped with the extension. How could I tell?

@alkatar21 I followed all of the steps you gave. You say "In the output under "sources"", but I don't see that at all. I can only assume I'm doing something wrong, but I have no idea what. (You say "Use a testfile and format imports" - I'm using right click -> "Sort imports", is that what you meant here?)

[alkatar21]

@pfmoore Oh, sorry about my inaccuracy, I'm not part of Microsoft or otherwise involved in development and am just trying to help. My complete settings are:

    "[python]": {
        "editor.codeActionsOnSave": { // Format imports with isort.
            "source.organizeImports": true
        },
    },
    "isort.args": ["--show-config"],

I have saved the file and isort runs automatically when saving with this setting. (Then there is the mentioned output)

Right click -> Sort imports: I do not know how that works. For me there is currently no isort executed then, only with Alt+Shift+o I can choose between sort imports and isort: Organize imports.

[pfmoore]

Ah, that did it. Here's the --show-config output:

Output

``` { "_known_patterns": null, "_section_comments": null, "_section_comments_end": null, "_skips": null, "_skip_globs": null, "_sorting_function": null, "py_version": "py3", "force_to_top": [], "skip": [ "venv", "__pypackages__", ".mypy_cache", "dist", ".git", ".venv", ".hg", "_build", ".pants.d", ".eggs", ".nox", "build", ".bzr", ".tox", "node_modules", ".svn", ".direnv", "buck-out" ], "extend_skip": [], "skip_glob": [], "extend_skip_glob": [], "skip_gitignore": false, "line_length": 110, "wrap_length": 0, "line_ending": "", "sections": [ "FUTURE", "STDLIB", "THIRDPARTY", "FIRSTPARTY", "LOCALFOLDER" ], "no_sections": false, "known_future_library": [ "__future__" ], "known_third_party": [], "known_first_party": [], "known_local_folder": [], "known_standard_library": [ "imaplib", "test", "pkgutil", "lzma", "sre_constants", "optparse", "ensurepip", "keyword", "poplib", "decimal", "doctest", "mailcap", "threading", "code", "gzip", "site", "statistics", "operator", "syslog", "macpath", "readline", "_dummy_thread", "copy", "subprocess", "asyncore", "difflib", "codecs", "resource", "graphlib", "collections", "hmac", "weakref", "socketserver", "http", "curses", "dummy_threading", "bdb", "warnings", "imghdr", "mmap", "pathlib", "builtins", "signal", "sndhdr", "struct", "array", "pty", "winreg", "fpectl", "token", "io", "string", "sys", "contextlib", "mailbox", "ipaddress", "nntplib", "fnmatch", "formatter", "cgitb", "xml", "parser", "multiprocessing", "json", "dataclasses", "profile", "sunau", "pickletools", "marshal", "tkinter", "posix", "math", "compileall", "heapq", "imp", "zoneinfo", "ssl", "ntpath", "zipfile", "bz2", "pyclbr", "selectors", "binhex", "shelve", "csv", "binascii", "importlib", "inspect", "msvcrt", "ossaudiodev", "enum", "cgi", "_thread", "getpass", "typing", "mimetypes", "encodings", "abc", "zipapp", "py_compile", "tarfile", "colorsys", "sre", "tokenize", "pstats", "pprint", "concurrent", "posixpath", "nis", "logging", "chunk", "tempfile", "pydoc", "types", "timeit", "select", "faulthandler", "zlib", "tracemalloc", "symtable", "dis", "turtledemo", "uuid", "tabnanny", "socket", "sysconfig", "webbrowser", "rlcompleter", "zipimport", "shutil", "urllib", "atexit", "shlex", "secrets", "pipes", "functools", "argparse", "filecmp", "lib2to3", "cmd", "ast", "locale", "gc", "getopt", "turtle", "crypt", "quopri", "pdb", "termios", "hashlib", "random", "wsgiref", "xdrlib", "distutils", "unittest", "uu", "re", "asynchat", "symbol", "codeop", "sqlite3", "winsound", "sre_parse", "spwd", "base64", "wave", "fractions", "smtplib", "sched", "runpy", "errno", "fileinput", "venv", "cmath", "datetime", "telnetlib", "configparser", "contextvars", "calendar", "fcntl", "plistlib", "msilib", "unicodedata", "linecache", "itertools", "traceback", "reprlib", "cProfile", "sre_compile", "platform", "textwrap", "time", "tty", "html", "os", "netrc", "trace", "audioop", "email", "aifc", "queue", "_ast", "pwd", "modulefinder", "xmlrpc", "grp", "stat", "pickle", "asyncio", "bisect", "smtpd", "ctypes", "dbm", "numbers", "stringprep", "copyreg", "ftplib", "gettext", "glob" ], "extra_standard_library": [], "known_other": {}, "multi_line_output": "VERTICAL_HANGING_INDENT", "forced_separate": [], "indent": " ", "comment_prefix": " #", "length_sort": false, "length_sort_straight": false, "length_sort_sections": [], "add_imports": [], "remove_imports": [], "append_only": false, "reverse_relative": false, "force_single_line": false, "single_line_exclusions": [], "default_section": "THIRDPARTY", "import_headings": {}, "import_footers": {}, "balanced_wrapping": false, "use_parentheses": true, "order_by_type": true, "atomic": false, "lines_before_imports": -1, "lines_after_imports": -1, "lines_between_sections": 1, "lines_between_types": 0, "combine_as_imports": false, "combine_star": false, "include_trailing_comma": true, "from_first": false, "verbose": false, "quiet": false, "force_adds": false, "force_alphabetical_sort_within_sections": false, "force_alphabetical_sort": false, "force_grid_wrap": 0, "force_sort_within_sections": false, "lexicographical": false, "group_by_package": false, "ignore_whitespace": false, "no_lines_before": [], "no_inline_sort": false, "ignore_comments": false, "case_sensitive": false, "sources": [ { "py_version": "py3", "force_to_top": [], "skip": [ "venv", "__pypackages__", ".mypy_cache", "dist", ".git", ".venv", ".hg", "_build", ".pants.d", ".eggs", ".nox", "build", ".bzr", ".tox", "node_modules", ".svn", ".direnv", "buck-out" ], "extend_skip": [], "skip_glob": [], "extend_skip_glob": [], "skip_gitignore": false, "line_length": 79, "wrap_length": 0, "line_ending": "", "sections": [ "FUTURE", "STDLIB", "THIRDPARTY", "FIRSTPARTY", "LOCALFOLDER" ], "no_sections": false, "known_future_library": [ "__future__" ], "known_third_party": [], "known_first_party": [], "known_local_folder": [], "known_standard_library": [ "imaplib", "test", "pkgutil", "lzma", "sre_constants", "optparse", "ensurepip", "keyword", "poplib", "decimal", "doctest", "mailcap", "threading", "code", "gzip", "site", "statistics", "operator", "syslog", "macpath", "readline", "_dummy_thread", "copy", "subprocess", "asyncore", "difflib", "codecs", "resource", "graphlib", "collections", "hmac", "weakref", "socketserver", "http", "curses", "dummy_threading", "bdb", "warnings", "imghdr", "mmap", "pathlib", "builtins", "signal", "sndhdr", "struct", "array", "pty", "winreg", "fpectl", "token", "io", "string", "sys", "contextlib", "mailbox", "ipaddress", "nntplib", "fnmatch", "formatter", "cgitb", "xml", "parser", "multiprocessing", "json", "dataclasses", "profile", "sunau", "pickletools", "marshal", "tkinter", "posix", "math", "compileall", "heapq", "imp", "zoneinfo", "ssl", "ntpath", "zipfile", "bz2", "pyclbr", "selectors", "binhex", "shelve", "csv", "binascii", "importlib", "inspect", "msvcrt", "ossaudiodev", "enum", "cgi", "_thread", "getpass", "typing", "mimetypes", "encodings", "abc", "zipapp", "py_compile", "tarfile", "colorsys", "sre", "tokenize", "pstats", "pprint", "concurrent", "posixpath", "nis", "logging", "chunk", "tempfile", "pydoc", "types", "timeit", "select", "faulthandler", "zlib", "tracemalloc", "symtable", "dis", "turtledemo", "uuid", "tabnanny", "socket", "sysconfig", "webbrowser", "rlcompleter", "zipimport", "shutil", "urllib", "atexit", "shlex", "secrets", "pipes", "functools", "argparse", "filecmp", "lib2to3", "cmd", "ast", "locale", "gc", "getopt", "turtle", "crypt", "quopri", "pdb", "termios", "hashlib", "random", "wsgiref", "xdrlib", "distutils", "unittest", "uu", "re", "asynchat", "symbol", "codeop", "sqlite3", "winsound", "sre_parse", "spwd", "base64", "wave", "fractions", "smtplib", "sched", "runpy", "errno", "fileinput", "venv", "cmath", "datetime", "telnetlib", "configparser", "contextvars", "calendar", "fcntl", "plistlib", "msilib", "unicodedata", "linecache", "itertools", "traceback", "reprlib", "cProfile", "sre_compile", "platform", "textwrap", "time", "tty", "html", "os", "netrc", "trace", "audioop", "email", "aifc", "queue", "_ast", "pwd", "modulefinder", "xmlrpc", "grp", "stat", "pickle", "asyncio", "bisect", "smtpd", "ctypes", "dbm", "numbers", "stringprep", "copyreg", "ftplib", "gettext", "glob" ], "extra_standard_library": [], "known_other": {}, "multi_line_output": "GRID", "forced_separate": [], "indent": " ", "comment_prefix": " #", "length_sort": false, "length_sort_straight": false, "length_sort_sections": [], "add_imports": [], "remove_imports": [], "append_only": false, "reverse_relative": false, "force_single_line": false, "single_line_exclusions": [], "default_section": "THIRDPARTY", "import_headings": {}, "import_footers": {}, "balanced_wrapping": false, "use_parentheses": false, "order_by_type": true, "atomic": false, "lines_before_imports": -1, "lines_after_imports": -1, "lines_between_sections": 1, "lines_between_types": 0, "combine_as_imports": false, "combine_star": false, "include_trailing_comma": false, "from_first": false, "verbose": false, "quiet": false, "force_adds": false, "force_alphabetical_sort_within_sections": false, "force_alphabetical_sort": false, "force_grid_wrap": 0, "force_sort_within_sections": false, "lexicographical": false, "group_by_package": false, "ignore_whitespace": false, "no_lines_before": [], "no_inline_sort": false, "ignore_comments": false, "case_sensitive": false, "sources": [], "virtual_env": "", "conda_env": "", "ensure_newline_before_comments": false, "directory": "", "profile": "", "honor_noqa": false, "src_paths": [], "old_finders": false, "remove_redundant_aliases": false, "float_to_top": false, "filter_files": false, "formatter": "", "formatting_function": null, "color_output": false, "treat_comments_as_code": [], "treat_all_comments_as_code": false, "supported_extensions": [ "pyi", "pyx", "py", "pxd" ], "blocked_extensions": [ "pex" ], "constants": [], "classes": [], "variables": [], "dedup_headings": false, "only_sections": false, "only_modified": false, "combine_straight_imports": false, "auto_identify_namespace_packages": true, "namespace_packages": [], "follow_links": true, "indented_import_headings": true, "honor_case_in_force_sorted_sections": false, "sort_relative_in_force_sorted_sections": false, "overwrite_in_place": false, "reverse_sort": false, "star_first": false, "git_ignore": {}, "format_error": "{error}: {message}", "format_success": "{success}: {message}", "sort_order": "natural", "source": "defaults" }, { "multi_line_output": 3, "include_trailing_comma": true, "force_grid_wrap": 0, "use_parentheses": true, "ensure_newline_before_comments": true, "line_length": 88, "source": "black profile" }, { "profile": "black", "line_length": 110, "source": "c:\\Work\\Projects\\pkg_metadata\\.isort.cfg" } ], "virtual_env": "", "conda_env": "", "ensure_newline_before_comments": true, "directory": "c:\\Work\\Projects\\pkg_metadata", "profile": "black", "honor_noqa": false, "src_paths": [ "C:\\Work\\Projects\\pkg_metadata\\src", "C:\\Work\\Projects\\pkg_metadata" ], "old_finders": false, "remove_redundant_aliases": false, "float_to_top": false, "filter_files": false, "formatter": "", "formatting_function": null, "color_output": false, "treat_comments_as_code": [], "treat_all_comments_as_code": false, "supported_extensions": [ "pyi", "pyx", "py", "pxd" ], "blocked_extensions": [ "pex" ], "constants": [], "classes": [], "variables": [], "dedup_headings": false, "only_sections": false, "only_modified": false, "combine_straight_imports": false, "auto_identify_namespace_packages": true, "namespace_packages": [], "follow_links": true, "indented_import_headings": true, "honor_case_in_force_sorted_sections": false, "sort_relative_in_force_sorted_sections": false, "overwrite_in_place": false, "reverse_sort": false, "star_first": false, "git_ignore": {}, "format_error": "{error}: {message}", "format_success": "{success}: {message}", "sort_order": "natural" } ``` I see in there: ``` { "multi_line_output": 3, "include_trailing_comma": true, "force_grid_wrap": 0, "use_parentheses": true, "ensure_newline_before_comments": true, "line_length": 88, "source": "black profile" }, { "profile": "black", "line_length": 110, "source": "c:\\Work\\Projects\\pkg_metadata\\.isort.cfg" } ] ```

So I guess that means my .isort.cfg is being read?

I wonder if the errors I was seeing were because I'd not reloaded the extension somehow? I'll do some more experimenting, but it'll have to be tomorrow now.

[alkatar21]

Yes, your config is read. So that is not the problem.

[pfmoore]

OK, having reviewed things, it looks pretty clear to me that the problem was that I created my .isort.cfg and then ran isort via pre-commit, which tidied up my files, but because I didn't restart the isort server in VS Code, the isort extension thought the corrected files were wrong.

Things just went worse from there, because changing the VS Code settings file appears to restart the isort server automatically, but changing the config file doesn't, so I ended up hopelessly confused as to what was going on 🙁

I have the following suggestions for making the behaviour less surprising:

1. Make the fact that you need to restart the isort server after a config change much more discoverable. I was going to say "add it to the documentation", but I'll be honest, I didn't read the documentation, and I didn't think there was any reason to assume the behaviour was complex enough that I'd need to read the docs.
2. Maybe monitor the config file(s) and automatically restart if they change? I can imagine this might not be as easy as it sounds, but it would be a huge boost in usability if it could be done.

Apart from those suggestions, I think this can be closed as "user error", though. Thanks for all the help.

[brettcannon]

@karthiknadig can you change this to a feature request to watch the config file(s) and restart the server as necessary?

[mjpieters]

[removed, known but different issue]