Open matteoettam09 opened 2 years ago
I have the same problem. isort suddenly started failing in my CI (in docker), while running it locally doesn't do anything. isort version is 5.10.1, python 3.8.12.
Hey @danielgafni and @matteoettam09, this issue seems relevant to this earlier one: https://github.com/PyCQA/isort/issues/1790, I don't exactly know if this is applicable here but sharing if this gives any insight. In the meantime, could you provide me the link to the repo so that I could have a look?
I'm sorry, my project is private.
I can only share my observations... the import line that was causing the issue was an import from the "scripts" directory.
The project only had one import from this directory. All the other imports were either from standard/external libraries or from the project package.
I hope this helps...
Hi @anirudnits, thank you. My project is unfortunately private as well.
In the meantime, I've figured it is possible to run the CI/CD locally, at least with Gitlab, so this could help identify the root cause.
I can only share my observations... the import line that was causing the issue was an import from the "scripts" directory.
Can you provide the exact error? Like is this an issue where in the cicd pipeline the import from scripts
directory is being classified as something other than firstparty?
@matteoettam09 Can you run isort
with verbose option to identify exactly what the issue is?
~ Aniruddha
Another private repo here. We have a number of packages pulled directly from gitlab like this in our requirements.txt:
-e git+ssh://git@gitlab.com/OurOrg/library_project.git#egg=library_project
In CI and on one developers machine these are listed as FIRST PARTY. On a second developers machine they are THIRD PARTY as reported by isort. Fixable by using known_first_party
in pyproject.toml file.
Both developers using virtual environments, (python 3.8.10) but the FIRST PARTY developer has virtualenv as a sub-directory of the main project. The THIRD PARTY developer has a separate /envs directory for virtualenvs in their home directory.
Don't know if it helps, just another datapoint.
Yes! This happened to me too when I moved my .venv directory into my project dir!
I encounter the same issue.
I run isort via pre-commit both locally in docker container and in CircleCI container, both use the same docker image.
However locally isort behaves like the right side of the diff, and cicd the left side.
default_language_version:
python: python3.8
- repo: https://github.com/pycqa/isort
rev: 5.10.1
hooks:
- id: isort
name: isort (python)
language_version: python3.8
- id: isort
name: isort (cython)
language_version: python3.8
types: [cython]
- id: isort
name: isort (pyi)
language_version: python3.8
types: [pyi]
Ps. delta is an open source package delta-spark.
Versions: python 3.8 pre-commit 2.17 isort 5.10.1 local OS: macos
same problem encountered as @Magicbeanbuyer, i'm using python 3.8.11 and isrot 5.10.1 (with black profile). could you help here @timothycrosley
I ran into the same issue and was able to reproduce it here.
In my case, the project has the following structure:
.
├── app
│ ├── pyproject.toml
│ └── util
│ └── constants.py
├── main.py
pyproject.toml
is:
[tool.isort]
force_sort_within_sections = true
If I move pyproject.toml
to the top level of the project structure, I am unable to reproduce the error. See the linked repository for a detailed log
I got something similar. isort behaves differently at local with isort --check--profile black .
. In addition, it behaves differently at project root and project sub-directory.
Here is my project structure:
.
├── Dockerfile
├── README.md
├── airflow
│ ├── airflow.cfg
│ ├── dags
│ │ ├── data_dag.py
If I run isort --check --profile black ./
at root. It returns:
> Skipped 1 files
If I run isort --check --profile black ./
at dags
folder. It returns:
> ERROR: ./airflow/dags/data_dag.py Imports are incorrectly sorted and/or formatted.
And the changes from isort --check --diff --profile black ./
:
from datetime import datetime, timedelta
import pendulum
-from utils.bash_util import gen_bash_command
-from utils.convert_util import A2B
-from utils.path_util import gen_path
-
from airflow import models
from airflow.operators.bash import BashOperator
from airflow.operators.python import PythonOperator
from airflow.providers.amazon.aws.hooks.s3 import S3Hook
+
+from utils.bash_util import gen_bash_command
+from utils.convert_util import A2B
+from utils.path_util import gen_path
default_language_version
does not work because isort
hook specifies language version.
See comment: https://github.com/pre-commit/pre-commit/issues/2586#issuecomment-1305994077
@rafrafek do you mean that you expect your PR #1987 to fix this issue?
No, sorry for the confusion. I just found this issue by searching for default_language_version
. I should create a new issue.
For me the issue manifested when installing a first party module from a git submodule, it was detected as Firstparty on my system (Windows11 python3.10) and some of the imports were incorrectly detected as Thirdparty on the ci system (Ubuntu20.04 python3.10) with all else being equal.
isort vvv outputs
local
from-type place_module for my_package returned FIRSTPARTY
from-type place_module for my_package.something returned FIRSTPARTY
ci
from-type place_module for my_package returned FIRSTPARTY
from-type place_module for my_package.something returned THIRDPARTY
As workaround I have added the following to my isort config.
[settings]
known_first_party=my_package,my_package.something
@anirudnits can we get the tags on this issue updated? I posted a MWE above, here's the link again
This is affecting us, too. Has anyone isolated the issue?
Not sure if related, but I'm seeing things like
else-type place_module for foo returned FIRSTPARTY
else-type place_module for foo.scan returned THIRDPARTY
I do have more than one foo directory if that matters, but only one is in the Python module search path, and that's the one which also has the scan subdirectory.
Now, I'm a first-time isort user so accompanying question: can I just 'merge' sections and tell isort to treat future/stdlib as one section, and have everything else go in another section? I don't really care about whether something is 'third' of 'first' anyway.
Not sure if related, but I'm seeing things like
else-type place_module for foo returned FIRSTPARTY else-type place_module for foo.scan returned THIRDPARTY
I do have more than one foo directory if that matters, but only one is in the Python module search path, and that's the one which also has the scan subdirectory.
Now, I'm a first-time isort user so accompanying question: can I just 'merge' sections and tell isort to treat future/stdlib as one section, and have everything else go in another section? I don't really care about whether something is 'third' of 'first' anyway.
you could try my workaround.
As workaround I have added the following to my isort config.
[settings] known_first_party=my_package,my_package.something
@Gornoka yes I'm aware, but I'd really rather not start manually listing modules, that's not really manageable.
I've seen the same behavior in this job. Locally, isort
separates the 3rd party groups, however the workflow groups them together.
In my case, my working copy was in folder named github
, which is the same name as an external module. Assumedly, this caused the external module to appear as local, which was obviously different in CI. Once I changed the local folder name to something else (ansible-github
), the CI action and my local action resulted in the same behavior.
So, in case others run into a similar issue, if your imports match the name of the current folder, you may have unexpected sorting results.
I've seen the same behavior in this job. Locally,
isort
separates the 3rd party groups, however the workflow groups them together.In my case, my working copy was in folder named
github
, which is the same name as an external module. Assumedly, this caused the external module to appear as local, which was obviously different in CI. Once I changed the local folder name to something else (ansible-github
), the CI action and my local action resulted in the same behavior.So, in case others run into a similar issue, if your imports match the name of the current folder, you may have unexpected sorting results.
I had the same issue, where isort would not change my files in local but then fail in the CI/CD job. Tested a lot of stuff, ensuring versions are the same, and even updated my Docker image for the runner.
Turns out, we have a folder named "NLTK" within the repo, and the error was happening because import nltk
would be moved around wrongly. The solution for me was to add :
known_third_party=nltk
to the .isort.cfg
, and that did the trick. I guess another solution for us would be to rename NLTK
to nltk_utils
instead. But it is working now, so thanks for this thread!
This same approach solved for me too.
We had the following import, the local pre-commit hook vs CI/CD pipeline were fighting over its placement:
from minio import Minio
I happened to have a subdir minio
created during development to store a local minio db. isort apparently considered that the name matched an import therefore it must be a first-party python package dir. The pipeline on the other hand didn't have the minio
subdir.
Renaming it to minio-db
fixed the issue.
Problem occurs for me because the pyproject.toml was located at a higher directory level than what I'm trying to isort, so it creates a found-local-dir vs installed-package conflict (because isort treats those differently). My solution was moving the pyproject.toml to be next to the package I'm trying to sort imports in instead of up a directory.
Why that only happens in my CI environment and not locally, I don't readily know, but here we are.
Hello,
have these lines in my .pre-commit.yaml:
and locally I have version 5.9.3 of isort. When I run pre-commit run --all-files locally, the isort is passed succesfully, but when the pre-commit is performed in git, there are some differences:
Locally I am in a venv, while remotely this is performed inside a docker image:
Any idea why this inconsistency? Thank you.