Different separation of the imports during pre-commit hook and cli command

[dimaka-wix]

I get different results of sorting imports during pre-commit hook and CLI command:

1. pre-commit:

- repo: https://github.com/asottile/reorder_python_imports
  rev: v2.5.0
  hooks:
    - id: reorder-python-imports
      stages: [push]

The result is:

import os
from typing import Text
from typing import TypedDict

from chaoshub_aws.ec2 import AwsEc2
from chaoshub_aws.ec2.decorator import ec2_injector

2. reorder-python-imports test_file.py The result is:

import os
from chaoshub_aws.ec2 import AwsEc2
from chaoshub_aws.ec2.decorator import ec2_injector
from typing import Text
from typing import TypedDict

I tried to path the same arguments in both CLI command and pre-commit hook, but the results are still different. I guess I'm just missing something ..?

[asottile]

I believe the tool is classifying your chaoshub_aws as "stdlib" because it is on sys.path and not in a *-packages root

[dimaka-wix]

You are definitely right! It is on sys.path

But is it possible that at the same time, the tool receives these libraries as the first-party ones during the pre-commit hook?

Otherwise, why it works differently running reorder-python-imports in CLI and during the pre-commit hook? And how can I set it to work the same in both cases?

[asottile]

the pre-commit hook runs from inside an isolated virtualenv

how is it on sys.path? you might want to adjust how that works so it is installed as things normally are into the various site-packages / dist-packages directories (instead of however you're doing it now)

[asottile]

(the pre-commit results are correct by the way, since they side-step the sys.path hackery that's happening outside of pre-commit)

[dimaka-wix]

I have a monorepo with several plugins. chaoshub_aws is the folder inside one of those plugins. It includes different services, resources, and wrappers.

To work with the monorepo I'm installing virtualenv with all the required packages. The path/lib/python3.8/site-packages is located inside of venv and all the plugins use these libraries

So in case of from chaoshub_aws.ec2 import AwsEc2 AwsEc2 is a kind of part of the application itself.

Do you mean that it should be inside of site-packages? Or am I missing your point? Thanks for such a quick response! =]

[asottile]

hmmm could you speak more about your actual layout?

does it look something like:

src/
    chaoshub_aws/...
venv/lib/site-packages/...

or ?

is it maybe something where --application-directories is what you intend to use? (see the README for more information)

(it's also possible that under pre-commit it's "wrong" as well in that case -- but without a third party library it's unclear whether it would be split into three sections correctly or not)

[dimaka-wix]

Sure! It looks like that:

> ROOT
|  > .vscode
|  > chaos-hub-experiments
|  > chaoshub-cli
|  |  > venv
|  |  |  > lib/site-packages/...
|  |  > scripts
|  |  > ...
|  > services
|  |  > aws-plugin
|  |  |  > chaoshub_aws
|  |  |  |  > ec2
|  |  |  |  > ...
|  |  |  > ...
|  |  |  > ...
|  |  > plugin2    
|  |  > plugin3
. . . .
. . . .

venv is installed ones, after running the chaoshub init command(one of the scripts)

[asottile]

sounds like you want services/aws-plugin as an application directory

[dimaka-wix]

Yes in this specific case, but when I make some changes inside another plugin, services/another-plugin going to be an application directory

[dimaka-wix]

And .pre-commit-config.yaml is common for all the plugins and is placed in ROOT directory

[asottile]

you can configure multiple invocations:

    hooks:
    -   id: reorder-python-imports
        name: reorder-python-imports (aws-plugin)
        args: [--application-directories=.:services/aws-plugins]
    -   id: reorder-python-imports
        name: reorder-python-imports (plugin2)
        args: [--application-directories=.:services/plugin2]
    # etc., etc.

[dimaka-wix]

Thank you so much!!! Finally, I understand what the problem was! The migration of several repositories into one large monorepo caused this problem! But thanks to you, I understood how to set up the proper configurations Now it works correctly =)

[asottile]

awesome!