jupyter-autopep8 reverses shell commands and Python imports

[drscotthawley]

Consider the following original code:

!pip install the_package
import the_package

Which jupyter-autopep8 reverses the order of, producing

import the_package
!pip install the_package

...thus breaking the code.

A workaround would of course be for the user to manually insert a new cell above in which to perform shell commands independently of Python code.

...Although Jupyter does allow Python programmatic execution of '!'-style shell commands, so this workaround wouldn't work for everybody. And yet for the following code, jupyter-autopep8 doesn't seem to change the order, leaving it as is:

if True:
    !pip install the_package
    import the_package

...so another workaround might be to if True: all your shell commands. ;-) But in fact this trick doesn't always work, I've got longer code examples where even with the if True:, it'll stick the shell command at the end of a very long list of imports.... or even stick it in the middle of the list of imports (?). I can't figure out what's triggering the reordering.

Ideally, 'prettifying' lines of code wouldn't change the order in which they are executed.

[drscotthawley]

According to PEP8,

"Imports are always put at the top of the file, just after any module comments and docstrings, and before module globals and constants."

...but given that the '!' construction is jupyter-specific, perhaps there needs to be an exemption for shell commands.

Context: Arrived at trying jupyter-autopep8 after trying the code_prettify extension and seeing it give syntax errors at shell commands. So...are any of the jupyter code-formatters compatible with shell commands?