python sendfile that have functions with blank line insides does not work

[NobodyXu]

I am using vim 8.1 and tmux 2.3 on debian 9.8, my vimcmdline configuration is as follows:

" vimcmdline mappings
let cmdline_map_start          = '\s'
let cmdline_map_send_and_stay  = '\l'
" Send file
let cmdline_map_source_fun     = '\f'
let cmdline_map_send_paragraph = '\p'
let cmdline_map_send_block     = '\b'
let cmdline_map_quit           = '\q'

" vimcmdline options
let cmdline_vsplit      = 1      " Split the window vertically
let cmdline_in_buffer   = 0      " Start the interpreter in tmux since vimcmdline doesn't support vim term

I also use pylint through pymode which strip all trailing space, including the empty lines inside functions.

And when I was sending split.py that have functions with blank lines inside:

# split.py
def find(s, start, predictor):
    for i in range(start, len(s)):
        if predictor(s[i]):
            return i
    return -1

def find_all(s, sep=" \t\n"):
    beg_of_nonsep = 0
    while beg_of_nonsep < len(s):
        beg_of_nonsep = find(s, beg_of_nonsep, lambda ch, sep_chs=sep: ch not in sep_chs)
        if beg_of_nonsep == -1:
            break

        end_of_nonsep = find(s, beg_of_nonsep + 1, lambda ch, sep_chs=sep: ch in sep_chs)
        if end_of_nonsep == -1:
            end_of_nonsep = len(s)

        yield (beg_of_nonsep, end_of_nonsep)

        beg_of_nonsep = end_of_nonsep + 1

split = lambda s: [s[beg: end] for (beg, end) in find_all(s)]

print(split(""))
print(split("     \t\n"))
print(split("     \tssss\n"))

it did not work:

Type "help", "copyright", "credits" or "license" for more information.
>>> def find(s, start, predictor):
...     for i in range(start, len(s)):
...         if predictor(s[i]):
...             return i
...     return -1
...
>>> def find_all(s, sep=" \t\n"):
...     beg_of_nonsep = 0
...     while beg_of_nonsep < len(s):
...         beg_of_nonsep = find(s, beg_of_nonsep, lambda ch, sep_chs=sep: ch not in sep_chs)
...         if beg_of_nonsep == -1:
...             break
...
>>>         end_of_nonsep = find(s, beg_of_nonsep + 1, lambda ch, sep_chs=sep: ch in sep_chs)
  File "<stdin>", line 1
    end_of_nonsep = find(s, beg_of_nonsep + 1, lambda ch, sep_chs=sep: ch in sep_chs)
    ^
IndentationError: unexpected indent
>>>         if end_of_nonsep == -1:
  File "<stdin>", line 1
    if end_of_nonsep == -1:
    ^
IndentationError: unexpected indent
>>>             end_of_nonsep = len(s)
  File "<stdin>", line 1
    end_of_nonsep = len(s)
    ^
IndentationError: unexpected indent
>>>
>>>         yield (beg_of_nonsep, end_of_nonsep)
  File "<stdin>", line 1
    yield (beg_of_nonsep, end_of_nonsep)
    ^
IndentationError: unexpected indent
>>>
>>>         beg_of_nonsep = end_of_nonsep + 1
  File "<stdin>", line 1
    beg_of_nonsep = end_of_nonsep + 1
    ^
IndentationError: unexpected indent
>>>
>>> split = lambda s: [s[beg: end] for (beg, end) in find_all(s)]
>>>
>>> print(split(""))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
TypeError: 'NoneType' object is not iterable
>>> print(split("     \t\n"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 1, in <lambda>
TypeError: 'NoneType' object is not iterable
>>> print(split("     \tssss\n"))

It ignores much of the function find_all, so the code is completely broken. (The third print actually never returned until I killed it with <ctrl + c>.

However, when I use python3 ./split.py to run the code, nothing went wrong:

[]
[]
['ssss']

It gave me exactly what I want.

[NobodyXu]

@jalvesaq Can I have your attention please? This problem here is really annoying.

It took me 1 hour googleing, debugging and even posting on stackoverflow and got deleted since there was no problem to my code at all before I figured it out that it is actually vimcmdline that is buggy.

[NobodyXu]

@jalvesaq According to here, this behavior is caused by difference in how python handle end of code block in interpreter mode and running a file:

Therefore the command line operates with one specific difference. A blank line will also end a code block. ...

One approach to fix this problem

is to change the action of running a file from copy paste line by line to:

On the first time sending file:

1. Type import importlib after vimcmdline is started as an initialization;
2. Save the file to its original file/the temporary file of which vim opens;
3. Type import filename into python interpreter on the first send file;
4. Type from filename import *.

Else:

1. Type import filename into python interpreter on the first send file;
2. Type importlib.reload(filename) into python interpreter;
3. Type from filename import *.

Note: The above approach is complex because from path import * have to be used to ensure compatibility. This is documented here. Using import file and importlib.reload(file) can be much easier to implement (and stateless).

Or, if importing importlib is not desirable and from path import * is needed:

1. Save the file to a temporary file that its name differs every time executed;
2. Type from tmpfile import * in python interpreter.

Note: temp file can be created by using mktemp(a command), mkstemp function family(glibc funciton).

Similar thing can be done to other functionalities which involves code block:

1. Save the desired block to a temporary file;
2. Type from tmp import * in python interpreter.

This approach can fix the problem and makes the python interpreter split looks cleaner since there will not be a lot of copy-paste. It will be really nice to have this since sometimes, a code block/function/file can be really long.

This approach requires a bit of modification to the part that send code block/function/file and it is actually inspired by how nvim-R do the same things in different way.

Another way to fix this

is to automatically indent the line when the next line is also indented. In this way, sending line, block, function, file will all get fixed.

This may requires less modification to the code compared to the first approach, but is has the following cons is that it still gives a lot of output when executing a long, long code block/function/file.

Personally, as a heavy python user

I would recommend to use the first approach to fix sending code block/file/function since I really want a cleaner output when sending long, long lines of code and use the second to fix the send line function.

[jalvesaq]

Please, try this branch: https://github.com/jalvesaq/vimcmdline/tree/pylines

Sending from vimcmdlinetmp.py import * works only once. If we do any change to the function and send it again Python seems to ignore the import command.

At least @kprimice @ianliu and @Mizudera have contributed with code to the PythonSourceLines function. They might be able to help with this issue.

[NobodyXu]

I tried that branch and it does solve the indentation problem.

In order to solve the from vimcmdlinetmp import only works once problem, the only way I found that can do this is to use importlib:

import vimcmdlinetmp # Import the file as a module so that importlib can reload it
import importlib
importlib.reload(vimcmdlinetmp) # Reload!

from vimcmdlinetmp import *

Edit:

I am currently working on a branch to fix this.

Though I cannot program in vim script, but I do can understand your commit in branch pylines and can program in python.

[jalvesaq]

I've merged your pull request. Now we have to wait for comments from other people.

[smilesun]

when i run localleader+f, i got %cpaste -q, which got stuck there, i have to use ctrl+c to escape, then there are quite lot of error message prompt_toolkit/eventloop/posix.py", line 154, in _run_task then the file got loaded.

[NobodyXu]

@smilesun Can you post the error message so that I can know where goes wrong and fix them? Thank you.

[smilesun]

@NobodyXu , sorry, I have been busy and almost forgot about this, for some reason, it already works with leader+f, does it work for you already?

[smilesun]

see #83

[NobodyXu]

Well, I tested leader+f on master, and it seems to be working for me:

[jalvesaq]

I code in Python only occasionally, and what people are asking in many issues and pull requests is a support for Python as good as the support for R provided by Nvim-R. However, I don't have either knowledge of Python internals or spare time for this. Note that I have never added support for R in vimcmdline because Nvim-R already does this. So, it would be really great if someone created a plugin like Nvim-R for Python.

[blayz3r]

Similar issue https://github.com/prompt-toolkit/ptpython/issues/144

[LutfiLokman]

Sending file in python needs to be handled differently than sending all lines. I suggest creating another function for saving and sending the whole file. Map it to a different key (I use F10). All variables in the file (or module) can be accessed in the interpreter afterwards

function! VimCmdLineSendFile()
    saveas %:t
    call VimCmdLineSendCmd("exec(open(" . "'" . expand('%:t') . "'" . ").read())")
endfunction

nmap <F10> :call VimCmdLineSendFile()<CR>

[jalvesaq]

It would be great if someone created a plugin to support Python as I have done for R (Nvim-R). Then, I would remove support to Python from vimcmdline.

Note that I have never added support for R in vimcmdline.

[jalvesaq]

I guess it works now... If not, please, reopen the issue.