Support Python

[vale981]

Hi :),

I spent the afternoon porting julia-vterm.el to python. Do you think it would be worthwhile to support python directly in this project or is it better to leave them separate?

[shg]

Hi @vale981 I am already working on an experimental Python version https://github.com/shg/python-vterm.el . I would appreciate your contribution!

[vale981]

oops, i was unaware of that and made that: https://github.com/vale981/python-vterm.el

I can rebase onto your repo and create an PR if you want. Otherwise, I'm happy to maintain the python version.

[vale981]

ok, i cherry picked / pathched in what seemed to be missing from my efforts

[shg]

Oh, you are already working on your version of python-vterm.el . Thank you for your interest anyway. I am happy if we can collaborate, since I use mainly Julia and can not do sufficient testing of Python support. But it's totally up to you of course. You might think my version does not fit your need.

By the way, I am also working on ob-python-vterm.el for Babel Python block execution. My goal is to make robust REPL interaction and Org Babel support, for both Julia and Python, with consistent interface on Emacs. And python-vterm.el is a part of it.

[vale981]

I made my own version by accident. What I was trying to say was that I'm happy to turn it back over to you. Alternatively, I'm happy to maintain the python version as I'll be using it a lot.

A third option would be to unite the two versions into one package that, for example, also handles org-babel and I'd be happy to help with that.

Cheers, Valentin

[shg]

Thank you I understood. So, how about we first try integrating the two versions by way of you forking my version and applying your modifications to it? We'll continue working on that way for a while and see how it works out.

By the way, as you can see, my version is very primitive and there are still some design decisions to be made. Especially about whether to expect iPython or not, or how to handle environment, etc. I would like to avoid unnecessary dependency and make it as flexible as possible. Your suggestions are very welcome.

[vale981]

Well I worked my version from your Julia version so I think that should be alright ;). Or do you want me to re-fork it from the python version? I'll be dogfooding it over the next couple of weeks and work out the kinks. I already implemented a hack to make it use the current script buffer's environment.

[vale981]

Meant forked not worked... Phone autocorrection haha

[shg]

Or do you want me to re-fork it from the python version?

Sorry I overlooked this... Well it's up to you. But it may be easier to refork my version of python-vterm.el and modify it. What do you think? I assume you will post PRs including your modifications on my version.

[vale981]

It's not that easy. I already did most of the things you did and I cherry picked one of the commits from your repo, so it's basically based on that already. I don't want to potentially introduce bugs by rebasing just to have a slightly different history and a new fork relation.

[shg]

Hi @vale981 Sorry for delayed respose.

I understand that rebasing your work onto my version might seem like a hassle, especially after all the effort you've already put in. But I think it will be necessary at some point, if we are interested in working together for the further development of python-vterm.el.

Both julia-vterm/ob-julia-vterm and python-vterm/ob-python-vterm are not complete yet. I will be continuously improving them, and I intend to share functions and codes between the two to maintain as much consistency as possible. What do you think?

[vale981]

I feel that I might not understand what you're trying to tell me. What I'm saying is that the code is already based on your version and changing the git history won't do anything :P. If you want that code to live in your repo, then you could just reset the main branch to what I've done. I could then redirect to your repo if you're worried about ownership :).

Yesterday I began porting the repl restart feature from the Julia version and the code diverged significantly. Conversely, I added some features in the python version that would be handy to habe in the Julia version too. So either we work on making a common abstraction (that works for most interpreters) or the two projects will be very tedious to keep in sync.

Cheers, Valentin

[shg]

Hi @vale981

Thanks for your input. I’m not concerned about ownership here. It's under the GPL. What I really want to understand is whether our goals align. If they do, I’d love to have your help. If not, it might be better for us to develop our versions separately.

For instance, you removed the paragraph about Babel from the README. This makes me think that perhaps you’re not interested in Babel and just want a well-functioning inferior Python mode for Emacs. If that’s the case, our goals might be a bit different.

Regarding the repository location, I aim to develop julia-vterm.el, python-vterm.el, ob-julia-vterm.el, and ob-python-vterm.el in a consistent manner. Therefore, I’d prefer to keep all four projects in the same place. Currently, they are under my account, but I’m open to moving them to an organization account if that works better. However, scattering them across different places wouldn’t be ideal, based on my goal.

To see if we share the same goals, I suggest we continue collaborating via PRs for a while. I think blindly merging modifications without confirming they align with our overall objectives could lead to failure. If things go well, I’d be happy to invite you to become a collaborator.

These are my thoughts from my perspective. You might have a different vision for your project, and that’s perfectly fine.

Looking forward to your thoughts.

[shg]

By the way, the reason why I haven't published python-vterm.el to MELPA yet is because I think there are still some necessary design decisions to make, including how this should interact with some popular python tools such as, IPython, venv, pyenv, poetry, rye, conda, etc. It would be appreciated if we could discuss these.

[vale981]

Hey, I think i was just confused :).

My goal is to have a /simple/ and robust interaction with a python repl that does not require any specific stack underneath. My worklow is consciously centered around python scripts with functions instead of org babel code cells. So I believe in that regard our goals differ indeed.

With respect to supporting specific tools: My goal is to keep the functionality compatible with most python interpreters. Currently, some functionality uses IPython magics which i'm going to replace with pure python. As to venv, poetry etc: the only concession I make is that the repl is started with the environment variables of the buffer in place. This works great for my specific use case (nixos & direnv) but may not work for others. The problem here is that the ecosystem is too diverse to fully integrate. Therefore I think one should rely on tools like envrc or poetry.el to handle those.

[shg]

Hi @vale981

Regarding the differences in our goals, I believe collaboration is possible if we can both be considerate of each other’s objectives.

My goal includes making python-vterm itself useful for various common use cases. This shouldn’t conflict with your goal. Would your goal be to optimize it only for the specific workflow? If you can take into account consistency with other workflows (including supporting Babel through ob-python-vterm), which I assume you can, then I am still positive about working with you. Are you interested in this approach?

[vale981]

Yes, of course ;).

I think I'll keep hacking on it for next couple of weeks and then we can try and consolidate.

Using the python-vterm stuff for work has been really awesome (and also a yak shaving time sink).

I think I could even get around to using Babel again eventually. Maybe my experience with emacs-jupyter has burnt me out a little in that regard.

Cheers, Valentin