Upcoming gpython worklist

[drew-512]

Below is my upcoming worklist I plan to take on in the coming weeks/months:

1. Add stack limit value to py.ContextOpts and implement (Issue 143)
2. Additions to py.utils, including funcs that use reflect to auto-populate Go structs from a given py.Object
3. Add exit() (Issue 140)
4. Add a shared set of golden util funcs to pytest/pytest.go so the same boilerplate doesn't have to appear for each golden main_test.go etc
5. Enhance examples/multi-context into a golden test.
6. Add real, imag, conjugate properties to py.Int, py.Float, py.BigInt, py.Bool plus tests (Issue 73)
7. For cleanliness and organization, I suggest we move all the built-in modules into the modules dir. In a future where gpython gets traction as an embedded interpreter and more and more modules appear, it makes sense to me that they show up the modules dir, not the project root dir. This is basically just a bunch of dirs moving and a few import lines changing. I would be happy to do this first or as soon as makes sense (since something like this is better done sooner than later). I think this is also very helpful (self-documenting) for newcomers to understand the project structure and organization.
8. Introduce py.ModuleImpl for os, offering support for a few commonly used functions -- e.g. os.system()
   • This is a security concern (e.g. python web demo). Thinking I add flags to py.ContextOpts that correspond to new py.Method flags which are checked when a py.Module is populated during an initial py.Context module import (instantiation).
   • I would add tests that test each security mode / flag
9. Add support for .format using pyfmt (Issue 116)
   • Before doing this, I would appreciate sign-off on pyfmt so I don't start investing time into that and then we don't want to add that as a dependency. We can cross this bridge as it comes, but I wanted to express the value here and that pyfmt is a natural fit with gpython!
   • I left a note to the dev

Also please lmk what order I should do these in if that's something that matters to people. Otherwise, I listed them in the order that makes the most sense to me.

@sbinet @ncw, please lmk how the PRs should show up. By default, I'll do a separate PR for each numbered item (citing the Issue # in the PR)

Drew

[drew-512]

Also with (6), I'd like to invite discussion and ideas around what that suggested repo layout for outside ("3rd party") gpython modules. For example, a side project I'm commissioned on is an exciting particle physics project that I've packaged into a gpython module (so it can be more easily consumed by non-developer types). I am interested in using its public release (date TBD) to also model suggestion for others on how to publish their gpython module.

So, perhaps the template looks like:

github.com/{{publisher}}/gpython-{{module-family-name}}/
    go.mod (1.15 or later)
    go.sum
    README.md
    module-foo/
        README.md
        module.gpython.go
        ...
    module-bar/
        README.md
        module.gpython.go
        ...
    ...

I invite any feedback or suggestions from you github and Go and Python warriors out there. 🗡️

[sbinet]

the work plan SGTM.

as for the pyfmt work, I am a bit worried that there might be some impedance mismatch: pyfmt seems to have been written to bring the .format facilities to Go, which may not be completely the same than implementing the .format facilities in Go for Python. also, the main dev seem to have moved one and we may not get fixes upstream. the licenses being compatible (BSD-3), it may well be just as efficient to adopt the initial files in our tree (with proper mentions of the original work), or start "from scratch".

as for the modules reorganization, why not. (in Go spirit, the directory holding them would be named module instead of modules, but that just me nitpicking).

seamlessly integrating 3rd-party modules into a "stock" gpython executable might prove a bit complicated, considering the state of the stdlib plugin package (and its lack of cross-platform support). this probably means people will need to compile their own gpython executable, with the 3rd-party modules they want (well, except if it's a pure python one, and not an extension one written in {C,G}Python)

[drew-512]

as for the pyfmt work, I am a bit worried that there might be some impedance mismatch: pyfmt seems to have been written to bring the .format facilities to Go, which may not be completely the same than implementing the .format facilities in Go for Python. also, the main dev seem to have moved one and we may not get fixes upstream. the licenses being compatible (BSD-3), it may well be just as efficient to adopt the initial files in our tree (with proper mentions of the original work), or start "from scratch".

Great points! That evaluation probably saves me a lot of time and process. I will do a deeper evaluation when I get to that point in the worklist. My basic goal was to provide support for the most basic forms of .format(), where a 20% implementation gets us 80% coverage sort of thing -- so newcomers trying out gpython aren't immediately met with an error when they excitedly try "{} {}".format("hello", "world") or whatever.

as for the modules reorganization, why not. (in Go spirit, the directory holding them would be named module instead of modules, but that just me nitpicking).

Ok, great. Lmk if you have a preference on me doing that sooner or later, otherwise, I'll stick with (7) so we can build momentum towards that (and will be a nice reward). I def think the repo will look and feel tighter and cleaner with that simple reorg.

seamlessly integrating 3rd-party modules into a "stock" gpython executable might prove a bit complicated, considering the state of the stdlib plugin package (and its lack of cross-platform support). this probably means people will need to compile their own gpython executable, with the 3rd-party modules they want (well, except if it's a pure python one, and not an extension one written in {C,G}Python)

Ha, I wasn't even thinking about dylib support, and more just focused on people in the mindset to offer their Go package (e.g. a particle physics computation framework) in the form of a cli build target that allows consumers to easily use the framework through Python scripts (as you described). This is really how I see gpython is offering a lot of value to Go projects. So, in sum, my vision is that gpython makes it easy to do this and the examples model and reflect this (e.g. examples/embedding).

[drew-512]

Hi @sbinet and all, I hope you've been having a prosperous summer and may blessings be with everyone.

In the coming several weeks, I'll be knocking out golden tests for the multi-context example along with some of the remaining items above.

With that done, I think we'll be at a great point to:

• update the project tagline with something a little more inviting. I'd love to hear what contributors think as gpython has become quote formidable as an embedding tool. We are seeing exciting success with it
• tag main with a release tag and official version -- v0.2.0 ??