search

renpy / renpy

The Ren'Py Visual Novel Engine
http://www.renpy.org/
4.93k stars 693 forks source link

[Request] More builtin packages #3302

Closed Booplicate closed 2 years ago

Booplicate commented 2 years ago

While not every package is being used by every developer, it's very handy to have them when you need them. Some packages you can add by yourself, but usually that's very hard and requires a lot of trial and error to add even simplest packages. You add one, it starts missing another. And then there's different libs for different platforms. There's 2 reasons why a developer may need it. First, you may want to use it yourself. Second, a 3rd-party package may require some of the builtins. With the latest Ren'Py we got a lot of new packages to work with, that's really nice and almost 1:1 to real Python programming, but sometimes almost isn't enough. As I understand, Ren'Py doesn't ship all default packages to save some space? I don't think that's a big issue. Although, maybe it's worse for mobile platforms? I don't know.

I want to propose to include all builtin Python packages with Ren'Py.

renpytom commented 2 years ago

Is there something you're thinking about that's missing? IIRC, the stuff that's missing is either because it has a dependency I don't want to ship wiht Ren'Py, or because it really makes no sense as part of a visual novel.

Andykl commented 2 years ago

I think it would be good to have access to your local installed packages during development. Right now I have to wrap around subproccess or other workaraounds. One of the packages I think may be good to ship with is PIL especially if there will be some way to transform its Image structure into pygame.Surface.

renpytom commented 2 years ago

I don't think that's possible. Ren'Py doesn't use your local Python interpreter.

Booplicate commented 2 years ago

One of our side packages requires sqlite3, for example. The package uses it to read some cities data from its db. We need it because it's one of our main features we can't drop now.

That one is probably the most important for us, but there are a few more. It's understandable if most VN won't need it, right now even we only need it because it's a dependency (although, there's a possibility we could use it in our code, too). But it'd be really cool if there was an option to select which builtin packages your VN will use, or if we could just build Ren'Py with the packages we need.

Long time ago we also had to port requests and logging for our build, plus some ssl/htlp libs, right now they are being shipped with renpy, which is very handy!

Gouvernathor commented 2 years ago

I think we should document a canonical easy way to include python builtin packages in a game, like including third-party ones currently is. Given that (pure) python misbehaves with rollback among other things, I don't think we should ship-in more packages than strictly necessary. It should be a voluntary thing done by devs knowing what they're doing.

renpytom commented 2 years ago

There are a couple of things going on here.

The first is that as of Ren'Py 7.4. we've moved to a model where all of the non-Python modules are linked together into a single binary, rather than having .pyd/.so/.dylib files all over the place. That had a bunch of advantages to it - the big one being that "antivirus" doesn't seem to be making nearly as many false positives with Ren'Py, and it also allows the one-step startup on Windows. But it does mean that swapping binary modules in an out isn't possible.

I don't think it makes sense to include sqlite with Ren'Py, either. Fundamentally, I don't think that a visual novel engine needs to ship with an SQL database engine, and I tend to think supporting a SQL database engine across multiple platforms would be hard. (I'm especially thinking of Android, where the game isn't unpacked into files on disk.)

I'd suggest focusing your efforts on your dependency, and figuring out how to extract the data from it into a format that doesn't need a database engine to access.

Beyond that, you can take a look at renpy-build. It should be possible to add a task (define it in tasks/sqlite3.py, add it in tasks/__init__.py) to build sqlite, and then edit the Setup files in source/ to include it as a Python module. Building Ren'Py requires a lot of resources (Probably about 50GB), but renpy-build does automate the process.

I don't think that as a project, we want to go down the rabbit hole of adding tons of binary dependencies in the hope of getting third-party Python modules to work. I don't think it's likely such modules will fit well into the Ren'Py environment if they're not designed for it, so we're just pushing the problem out a single step. Modules that do work with Ren'Py should limit themselves to the set of supported modules.

Booplicate commented 2 years ago

Sadly we haven't had any luck, possible because we don't have ubuntu (it's specified in the requirements). We might try again later. We also discovered that we could use the unittest module in our project, too. So not giving up yet.

I'll close this as the former question has been answered. That said, would be cool to get the building process a bit more documented.

renpytom commented 2 years ago

You should be able to get Ubuntu running in a VM, and build from there. (The specific version of Ubuntu is also important.) That's how I test the build process.