Sub-Packages - proper local dependencies #138

Open Southclaws opened 6 years ago

This came from a discussion of #55 as well as the structure of Scavenge and Survive.

Scavenge and Survive is a very open gamemode, upon moving to the sampctl workflow it changed from a single repo to a set of repos inside an organisation. This meant each component is now an isolated package and can be sampctl package run in order to test its features without other code from the gamemode interfering.

But this open-ness isn't how gamemodes are normally written. If a closed-source server wanted to use this development workflow, they would need to create a private git repo for each package - on GitLab, Bitbucket or another private git service.

What I want to do to solve this issue - that also creates a slightly nicer architecture that doesn't require hundreds of git repositories - is to introduce the concept of "Sub-Packages".

What is a Sub-Package?

A Sub-Package is similar to a submodule in git. It's a package within another package. Also similar to how you can nest package.json files in nodejs.

How this would work is:

sampctl will recursively walk the directory tree starting from the package root - not including dependencies/.
it will note down packages (folders containing package definition files)
these packages will be added to the -i list upon build

This means sub-packages can be included just like regular packages - with #include <entry>.

It also means all subpackages may share the same dependencies/ - to prevent repository bloat and, given that subpackages are meant to be compiled into one single gamemode, ensure they are all compatible with the same dependencies (you wouldn't want your furniture system dependent on streamer 2.9.3 when the rest of your gamemode depends on streamer 2.8.2).

What are the Benefits?

What has been listed so far isn't anything really new. It's possible already, you just need to use explicit paths with #include "path/to/inc.inc" - most gamemodes are already structured like this.

sampctl being aware of these subpackages allows them to be built and loaded individually, just like the Scavenge and Survive packages. The difference here is that sampctl can build these as filterscripts and automatically swap them out at runtime - a-la y_master! This means you can work on a package while "hot-reloading" it (just like modern frontend web development workflows) resulting in a much faster development workflow.

The only manual process here would be un-including the package from the gamemode, unless this can be solved with some simply regular expression searches (this is another thing that would be trivial to resolve when Southclaws/pawn-parser is in a working state).

Proposed Workflow

Here's a quick run-down of what a possible workflow would look like. Assuming the user is working on a roleplay gamemode and are implementing a furniture system.

Create a new sub-package sampctl package sub init enter path, generate files, etc.
Fire-up the server, without modifying any of the source or including the new (currently empty) subpackage
Somehow tell sampctl that you're working on the furniture subpackage and sampctl will --watch the subpackage directory for changes - not sure what the UX for this will be yet, it may involve two instances of sampctl talking to each other via RPC or a single instance fired up with some flags/args.
While you're editing the furniture subpackage, sampctl rebuilds and passes reloadfs ../gamemodes/subpackages/furniture/furniture to the server
You finish implementing the furniture package using y_master to communicate with the gamemode - you never have to restart the gamemode or modify the gamemode's source
You add the necessary #include <furniture> line to your gamemode's script and then sampctl package build and sampctl package run - the furniture package is now running as part of the gamemode.

Feedback

This is quite a substantial addition to the toolset, so I'd love some feedback from library developers - particularly @Y-Less being the developer of y_master (I've never actually used it myself!) and people who are currently developing complex gamemodes, off the top of my head: @PatrickGTR @Rehasher

Maybe not totally related, but worth mentioning. Before sampctl, I wrote a set of scripts to set up a hierarchy of git repos, with associated server-side permssions, such that the source code for each module would be in a separate repo, and only people with access to that module's source could check out that repo. Everyone else would just be provided an auto-server-side compiled filterscript of that module (with dev limitations like low MAX_PLAYERS built in). Thus devs could test the whole mode, including parts of the code they weren't working on, without anyone ever having the whole code. You could limit building the whole thing and deploying to a server to just an automated build system so literally no-one could steal the whole source code if you so desired.

That's pretty cool! I'm wondering if a similar workflow could come of this. I recently set up a submodule system for BA:RP so people can edit maps and Rehasher can pull the submodule into the main production server. Similar concept, just not with code.

If the sub-packages themselves are git repos (submodules) this could also work - but it would need a bit of thought for the workflow of the "slave" side, how do they get the .amx, could that be automated? Could sampctl act as some kind of build server for teams too (#38)?

Another question for all of this is, how worth it is it? I'm not sure how many servers actually have large development teams. And even then, the existing ones might not be too happy to restructure their code. I want to keep this in mind while working on this as time is scarce!

Southclaws / sampctl