Migrating Package to ST4

[tajmone]

I strongly advocate for making this an explicit Sublime Text 4 package.

Although right now the syntax doesn't use any of the new ST4 features, the completions are already exploiting new features like completion metadata, which are not supported by ST3 (although they don't break backward compatibility).

My point is that as ST4 features will evolve (which they do, and rather fast), it will become increasingly harder to ensure that the package will still work as expected for, even for ST3 — in theory, new features are simply ignored; in practice it might not be so. Furthermore, why limit the package potential only to ensure backward compatibility with an old and unmaintained version of the editor?

In my local fork, I've already enabled the new ST4 syntax engine by adding version: 2 to the syntax. All tests work fine, which means that currently the syntax doesn't contain any contexts leading to behavioural differences.

But the question of ST3 vs ST4 also affects the API Environments, i.e. which Python version to use within the package — it's either Python 3.3.6 or 3.8, you can't use both. This is probably the crucial point in deciding whether to stick with backward compatibility for ST3 or move along with ST4.

Since this package is currently registered on Package Control as a ST3 package, the decision on whether to switch to ST4 should rather be handled sooner than later, since the switch would come at the detriment of ST3 users who would end up with a package they can no longer update.

In any case, switching from ST3 to ST4 requires some further repository and package interventions:

1. The CI build job needs to use the new ST4 syntax test suite.
2. The package version requirements need to be updated on the package control channel.

Amending Package Requirements

Currently this package is registered as working with ST >3092 in the p.json file:

        {
            "name": "PureBasic",
            "details": "https://github.com/peterthomashorn/purebasic-language-for-sublime-text",
            "labels": ["language syntax"],
            "releases": [
                {
                    "sublime_text": ">3092",
                    "tags": true
                }
            ]
        },

so I guess that would have to be amended to >4109, to prevent ST3 users from installing it or further updating it.

NOTE — This pacakge registration system seems rather tedious and inefficient, especially since PRs take weeks to be reviewed. In theory, the version info should be a precise value in order to prevent updating a package beyond the supported version by the user, in practice once a package is submitted that info never gets updated, regardless of the new features used by the package later on.

CI-Testing ST4 Syntaxes

As for the the ST4 syntax tests API, I only had a brief peek at it and, IRC, it's a Python package and no longer a binary executable, as it used to be with ST3. So its setup might require some documentation reading. But I'm not entirely sure about this, and the official documentation isn't much helpful either (hard to find any reference).

I remember having stumbled upon a package that used a Python library for validation, but can't recall any links right now.

But also I see that the official ST syntaxes repository uses a downloadable binary file to validate ST4 syntaxes:

https://github.com/sublimehq/Packages/blob/master/.github/workflows/ci.yml#L50

Probably, right now the current Workflow is just fine, since the syntax is not using any of the new ST4 features or bug fixes. So it might be OK to postpone the migration to the new validator tool, for the time being.

[p4t5h3]

https://github.com/sublimehq/Packages/blob/master/.github/workflows/ci.yml#L50

I do not remember why I copied the current test action configuration from. I was digging in the official Sublime Text packages, too. Let's see. Maybe I can fix the remaining problem of #39 with it.

[p4t5h3]

Ah, yes, from here. It is the one with the binary.

Besides: I agree the minimum requirement of Sublime Text 4, if the support of the previous major version implies unreasonable effort. Due to the limited resources the guideline should always be the most recent stable releases of Sublime Text and PureBasic.

[p4t5h3]

Okay, that did not work out successfully. I am still new to GitHub workflows- I am familiar only with Jenkins and GitLab CI until now. I will try to fix this in a feature branch before it escalates in an awkward series of pushes to main.

[p4t5h3]

Noice. 😎

[p4t5h3]

In my local fork, I've already enabled the new ST4 syntax engine by adding version: 2 to the syntax.

Where is that from? I do not know anything about that. I thought the YAML syntax definitions are "the new ones" already.

[tajmone]

Where is that from? I do not know anything about that.

For full details, see:

• https://www.sublimetext.com/docs/syntax.html#compatibility

and for the new- vs old-Python API:

• https://www.sublimetext.com/docs/api_environments.html

I thought the YAML syntax definitions are "the new ones" already.

The YAML .sublime-syntax was introduced in ST3, replacing the old JSON/TextMate .tmLanguage of ST2 (maybe also ST1?).

ST4 extends the .sublime-syntax adding new keys and features, and the editor supports both the old ST3 engine (for backward compatibility, bugs and all) as well as the new ST4 engine (faster, more features) via the version: key (which is ignored by ST3).

Among the new powerful features that are enabled using version: 2:

• Context branching and stack-rewinding (branch, fail)
• Syntaxes inheritance (extends, meta_prepend, meta_append).
• Multi-level stack pop (e.g. pop: 2, etc.).
• Improved parent scope handling for push and set operation.
• Various bug-fixes that were preserved in the ST3 syntax engine for packages compatibility.

My Fork

You can always peek at my fork, where I have a dedicated myfork branch where I test new features locally. You can see that I've enabled version: 2 in my local fork:

• https://github.com/tajmone/purebasic-language-for-sublime-text/blob/myfork/PureBasic.sublime-syntax#L8

My PRs here are all cherry-picked from commits which I've tested on the myfork branch, and once you merge them and I rebase myfork on main they disappear from the myfork branch history (since they become redundant). So it's pretty straightforward for me to pile up new features in my local fork, test them, create individual PRs for each feature and yet still be able to keep history divergence between the myfork branch and main down to a minimum in the course of the time.

[p4t5h3]

As far as I understand the Python API is irrelevant at this moment because the package does not make any use of it (yet).

[p4t5h3]

I created a pull request to raise the minimum required release to the first stable release of Sublime Text 4.

[tajmone]

I created a pull request to raise the minimum required release to the first stable release of Sublime Text 4.

Great! BTW, you can always add my forked repo as a remote to your local repository (just as I'm doing with yours), so you can checkout my myfork branch locally and test new features I'm experimenting with, and if you like any of them just cherry pick their commit into your main branch, instead of having to go through a pull request — after all, Git is a distributed version control system!

E.g. you could cherry pick my commit were I enable the version: 2 key. I tend to stick to one feature per commit, and write decent commit messages, so they should be ready for integration.

I often use this approach with projects that have multiple forks, so I can pull into my fork features added by third parties.

[p4t5h3]

The pull request got merged. So the version: 2 in the syntax definition is the only thing left, if I skimmed this issue correctly.