Warn when old k6 version is used

[na--]

After https://github.com/grafana/xk6/pull/44, xk6 will not always try to use the latest k6 version to build the binary, it will instead use the one specified in the extensions' go.mod file(s). However, as this forum thread has shown, this can cause problems for unmaintained extensions, when users expect to be able to access newer k6 built-in APIs.

So, it would be nice if xk6 emits a warning when we are going to use an older version of k6. It seems like we can do it by running go list -m -json -u -mod=readonly go.k6.io/k6 for the repo, it will produce something like this:

{
    "Path": "go.k6.io/k6",
    "Version": "v0.33.1-0.20210825161650-c932a28ff940",
    "Time": "2021-08-25T16:16:50Z",
    "Update": {
        "Path": "go.k6.io/k6",
        "Version": "v0.40.0",
        "Time": "2022-09-08T07:54:34Z"
    },
    "Dir": "/go/pkg/mod/go.k6.io/k6@v0.33.1-0.20210825161650-c932a28ff940",
    "GoMod": "/go/pkg/mod/cache/download/go.k6.io/k6/@v/v0.33.1-0.20210825161650-c932a28ff940.mod",
    "GoVersion": "1.16"
}

I guess we can compare the Time or semver-compare the Version attributed between the main object and Update :thinking: Or we can figure out some other way to detect newer k6 versions.

[mstoykov]

I wonder if we should just always emit the warning unless a version is provided on the command line :thinking: (irregardless of it is latest or v0.32.0).

This way the user will know they are building with a particular version (unless they use latest ;)).

We won't need to make any checks as well :shrug:

On the other hand I guess if they just hard code xk6 build v0.31.0 --with ... in some script they will never get a warning. But should they get a warning if they have provided a specific version either way?

[na--]

:thinking: good points... Not sure either way, but I am leaning slightly towards always warning when the version is old, even if it was explicitly specified :thinking: Just because it's in our interest to minimize the number of users with old k6 versions as much as we can... :sweat_smile:

[szkiba]

I would like to bring up an earlier idea of ​​mine here: k6 extension linter and rating/badge. The extensions included in the registry would be checked and qualified by the extension linter periodically (in the case of a new version of the given extension and in the case of a new k6 release). This information could be used in the case of a k6 build, even for warning purposes. In the specific case, if I understand correctly, the primary problem is not that xk6 will build old k6, but that the extension is not maintained.

[pablochacin]

if I understand correctly, the primary problem is not that xk6 will build old k6, but that the extension is not maintained.

I think this particular issue is regarding how to communicate this to the user, not about the registry.

That said, I think adding this information to the registry will be useful so that the user decide how to handle the situation.

I'm not so in favor of making the build process dependent on this information, not at least until we define more clearly the policies about extensions (official, supported, etc.)