Open docwhat opened 3 years ago
The python code would be run periodically by a CI job. Though I guess we could include it as python3 is pretty common now... and fall back to the pre-generated version.
The DB is about 2.2Mb but compresses really well to about 183kb; we can just use zcat
on it.
It would be an awesome contribution.
@syndbg Would it be okay to have a run-time dependency on python3? I can use any language or tool you want; it just needs to be able to translate JSON to text.
I could even download something like hairyhenderson/gomplate or docwhat/temple instead of depending on python3.
Or I could even build a tiny GoLang program to do the translation from JSON to the DB file format.
Re: "Would it be okay to have a run-time dependency on python3?" This 'should just work' on many modern OSes, but I would prefer to avoid it since it adds extra complexity.
this would be an excellent feature - our CI uses goenv, but we have to rebuild the whole image just to refresh the list of supported golang version! Seems silly
consider: instead of throwing a definition not found
error, first see if it exists remotely?
Hey, sorry for the delayed reply, on a vacation 'till the end of the year.
So, answering the questions.
Would it be okay to have a run-time dependency on python3? I can use any language or tool you want; it just needs to be able to translate JSON to text.
I'd prefer not to depend on anything Python. If you're looking for a specific language to do it, I'd prefer Golang.
Just to be clear, this is only a feature for the CI.
My opinion on what goenv
should solve is still the same - reliable, secure (as much as checksums are secure) and predictable golang version manager and installer.
I still prefer for this "version golang releases/versions discovery feature" to run in CI, create PR (if needed) and still require manual approval.
this would be an excellent feature - our CI uses goenv, but we have to rebuild the whole image just to refresh the list of supported golang version! Seems silly consider: instead of throwing a definition not found error, first see if it exists remotely?
Being able to install versions fetched dynamically from remote might be amazing as a user experience, but I prefer it the other way - a specific version of Goenv can install only N versions of Golang from well-determined sources and expected checksums.
The reason why I prefer this is rather "lame". It's easier from a security perspective (InfoSec) to review and sign-off that this software is "secure". Enterprise/Grown-up friendly... in a way.
The reason why I prefer this is rather "lame". It's easier from a security perspective (InfoSec) to review and sign-off that this software is "secure". Enterprise/Grown-up friendly... in a way.
Unless someone is reviewing each release, the only security benefit is that the window of attack is smaller (golang.org would have be compromised during a CI build vs. any time someone installs Go) and that a bad release can be removed (tho not revoked since goenv lacks a revoke system).
💡idea: Since we have to trust golang.org anyway, how about we pin the SSL certificate in the curl request? Whether we do it only in CI or in run-time.
Is there something we could do to increase the security of gathering the available go versions at run-time?
See #261..waiting on being able to add a PAT to the repo 😄
@ankitcharolia we use the go dev API for automation of daily version checks. I don't mind that you have your own version of goenv that depends on go. Our implementation doesn't depend on any external dependency and uses native shell to achieve the same outcomes.
Now that https://golang.org/dl?mode=json&include=all works, we can use this to fetch a complete list of go archives and installers.
`golang-versions.py`
```py #!/usr/bin/env python3 """ Use this to pipe the JSON from 'https://golang.org/dl/?mode=json&include=all' to generate shell compatable data for 'go-build'. Example: $ { echo 'declare -a versions=()'; curl "https://golang.org/dl/?mode=json&include=all" | python3 this-script.py | grep '#arch:amd64#.*#kind:archive#.*#os:darwin#.*#stable#' | grep '^versions' | uniq; echo 'for v in "${versions[@]}"; do echo $v; done' } | /bin/bash """ import json import sys from shlex import quote def emit_one_file(out, file, tags): for k, v in file.items(): if k in ("filename", "size", "sha256", "version"): continue tags.append("{}:{}".format(k, v)) tags.sort() for k in ("filename", "size", "sha256", "version"): out.write("{:80} ".format("{}={}".format(k, quote(str(file[k]))))) out.write(" ".join(["#{}#".format(tag) for tag in tags])) out.write("\n") out.write("{:80} ".format("versions+=({})".format(quote(str(file["version"]))))) out.write(" ".join(["#{}#".format(tag) for tag in tags])) out.write("\n") def emit_one_version(out, one_version): tags = [ "version:{}".format(one_version["version"]), "stable" if one_version["stable"] else "unstable", ] for file in one_version["files"]: emit_one_file(out, file, [t for t in tags]) out.write("\n") def emit_lookup_data(writer, versions): for one_version in versions: emit_one_version(writer, one_version) if __name__ == "__main__": data = json.load(sys.stdin) emit_lookup_data(sys.stdout, data) ```The generated file output can be used to find versions with
grep
.And you can find available versions by using the
versions
variable: