search

for-GET / jesse

jesse (JSon Schema Erlang) is an implementation of a JSON Schema validator for Erlang.
https://github.com/for-get/jesse
Apache License 2.0
124 stars 64 forks source link

Prevent a warning issued by OTP 24 #103

Closed paulo-ferraz-oliveira closed 3 years ago

paulo-ferraz-oliveira commented 3 years ago

Sure, @seriyps, thanks. No hurry, though. OTP 24 is not yet around the corner ๐Ÿ˜„ and I may find something else in the meantime.

paulo-ferraz-oliveira commented 3 years ago

@seriyps, I'm pinging you ๐Ÿ˜„... a release would be appreciated after that, unless you want to wait for OTP 24, first.

seriyps commented 3 years ago

Thanks @paulo-ferraz-oliveira ! Do you know if it's already possible to add OTP-24-RC3 to the GitHub CI pipelines?

andreineculau commented 3 years ago

Thank YOU, Sergey for all your contributions stepping in!

About GitHub CI - there's no language support as it's custom to many other CIs. There's a setup-erlang action, which we actually don't use. We use the erlang docker images https://hub.docker.com/_/erlang?tab=tags&page=1&ordering=last_updated and there are 24-rc3 images published.

-- http://andreineculau.com

On 3 May 2021, at 21:05, Sergey Prokhorov @.***> wrote:

๏ปฟ Thanks @paulo-ferraz-oliveira ! Do you know if it's already possible to add OTP-24-RC3 to the GitHub CI pipelines?

โ€” You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

seriyps commented 3 years ago

Thanks @andreineculau. I'll open a PR to add 24-rc3 for now. And we can update it to the release version when it's ready.

FYI: jesse is still in active use in Klarna/KRED. But we upgraded it from super-outdated https://github.com/klarna/jesse to for-GET/jesse just less than a year ago =)

andreineculau commented 3 years ago

just less than a year ago =)

hashtag agile >_<

paulo-ferraz-oliveira commented 3 years ago

Thanks @paulo-ferraz-oliveira ! Do you know if it's already possible to add OTP-24-RC3 to the GitHub CI pipelines?

If you'd be using action setup-beam it would be possible to target 24 and this would always mean the latest as per semantic versioning rules, but there might be limitations with it (IIRC jesse's CI was not straightforward).

andreineculau commented 3 years ago

uff, I think it's the second time I'm ignoring setup-beam, Paulo! And I can even see you're a contributor. Apologies for my ignorance, it's not intended.

NOTE the following is totally me digressing, and is in no way a critique to anyone

just FYI as to where I (thus nothing in absolute terms, and given my limited attention span for jesse/katt, I'm open to ideas) stand on github CI "actions" i.e. the plugins that you can call from within a github CI workflow. I'm sharing these thoughts because maybe it will make you and anyone else reading this more... aware, if not critical of them:

Calling an "action" means implicitly calling a javascript piece of code in another github repo, optionally tagged. On the surface that's maybe ok, but imagine this is your CI, maybe even your CD. I'm a core JavaScripter, it's my first go-to language for various reasons and I like the core of the language, but the extras and the ecosystem are beyond repair. As if it wasn't enough to deal with those for frontend and maybe even backend development, now you deal with those for CI/CD. I want none of that.

Someone wants to see what setup-beam actually does?! Good luck looking at a glued javascript file, 5000 LoC long, with boilerplate https://github.com/erlef/setup-beam/blob/main/dist/index.js Like I'm not even going to ask why webpack is needed. I'm sure that there's a good reason for it, I really do, but it's just pure madness to find it there given the expectation is more on the level of "given a semver range, find the latest installable version, run some shell commands to install". Many if not most actions need to call other executables, so they need all sorts of boilerplate for safely doing shell calls (manage stdin, stderr, stdout, quote args, etc) which setup-beam needs as well. All of this in the name of cross-platform support and de-duplication (i.e. no need to write the same logic, once for unix, once for windows).

Some might say that setup-beam is "doing it wrong (tm)" and that you have actions that are simpler to understand, like setup-erlang https://github.com/gleam-lang/setup-erlang/blob/main/main.js which ends up calling a bash or a powershell script, depending on the platform. Very readable. Great. Just to be fair all the way - it only handles OTP, and only a strict version, no semver range. Alright, why do I need to call an external script which handles only debian distros with aptitude and wget installed? But it has support for windows, for which you might note care at all. For starters, given the simplicity of the installation, I can just as easily handle the installation myself, and whenever I need to target another platform, I can just as easily branch my code, and handle that extra platform. That code is part of the repo and thus becomes forever readable, maintainable and alive. More importantly, if done right - it can be used locally to bootstrap a developer machine, a docker image, etc. There's no need to understand/debug/maintain 5000 LoC of javascript in 1-2-3-4-5-... years when something is going to stop working and there's no maintainer in sight, and I won't be able to install node and npm and webpack to build from source, etc etc. And even if I make a fix, then it's on me to release a new github action etc etc.

I recently had to deal with Windows, WSL and github CI. While investigating this and looking for some performance improvements to a one-liner, someone else was writing a github action to setup WSL. In 6000+ lines of Kotlin. Then this happened https://github.com/rokmoln/support-firecloud/commit/ddfc65a19d20fa87954cee487f0b3137d93e6882#r42031652

There's a great joy in keeping things simple, and that's why we landed with using the official erlang docker images in jesse/katt under github CI, even though Paulo and I had to find the right magic to upgrade git on their images https://github.com/for-GET/jesse/pull/101/files :) Wasn't straightforward but it is now, and will probably always be. While it might not to be very obvious, the current setup is "simpler, safer and more fun" (such a great motto as opposed to "smoooth" I think :) ) as opposed to any github action, no matter how perfect its code might/can be.

As far as I'm concerned, the interface should have been a checked in file .gitmodules.github-action with support for semver ranges (as opposed to hashes for git modules), a git-github-action script to locally fetch those actions' git repos and call them cross-platform. Something like this:

#.gitmodules.github-action
[submodule ".github/actions/erlef/setup-beam"]
    path = .github/actions/erlef/setup-beam
    url = git://github.com/erlef/setup-beam.git
    branch = v1.x.x
# call setup-beam scripts normally, via the shell, in a github ci workflow, on your local machine, etc
git github-action call erlef/setup-beam --otp-version=22.2 --elixir-version=1.9.4
# it will magically call a main.sh/main/main.ps1 script, depending on the platform and/or a manifest file
# random ideas
git github-action install # fetches all git "submodules" under .github/actions
git github-action install erlef/setup/beam@2 # updates .gitmodules.github-action to branch=v2.x.x

Apologies for the wall of text, if you've actually read it :)

A little copying is better than a little dependency. https://go-proverbs.github.io/

paulo-ferraz-oliveira commented 3 years ago

@andreineculau, I read your text to the end. I was merely proposing using setup-beam (hadn't noticed you'd ignored it before; don't feel offended for you doing it, or for not using it, or for criticising it - it is what it is, built to solve a specific issue), not forcing it or anything like that. I understand now, better, you don't want it, so will not propose it ๐Ÿ˜„. (if I do it's because I forgot I wrote this, which might also happen, in the future).

I'll try to complement some of the information you wrote (there's no questions, so I won't actually be replying to anything - think of this more like a commentary), not trying to dig deeper on what existential questions might exist around code bundles and dependencies of dependencies. I also work with Erlang everyday, and don't feel that, for that, I have to have complete knowledge of the whole underlying system (Erlang) or what it sits on top of (e.g. a container) or what that sits on top of (OS), so I could say "Yeah, but running a 4-line app is 6 million lines of code", but I will steer way from that argument.

make you and anyone else reading this more... aware, if not critical of them:

Oh, I am enough critical, but they do help me solve issues, so...

I like the core of the language

Me too; have done JavaScript for quite a few years and there's some good stuff in it (thought most just criticise for the appearance, it seems, or whatever surrounds it)

... now you deal with those for CI/CD. I want none of that.

That's acceptable. I reiterate. I'm not forcing you or the project. I understand you don't want it, and I'll not propose it further.

Someone wants to see what setup-beam actually does?! Good luck looking at a glued javascript file, 5000 LoC long, with boilerplate https://github.com/erlef/setup-beam/blob/main/dist/index.js

Yeah, you're right, it is index.js that runs, but that's not the file that's developed; the developed part is around 50 lines of shell script and 300 lines of JavaScript. If you put it like that (sure, you're selling a point) we could also argue that you need to understand a generated grammar parser and lexer and not the input. (what I'm going to say might be heresy to you, but there's some sort of "trust" here, right?).

Like I'm not even going to ask why webpack is needed.

Nor am I; the same why I don't ask why most of my OS has code I don't use; or even the Erlang VM, FWIW.

All of this in the name of cross-platform support and de-duplication (i.e. no need to write the same logic, once for unix, once for windows).

Yes, two main problems it solves is cross platform and semver ranging, as well as making you (and you might be a developer with very little free time) depend less on code you wrote yourself - thus dependency, not copy.

Some might say that setup-beam is "doing it wrong (tm)" and that you have actions that are simpler to understand

Oh, some will, for sure, and those should not use it, but it doesn't mean we'll stop developing it because there's criticism ๐Ÿ˜„

, like setup-erlang https://github.com/gleam-lang/setup-erlang/blob/main/main.js which ends up calling a bash or a powershell script, depending on the platform. Very readable. Great.

Yes, it is readable, but not everything in life has to be void of complexityโ„ข๏ธ, and in setup-beam's case, most of it goes to:

  1. installing Elixir after Erlang from a well known source,
  2. installing mix after Elixir,
  3. installing rebar3 after Erlang from a well known source,
  4. finding a version based on a semver range for Elixir, Erlang and rebar3,
  5. not having you worry about pairing an OS version with an Erlang version

But it has support for windows, for which you might note care at all.

We do care, which is why we have an action item for it and it's being discussed in the Erlang Ecosystem Foundation's Build and Packaging Working Group.

For starters, given the simplicity of the installation, I can just as easily handle the installation myself,

Yes, I can too, but why should I repeat myself? I wouldn't use kerl for managing my Erlang installations, otherwise, for example, but keep doing that by end, thus wasting precious time I can allocate to other chores. Mind you, I'm not a DevOps, an IT guy or a CI guy, and I might not have the manpower for that either, so I want something that "just works"โ„ข๏ธ.

There's no need to understand/debug/maintain 5000 LoC of javascript in 1-2-3-4-5-... years when something is going to stop working

Surely there isn't, which is why the action does that for you.

and there's no maintainer in sight, and I won't be able to install node and npm and webpack to build from source, etc etc.

And you don't. The action runs remotely ๐Ÿ˜„ unless you're talking about developing it yourself, which changes scope. I would also like to develop the Erlang VM without requiring gcc, but alas, somebody maintains it.

And even if I make a fix, then it's on me to release a new github action etc etc.

But isn't that what most FOSS software is about? You either trust the ones maintaining it, or you fork it and maintain it yourself. EEF chose the former, because we want no 1 or 2 developers responsible for a single thing...

Then this happened rokmoln/support-firecloud@ddfc65a#r42031652

Sure, though I don't see the relation, unless you think I was selling it to you.

Context: "

Thanks @paulo-ferraz-oliveira ! Do you know if it's already possible to add OTP-24-RC3 to the GitHub CI pipelines?

If you'd be using action setup-beam it would be possible to target 24 and this would always mean the latest as per semantic versioning rules, but there might be limitations with it (IIRC jesse's CI was not straightforward). "

  1. I don't mention "my action", especially since it's not mine,
  2. I was kindly replying to a question I was asked, and merely marginally proposed a change, while actually not making a bit deal out of it (I now feel sorry for having tried to help, actually)...
  3. "if" "you'd be" "would be" "would mean" are all for "conditional"; I could have just as well written "use setup-beam for that", but that's not how I usually approach issues,
  4. I acknowledge "there might be limitations with it",
  5. I didn't add one emoji, so I wasn't trying to be witty. (I'm sorry if it came out like that)

There's a great joy in keeping things simple,

Yes, there is, but it's not always possible, and I do think that 300 lines of JS + a few more of shell script for such convenience is not overly complicating stuff. I can easily remove around 10 of the shell script lines and 50 of the JS if I'm aiming for simpler but less easy to read > I'm not ๐Ÿ˜„

and that's why we landed with using the official erlang docker images in jesse/katt under github CI, even though Paulo and I had to find the right magic to upgrade git on their images https://github.com/for-GET/jesse/pull/101/files :)

Sure, though I didn't know better at the time... not everything I do has to be either a learning experience or painful. ๐Ÿ˜„

Wasn't straightforward but it is now, and will probably always be. While it might not to be very obvious, the current setup is "simpler, safer and more fun" (such a great motto as opposed to "smoooth" I think :) ) as opposed to any github action, no matter how perfect its code might/can be.

Not opposing to that, especially since this is not my repo., or one for which I'm a huge contributor.

That last bit is incoherent to me, and I have no context on what you were thinking, so can't actually comment on that, but if you want to explain I can probably add some more.

I invite you to be part of the Erlang Ecosystem Foundation and/or a working group (it's as free as you want it to be), where issues like this (the need for actions, better tooling, better documentation, ...) are often discussed and acted upon, and you can also reach my via the Erlang Slack, in case you want to dig deeper.

Disclaimer: no part of this text is supposed to be a criticism or offence to you, jesse or anyone involved. No BEAM was harmed in the writing of this text.

Oh, and while we're at it, I invite you to participate in https://spawnfest.org/ where you have a full 48 hours to create a viable alternative to the setup-beam GitHub Action and win some cool prizes while at it.