Error "/usr/bin/jq: Permission denied" with heroku-buildpack-nodejs

[kingdonb]

I ran into this issue: (heroku/heroku-buildpack-nodejs#856) which is in summary, related to an outdated slug base image, or a base image that does not match closely enough the upstream Heroku base image (apparently the main issue is that we did not include a jq binary. Not sure how close or far we are from upstream in other ways, or if we could simply pull an upstream to resolve this; I haven't checked yet, wanted to document the workaround first...)

My project that triggered this issue is the Team Hephy blog, which is using the multi-buildpack (using .buildpacks to enumerate several specific buildpacks that should be loaded together.)

The .buildpacks referenced directly --> https://github.com/heroku/heroku-buildpack-nodejs.git <-- (the HEAD of nodejs buildpack), this issue is in the current release which as of this time is v182, and all the way back to v177.

I was able to get my build going again by updating .buildpacks to point directly at the v176 release, (which does not suffer from this issue according to the reports referenced above, in the Heroku upstream buildpack.)

Like this:

https://github.com/heroku/heroku-buildpack-nodejs.git#v176
https://github.com/heroku/heroku-buildpack-ruby.git
https://github.com/yebyen/heroku-buildpack-jekyll.git

Unfortunately my release failed for an unrelated reason (I have upgraded my cluster to K8s v1.16+ without first doing the v2.22 upgrade, whoops, and now I'm pretty sure I'm going to have to reinstall Workflow from scratch, (but unfortunately may ultimately be blocked again from doing this, by the Azure storage changes which I'm sure I haven't resolved yet either...))

Fortunately, even with all of that, the build itself succeeded, and it is still possible for people in my situation to roll releases ahead manually by editing the deployment to point to the new release commit hash and secret (great news here, secrets and configmaps were not affected by apps/v1 api transition in K8s v1.16) so despite this comedy of errors, the Hephy Workflow Blog's announcement of Workflow v2.22.3 release has been published, (and just about two weeks after the fact! 🎉🥉)

https://blog.teamhephy.info/blog/posts/announcements/release-v2-22-3.html

[Cryptophobia]

Very nice find @kingdonb and thank you for publishing the new release on the blog!

Sounds like for the issue, we should just find the best way to:

1. Install jq in the slugbuilder as that seems to be the main problem for the nodejs buildpack.
2. Figure out how to stay as close as possible to other slugbuilders out there in terms of dependencies in our image in the future.

What do you think?

[kingdonb]

It looks like this might be fixed by heroku:20-build - I have found the jq binary in there, and am able to build again using the latest version of dockerbuilder from #26 (pushed as docker.io/kingdonb/slugbuilder:git-5b32b8a)

I guess heroku 18 is EOL with respect to the nodejs buildpack?

[yebyen]

I think the build for #26 failed this last time because of rate limiting from GitHub on Travis

It should succeed on the next attempt, I will rebase and squash tomorrow to give it another try 👍

[Cryptophobia]

Great work! If we upgrade to heroku-20 it may break previous slugs so users may be forced to rebuild their slugs. However, I see that as something that is inevitable with a new heroku-20 upgrade.

I triggered travis manually and looks like it is passing now!

[kingdonb]

I am not sure if this is true in the reverse, or universally true, but I was able to use heroku-20 slugs with a heroku-18 slugrunner. I have not tested combinations exhaustively, it would make sense that users would need to rebuild slugs though.

It may break something and it is worth testing, but I don't think there's a particularly high degree of risk (and things are already objectively broken for anyone who has been keeping up with the latest nodejs buildpack at this point, which we're already shipping by default to users who haven't selected any buildpack/version...)