Closed rstammer closed 1 month ago
rstammer
commented
2 months ago Little update: For some reason this only occurs when I add Rails' HTTP basic authentication mechanics, i.e.
class ApplicationController < ActionController::Base
http_basic_authenticate_with name: "something", password: "password"
end… adding this will result in the error described above, leaving out does not.
Edit: This error persists even after upgrading to Rails 7.2.1 and Ruby 3.3.5.
rstammer
commented
1 month ago Interestingly, the error messages were completely misleading here and the HTTP Basic auth was not the problem.
Instead, an invocation of git was causing the problems as git is no longer available by default for heroku-24 during app runtime. Removing the invocation of git fixed the build and we're successfully running on heroku-24 now.
Thx to @dansch from @dotruby for spotting the problem and providing the deciding hint 🙌
edmorley
commented
1 month ago @rstammer Ah interesting, thank you for the update. Yeah Git is no longer available at run time as documented on the Heroku-24 page.
I'm curious where the Git usage was that could lead to that error message? Was it in a dependency or app code? Do you know why the failure to find Git didn't raise an error at the point of usage?
rstammer
commented
1 month ago Unfortunately the reason why we did not observe a direct error popping up from using git is completely mysterious to me and I cannot explain from my current information.
It was part of the application code in the config/config.yml of the Rails app. The corresponding line sets some convenient default for development purposes where it tries to guess the local user's email address by employing git this way:
development:
default_recipient: <%= `git config user.email`.presence || "somebody@example.com" %>Removing this fixed the build for me, that's all I can see so far. Adding it will cause the weird error message as described above. I'd be also interested in learning where the connection is located 😅
edmorley
commented
1 month ago Ah I believe this may be due to the way Ruby's backticks work: https://ruby-doc.org/3.3.5/Kernel.html#method-i-60
On Heroku the command would have previously been exiting 1 (since there is no global Git config with a user set). eg:
$ docker run --rm -it heroku/heroku:22 bash -c 'git config user.email; echo $?'
1However, Ruby backticks require explicitly checking the exit code:
$ docker run --rm -it ruby
irb(main):001> `git config user.email`
=> ""
irb(main):002> $?
=> #<Process::Status: pid 8 exit 1>...so with older stacks this would previously have been silently failing on Heroku.
But backticks do raise when the command isn't found (so will raise if Git isn't found):
$ docker run --rm -it ruby
irb(main):001> `nonexistent-command`
(irb):1:in ``': No such file or directory - nonexistent-command (Errno::ENOENT)
from (irb):1:in `<main>'
from <internal:kernel>:187:in `loop'
from /usr/local/lib/ruby/gems/3.3.0/gems/irb-1.13.1/exe/irb:9:in `<top (required)>'
from /usr/local/bin/irb:25:in `load'
from /usr/local/bin/irb:25:in `<main>'I'm presuming this interacted badly with the templating in config/config.yml?
Perhaps @schneems knows more? :-)
schneems
commented
1 month ago But backticks do raise when the command isn't found (so will raise if Git isn't found):
TIL. I didn't know it would do that. I guard all shell-ing out because I've been bit by this problem so many times. I usually write a run!() method for example https://github.com/ruby/syntax_suggest/blob/67ce884935aaa71c07706b6e159dbee658c30c1a/spec/spec_helper.rb#L85-L89.
My best guess is that there's something that added a git executable to the path such as another buildpack or a bin/git binstub that would fail if the system git doesn't exist but which git would return success. The Ruby community is very keen to add aliases and binstubs.
Alternatively, if a templating library was catching and ignoring a raised exception it could cause something like that. It's hard to say without more information or context.
@rstammer I generally recommend support tickets for application specific problems. It's much easier for me to debug when I have access to the whole app and it's associated configuration. I tend to reserve issues for buildpack specific problems such as regressions "It used to work, but now it doesn't." In this case, the issue was found and resolved so I don't think that's strictly required.
At this stage, debugging would be to find out the root cause. If it happens to be 3rd party buildpack then it could help us save future customers from the same confusing problem. If that's where the issue came from, and we can address the problem in that location. Could you list out the buildpacks the app is using?
For example:
$ heroku buildpacks
=== arcane-beach-22672 Buildpack URL
heroku/rubyThanks a lot for diving in and sharing your insights, @schneems & @edmorley 👍 That is very valuable for us!
Your suggestion makes totally sense! I already have filed a suppport ticket some days before I created the issue here but decided to open an issue here anyway as my naive reasoning concluded that the buildpack logic may in the culprit and others may experience a similar issue because our general application setup is pretty default and not too crazy.
To anyone reading this later on: The error was raised in loading the config, but the error got "swallowed" by a rescue block that was adversely wrapping loading the YAML file. That's why we did not observe the initial error. Many thanks to the detective work by the heroku team figuring this out and we surely have to re-think that rescue-ing🕵️
Thanks everybody for engagement and a good weekend!
Hi there! We're using Rails v7.1.3 (edit: same problem with 7.2.1) running on C-Ruby v3.3.1 and do have issues deploying the application using the
heroku-24stack with the usual heroku Ruby buildpack.What we've tried to fix it:
Here's the error log for the variant with Ruby 3.3.1👇 What made me suspicious is that in the logs some gems are loaded from
/app/vendor/bundle/ruby/3.3.0and some from/app/vendor/ruby-3.3.1/which seems paradox at first but the Gemfile specifies 3.3.1 as well as the.ruby-versionfile.With
heroku-22stack all is fine so far. Are we doing something wrong here or is this a buildpack problem 🤔