404 error page is served with wrong Content-Type header

[wolfgangrittner]

Bridgetown Version: 0.21.0

To Reproduce

• run yarn start
• open a browser and navigate to your bridgetown instance, try to access a page that doesn't exist so you get your 404.html page served, example: http://localhost:4000/some-page-that-does-not-exist

Current behavior

If the 404.html page contains any non-escaped special characters (example: German umlauts, like ü or ä), they are not displayed correctly. The Content-Type response header for the 404 response is content-type: text/html; charset=ISO-8859-1.

Expected behavior

Any special characters are displayed correctly, and the 404 response is served using the same content-type and encoding as any other response. In my case, and that's the default as far as I know, I'd expect content-type: "text/html; charset=UTF-8".

Screenshots

(instead of "zurück zur Startseite")

Computing environment (please complete the following information):

• OS: Mac OS 11.2
• Browser: latest chrome, safari, firefox, curl (doesn't matter)
• Ruby Version: 2.7.2p137
• output of locale in the terminal where yarn start was running:

  LANG="en_US.UTF-8"
  LC_COLLATE="en_US.UTF-8"
  LC_CTYPE="en_US.UTF-8"
  LC_MESSAGES="en_US.UTF-8"
  LC_MONETARY="en_US.UTF-8"
  LC_NUMERIC="en_US.UTF-8"
  LC_TIME="en_US.UTF-8"
  LC_ALL=

Additional context

I suspect the culprit is this line here: https://github.com/bridgetownrb/bridgetown/blob/main/bridgetown-core/lib/bridgetown-core/commands/serve.rb#L93 When serving the 404 response, it tries to set a Content-Type header (title cased), but there's already a content-type header (lower case) present, which seems to take precedence.

I believe I've a fix for this already, would be happy to provide a PR if that's cool?

[jaredcwhite]

Thanks for the details on this. We're actually in the process of removing Webrick and switching to a Rack-based server infrastructure, so I wouldn't bother trying to patch what's there now. I'll leave this issue open though so we can double-check the new system fixes the problem.

[jaredcwhite]

I can't seem to repro with the new Puma/Rack server…and on Ruby 3 at least I can't even repro with Webrick either (returns a UTF-8 charset). Closing.