I think it's because different algorithm is used for tone mapping.
Firefox: jxl-oxide-wasm detects Firefox and requests jxl-oxide to do tone mapping. jxl-oxide uses Rec. 2408 YRGB method with peak detection (instead of using signalled peak luminance), so it may appear brighter than needed.
Chrome: jxl-oxide-wasm passes decoded HDR image as-is, so Chrome performs tone mapping. IIRC it uses different tone mapping algorithm, so colors look different.
GIMP: It seems to use Rec. 2408 without peak detection to me, not sure though.
Unfortunately there's no single correct way to tone map to SDR. I suppose it will look better without peak detection though.
This image:
Looks so different from GIMP that it seems something with colorspace isn't right.
Screenshot with Firefox, Chrome and GIMP.
In my opinion: