Case sensitive headers

[FabianOudhaarlem]

Hi there,

I am trying to consume a Restfull JSON API from a offsite payment provider but they decided header namings are in fact case sensitive for them. I need to provide an api_key (all lowercase) header but the httparty gem is forcing capitalized header namings.

Is there any way i can disable this modification of header namings?

Thanks in advance.

[jnunemaker]

Pretty sure httparty just passes headers to net/http, so there isn't anything we can do. If anyone can show different, happy to dig in.

[FabianOudhaarlem]

For anyone having the same problem, overriding the immutable header key class from net::http is a solution.

require 'net/http'

class Net::HTTP::ImmutableHeaderKey
  attr_reader :key

  def initialize(key)
    @key = key
  end

  def downcase
    self
  end

  def capitalize
    self
  end

  def split(*)
    [self]
  end

  def hash
    key.hash
  end

  def eql?(other)
    key.eql? other.key.eql?
  end

  def to_s
    key
  end
end

And in your HTTParty class:

headers Net::HTTP::ImmutableHeaderKey.new('api_key') => 'HIDDEN_API_KEY_VALUE'

[arjun-urs]

+1. Thanks @FabianOudhaarlem

However, this worked in ruby 2.0.0 but not in 2.3.0

[olivierlacan]

Aside from the idea of hacking Net::HTTP to work with case sensitive headers you should probably note that https://www.w3.org/Protocols/rfc2616/rfc2616-sec4.html#sec4.2 defines HTTP message headers as follows:

HTTP header fields, which include general-header (section 4.5), request-header (section 5.3), response-header (section 6.2), and entity-header (section 7.1) fields, follow the same generic format as that given in Section 3.1 of RFC 822 [9]. Each header field consists of a name followed by a colon (":") and the field value. Field names are case-insensitive. The field value MAY be preceded by any amount of LWS, though a single SP is preferred. Header fields can be extended over multiple lines by preceding each extra line with at least one SP or HT. Applications ought to follow "common form", where one is known or indicated, when generating HTTP constructs, since there might exist some implementations that fail to accept anything

I'm not trying to be the one that yells fix your server — since a third-party API is rarely something you can fix – but I'd suggest filling a bug report for any server that refuses identical header keys with a different case.

[earthboundkid]

For future googlers, this works in Ruby 2.3 AFAICT:

module Net::HTTPHeader
  def capitalize(name)
    name
  end
  private :capitalize
end

In my initial tests, this solved the problem with my communication with the non-RFC-compliant server.

[johnnaegle]

I think @FabianOudhaarlem solution doesn't work on Ruby 2.3 because to_s is called here:

  def capitalize(name)
    name.to_s.split(/-/).map {|s| s.capitalize }.join('-')
  end
  private :capitalize

That would turn the Net::HTTP::ImmutableHeaderKey into a regular string.

[johnnaegle]

Here is a terrifying hack that does work on Ruby 2.3. The first time to_s is called, the ImmutableHeaderKey returns a version that can't be capitalized. After that, it just returns a string.

class Net::HTTP::ImmutableHeaderKey
  attr_reader :key

  def initialize(key)
    @key = key
  end

  def downcase
    self
  end

  def capitalize
    self
  end

  def split(*)
    [self]
  end

  def hash
    key.hash
  end

  def eql?(other)
    key.eql? other.key.eql?
  end

  def to_s
    def self.to_s
      key
    end
    self
  end
end

You have to create a new key for each request. No headers in the class:

  response = HTTParty.get('https://thisstupidserverusescasesensitiveheaders.com/woops', headers: {
      'Content-Type' => 'application/json',
      Net::HTTP::ImmutableHeaderKey.new('api_key') => 'OFMG',
      Net::HTTP::ImmutableHeaderKey.new('api_secret') => 'NOOO'
    }
  )

This works only because the ruby internals does this:

module Net::HTTPHeader
  def capitalize(name)
    name.to_s.split(/-/).map {|s| s.capitalize }.join('-')
  end
  private :capitalize
end

If the behavior changed to not call to_s or call it more than once, its probably going to blow up.

YMMV

[yxhuvud]

Typhoeus can send lowercase header names if they are supplied as strings, so that is a nicer workaround if monkeypatching Net::HTTP isn't acceptable.

[theasteve]

What about passing numbers in the headers? I created a curl get request that works with the header of "AuthDate: 1531403501" However, in Ruby I get the following error:

\"Authdate\":\"1531403501\"}" }, { "error_code": "external_auth_error", "error_message": "Date header is missing or timestamp out of bounds" } ] }

I tried passing it hard copied without success:

def generate_headers
      {
        "Authorization"  => "HMAC-SHA256 api_key='#{@api_key}' signature='#{@signature}'",
        "AuthDate" => @timestamp
      }
 end

After using @carlmjohnson patch I get all the fields lower cased

[ { "error_code": "missing_header", "error_message": "Request is missing header: Authorization" Scheme\":\"https\",\"authorization\":\"HMAC-SHA256 api_key='XXXXXXXXX' signature='XXXXXXXXXXX'\",\"authdate\":\"1531422290\"}" } ] }

In that way not being able to have a successful request. Could something like this work with your patch?

def generate_headers
      {
        "Authorization"  => "HMAC-SHA256 api_key='#{@api_key}' signature='#{@signature}'",
       Net::HTTP::ImmutableHeaderKey.new('AuthDate') => '#{@timestamp}'
      }
end

Thank you @johnnaegle I was able to fix the error by using the monkey patch you shared. I had to change the way I generated the UNIX timestamp from@timestamp = Time.now.to_i.to_s to @timestamp = Time.now.to_i

[zw963]

module Net::HTTPHeader def capitalize(name) name end private :capitalize end

This work for Ruby 2.5.1

[jatindhankhar]

Instead of monkey patching the Net::HTTPHeader, a new extended String class with new capitalize can fix the issue. I found following solution on Calvin's blog https://calvin.my/posts/force-http-header-name-lowercase (Thanks Calvin)

Posting the solution, in case url ever goes down

 class ImmutableKey < String 
         def capitalize 
               self 
         end 
 end

and use it as following

  {
    ImmutableKey.new("key-should-be-small") => "val",
   "Key-can-be-uppercase" => "r"
 }

Use it only for keys that need to be preserved in a particular case

[merchang]

Instead of monkey patching the Net::HTTPHeader, a new extended String class with new capitalize can fix the issue. I found following solution on Calvin's blog https://calvin.my/posts/force-http-header-name-lowercase (Thanks Calvin)

Posting the solution, in case url ever goes down

 class ImmutableKey < String 
         def capitalize 
               self 
         end 
 end

and use it as following

  {
    ImmutableKey.new("key-should-be-small") => "val",
   "Key-can-be-uppercase" => "r"
 }

Use it only for keys that need to be preserved in a particular case

Note for 2.3.0 and above, you will also need to include definitions for downcase & to_s

https://stackoverflow.com/a/42121370/18820441

[NotGrm]

I encountered exactly the same issue when integrating one of our partner at work (their API is pretty old, and require a specific header to be sent downcased; and they don't want to touch the API to follow HTTP standard).

Here is how I managed to fix the problem

CaseSensitiveHeader = Data.define(:key) do
  def split(*)
    key.split(*).map { |k| self.class.new(k) }
  end

  def capitalize = key
  def downcase = self
  def to_s = self
end

# Usage
HTTParty.get('url', headers: { CaseSensitiveHeader['x-sensitive-header'] => 'value' })