Don't check ident request remote address

[BtbN]

It is not part of the actual request, and in case of forwarded requests outright breaks being able to deliver the correct response.