Unexpected 449 Retry With on regular GET request to a public resource

[DesiredName]

Bug Description

I'm righting a news aggregator app and faced strange behavior when I issue a fetch GET requests to the URLs:

https://www.o2.pl https://www.wp.pl

I get back 449 Retry With.

Requesting the same URL in BUN runtime with fetch or curl, or powershell, or postman results in regular 200 Ok code. Requesting other sites (yahoo, msn, etc) results in 200 also.

It might not be a bug, but look really strange that only Node only for this two sites results in 449.

Reproducible By

fetch("http://www.o2.pl/").then((response) => console.log(response.status)); // 449 fetch("http://www.wp.pl/").then((response) => console.log(response.status)); // 449

Expected Behavior

200 Ok code

Logs & Screenshots

Environment

Node 18, 20, 22, 23 NPM 10 Windows 11

Additional context

[metcoder95]

The same happens for node:http, I'm sure they are using TLS Fingerprint to block Node.js client's request. I could bypass that by shuffling the Cipher order to change the fingerprint and was able to have a 200.

Sadly, most likely you'll need to do this trick as well:

import { request } from 'node:https'
import { DEFAULT_CIPHERS } from 'node:tls'

import { Agent } from './index.js'

const defaultCiphers = DEFAULT_CIPHERS.split(':')
const shuffledCiphers = [
  defaultCiphers[0],
  // Swap the 2nd & 3rd ciphers:
  defaultCiphers[2],
  defaultCiphers[1],
  ...defaultCiphers.slice(3)
].join(':')

const agent = new Agent({
  // allowH2: true,
  maxRedirections: 0,
  connect: {
    ciphers: shuffledCiphers
  }
})

agent.dispatch(
  {
    method: 'GET',
    origin: 'https://www.wp.pl',
    path: '/',
    headers: {
      'User-Agent': 'curl/8.7.1',
      Accept: '*/*',
      Range: 'bytes=0-99'
    }
  },
  {
    onConnect (abort) {},
    onHeaders (statusCode, headers) {
      console.log('Status:', statusCode) // 200
      return true
    },
    onData (data) {
      return true
    },
    onComplete () {},
    onError (error) {
      console.error('Error:', error)
    }
  }
)

const handle = request({
  // origin: 'https://www.wp.pl',
  hostname: 'www.wp.pl',
  path: '/',
  method: 'GET',
  ciphers: shuffledCiphers
})

handle.on('response', res => console.log(res.statusCode) || res.destroy()) // 200

handle.end()

[DesiredName]

@metcoder95 , hi! Thank you for your interest! This is interesting - they don't like NodeJS... Yes, with your code I also got a 200 response. SO that is not a bug of Node, I think, I'd better close this now?

[metcoder95]

Yeah, its indeed not a bug but rather a server thing; let's close it!