Use fetch instead of axios

glynnbird commented 1 year ago

Overview

Replaces axios HTTP library with fetch, powered-by undici. The upshot of this is that we reduce the number of dependencies to zero.

Comments and advice welcome.

fetch

Some history: originally Nano was built on top of the request library which was later deprecated. At this point I reworked it to use axios instead. This PR eliminates axios and other axios-related dependencies and instead uses the new kid on the block: the fetch API.

The fetch feature has found widespread adoption in web browsers as a means of handling outbound HTTP requests. It has found its way into Node.js as a global function and is marked as an experimental feature in Node 18/19 and is mainstream in Node 20 and beyond.

Node.js's fetch capability is powered by the undici package which is bundled with Node.js and in turn uses Node's low-level network libraries instead of being based on the higher-level http/https built-in modules. It purports to be significantly faster (according to its own benchmarks) than traffic routed through http/https modules, as is the case with other HTTP libraries like axios & request.

Automated testing

Replacing axios with fetch means also ditching the nock library for mocking HTTP requests and responses, because nock works by intercepting requests originating from the http/https layer, which is bypassed by undici. Fortunately, undici provides its own Mocking tooling.

The outcome

This branch's needs no runtime dependencies. Current dependencies:

  "dependencies": {
    "http-cookie-agent": "^4.0.2",
    "@types/tough-cookie": "^4.0.2",
    "axios": "^1.1.3",
    "qs": "^6.11.0",
    "tough-cookie": "^4.1.2",
    "node-abort-controller": "^3.0.1"
  },
  "devDependencies": {
    "@types/node": "^18.11.9",
    "jest": "^29.2.2",
    "nock": "^13.2.9",
    "standard": "^17.0.0",
    "typescript": "^4.8.4"
  }

Post-PR dependencies:

  "dependencies": {
  },
  "devDependencies": {
    "undici": "^6.2.1",
    "@types/node": "^22.3.0",
    "typescript": "^5.5.4"
  }

Backwards compatibility

None of Nano's API has changed except when a user is supplying non-default connection handling parameters. Gone is requestDefaults which dates back to the "request" days and instead an optional agentOptions can be provided which is documented in the README and in TypeScript.

const agentOptions = {
  bodyTimeout: 30000,
  headersTimeout: 30000,
  keepAliveMaxTimeout: 600000,
  keepAliveTimeout: 30000,
  keepAliveTimeoutThreshold: 1000,
  maxHeaderSize: 16384,
  maxResponseSize: -1,
  pipelining: 6,
  connect: { 
    timeout: 10000
  },
  strictContentLength: true,
  connections: null,
  maxRedirections: 0
}
const undici = require('undici')
const undiciOptions = new undici.Agent(agentOptions)
const nano = Nano({ url: 'http://127.0.0.1:5984', undiciOptions })

Node versioning

As Node 16 is EOL, we can release v11 of Nano and make it the Node 18+ version. The v10 series of Nano would still be supported for the time being for older versions of node.

Testing recommendations

The test suite has been rewritten to use the built-in Node.js test runner (one fewer dependency!) and uses the undici.MockAgent to simulate responses for each of Nano's API calls, just as Nock did previously.

Run with:

npm run test

GitHub issue number

Fixes https://github.com/apache/couchdb-nano/issues/307

Related Pull Requests

n/a

Checklist

[x] Code is written and works correctly;
[x] Changes are covered by tests;
[x] Documentation reflects the changes;

glynnbird commented 1 year ago

My simple benchmarking of small HTTP requests measures that Nano "11" would be 0 to 10% faster than Nano 10.1.

Testing method: create simple Node Express app to deliver canned responses to nano requests. Time Nano 10.1 & Nano 11 in doing different nano calls over many cycles and calculate the average response time (ms).

	Nano 10.1	Nano "11"	% speed up
/_all_dbs	0.59	0.5309	10.0
/db/id	0.573	0.543	5.2
/db/_all_docs?limit=100&include_docs=true	0.793	0.779	1.8
/db/_all_docs?limit=10000&include_docs=true	23.425	23.388	0.2

^ my reading of these findings is that the smaller the response payload, the more the efficiency gains of using undici for outbound requests shows up. For larger response bodies, the benefit is swallowed up by the physics of transferring packets over the network.

2colours commented 5 months ago

@glynnbird Hi, any update or idea about how to progress this? It's not just one less dependency but axios itself has questionable code quality - along with the workarounds that currently make this package work with axios.

apache / couchdb-nano