When trying to include URLs to walmart.com, I found out that Walmart was sending headers that were too large for node's default 8 MB limit (which they dropped down from 80 MB). In a later version of Node (I think it was Node 16), they added a flag to override the default limit. Since I was updating node anyways, I updated it to the latest LTS release as well as added some documentation in the README file describing the gotcha.
xD Wow, how'd I get that measurement stuck in my head. They're sending 18 KB of headers, not MB. My bad. Every time I said MB I meant KB. Sorry about the confusion.
When trying to include URLs to walmart.com, I found out that Walmart was sending headers that were too large for node's default 8 MB limit (which they dropped down from 80 MB). In a later version of Node (I think it was Node 16), they added a flag to override the default limit. Since I was updating node anyways, I updated it to the latest LTS release as well as added some documentation in the README file describing the gotcha.