snoyberg / http-client

An HTTP client engine, intended as a base layer for more user-friendly packages.
281 stars 193 forks source link

Allow disabling or customizing value max header value length that raises OverlongHeaders exception #502

Closed codygman closed 1 year ago

codygman commented 1 year ago

Related to #35

I see someone had the same issue I do in:

https://github.com/hasura/graphql-engine/issues/9296#issuecomment-1357208847

Essentially hitting the api like:

#+begin_src restclient :noweb yes
POST https://api.monday.com/v2/
Content-Type: application/json
Authorization: <<monday-auth-token>>

{"query":"query{boards(limit:10 order_by:used_at){id name}}"}
#+end_src

Results in a 5735 character response, presumably for the reason of being able to embed iframes or something.

content-security-policy value of 5735 characters ``` frame-ancestors https://monday.com https://*.monday.com https://bigbrain.me https://*.bigbrain.me https://teams.microsoft.com https://*.teams.microsoft.com https://*.microsoftonline.com https://*.office365.com https://*.microsoft.com https://webbyawards.com https://www.webbyawards.com https://msteams.backend.monday.app https://monday.lightning.force.com https://monday.force.com https://www.office.com https://*.www.office.com https://outlook.office.com https://outlook-sdf.office.com https://outlook.office365.com https://outlook-sdf.office365.com https://outlook.live.com https://outlook-sdf.live.com https://app.eu.pendo.io https://msteams-eu.backend.monday.app https://msteams-au.backend.monday.app https://monday.vitally.io; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://*.monday.com https://webpack.llama.fan:3444 https://*.microsoft.com https://*.hsforms.com https://*.pusher.com https://accounts.google.com https://ajax.googleapis.com https://api.embed.ly https://apis.google.com https://app.box.com https://appvizer.one/ariadne/v1/ariadne.js https://bat.bing.com https://cdn.broadcast.am https://cdn.pdst.fm https://cdn.simpo.io/actionbar.js https://cdn.simpo.io/simpo-client.js https://cdn.taboola.com https://cdn.walkme.com https://cdnjs.cloudflare.com https://code.highcharts.com https://connect.facebook.net https://ct.capterra.com https://d18vk66ftlazd2.cloudfront.net https://d2c7xlmseob604.cloudfront.net https://edge.fullstory.com https://rs.fullstory.com https://googleads.g.doubleclick.net https://js.hsforms.net https://js.live.net https://maps.googleapis.com https://monday.com https://s.pinimg.com https://s.ytimg.com https://snap.licdn.com https://snippet.growsumo.com https://songbird.cardinalcommerce.com https://static.cloudflareinsights.com https://static.zdassets.com https://tpc.googlesyndication.com https://translate.googleapis.com https://*.zopim.com https://ws.bluesnap.com https://www.dropbox.com https://www.google-analytics.com https://www.google.com https://www.googleadservices.com https://www.googletagmanager.com https://www.gstatic.com https://www.youtube.com https://bigbrain.me https://*.bigbrain.me https://informer-cdn.monday.com https://cdn.eu.pendo.io https://app.eu.pendo.io https://data.eu.pendo.io https://pendo-eu-static-6485021788340224.storage.googleapis.com https://js.appboycdn.com/web-sdk/3.2/appboy.no-amd.min.js https://js.appboycdn.com/web-sdk/3.2/appboy.min.js https://sdk.iad-06.braze.com https://*.cdn2.monday.app https://microfrontends.monday.com https://js.braintreegateway.com https://assets.braintreegateway.com https://*.paypal.com https://cdn.monday.com https://browser.sentry-cdn.com https://*.hotjar.com https://static.ads-twitter.com https://analytics.twitter.com https://analytics.tiktok.com https://s.yimg.jp/images/listing/tool/cv/ytag.js https://cdn.linkedin.oribi.io https://trc.taboola.com https://pips.taboola.com https://cds.taboola.com https://cdn.servicebell.com https://api.servicebell.com wss://api.servicebell.com wss://ws.servicebell.com; worker-src 'self' 'unsafe-inline' blob:; connect-src 'self' https://*.monday.com https://monday.com wss://webpack.llama.fan:3444 https://webpack.llama.fan:3444 https://grsm.io https://forms.hsforms.com https://*.algolia.net https://*.algolianet.com https://bat.bing.com https://*.braze.com https://api.smartling.com https://us-central1-adaptive-growth.cloudfunctions.net https://appvizer.one https://www.facebook.com https://graph.microsoft.com https://graph.facebook.com https://api.giphy.com https://ct.pinterest.com https://storage.monday.app https://trc-events.taboola.com https://broadcast.am https://stats.g.doubleclick.net https://rs.fullstory.com https://*.cloudfront.net https://dapulse-res.cloudinary.com https://static.cloudflareinsights.com https://*.bigbrain.me https://www.dropbox.com https://www.googletagmanager.com https://ipinfo.io https://*.cardinalcommerce.com https://www.bluesnap.com https://connect.facebook.net https://app.box.com https://code.highcharts.com https://js.live.net https://monday.zendesk.com https://static.zdassets.com https://ekr.zdassets.com https://ekr.zendesk.com https://maps.googleapis.com wss://*.pusher.com https://*.pusher.com wss://*.zopim.com https://*.zopim.com https://www.google-analytics.com https://api.simpo.io https://cdn.simpo.io https://zh081jts88wj.statuspage.io https://www.googleapis.com https://cdn.eu.pendo.io https://app.eu.pendo.io https://data.eu.pendo.io https://pendo-eu-static-6485021788340224.storage.googleapis.com https://api.braintreegateway.com https://client-analytics.braintreegateway.com https://*.braintree-api.com https://*.paypal.com https://*.sentry.io https://cdn.jsdelivr.net https://prod-use1-crm-billing.s3.amazonaws.com https://prod-use1-crm-communication.s3.amazonaws.com https://files-monday-com.s3.amazonaws.com https://prod-apse2-crm-billing.s3.ap-southeast-2.amazonaws.com https://prod-apse2-crm-communication.s3.ap-southeast-2.amazonaws.com https://prod-apse2-files-monday-com.s3.amazonaws.com https://prod-euc1-crm-billing.s3.eu-central-1.amazonaws.com https://prod-euc1-crm-communication.s3.eu-central-1.amazonaws.com https://prod-euc1-files-monday-com.s3.amazonaws.com https://microfrontends.monday.com https://*.hotjar.com https://*.hotjar.io https://static.ads-twitter.com https://analytics.twitter.com https://analytics.tiktok.com https://s.yimg.jp/images/listing/tool/cv/ytag.js https://cdn.linkedin.oribi.io https://trc.taboola.com https://pips.taboola.com https://cds.taboola.com https://cdn.servicebell.com https://api.servicebell.com wss://api.servicebell.com wss://*.hotjar.com wss://ws.servicebell.com https://rum.browser-intake-datadoghq.eu https://session-replay.browser-intake-datadoghq.eu; ```

I suppose it would be possible to just increase the limit again, but the current limit is probably a pretty good default.

That's why instead I request that it be made configurable since these kinds of edge cases will likely always exist, whether justified or not.

codygman commented 1 year ago

I also see that for 2.3.8 parseRequest says:

A Request created by this function won't cause exceptions on non-2XX response status codes.

Is it expected for OverlongHeaders to be thrown when using parseRequest?

snoyberg commented 1 year ago

No objection to a PR to allow overriding the defaults.

And yes, it's expect3ed that OverlongHeaders be thrown. The comment for parseRequest means that a non-2XX status code doesn't generate an exception, which was the old behavior of the library.