Azure / azure-sdk-for-js

This repository is for active development of the Azure SDK for JavaScript (NodeJS & Browser). For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/javascript/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-js.
MIT License
2.06k stars 1.19k forks source link

Cosmos DB occasionally throws Invalid URL error when using items.create #30291

Closed joelwright-dev closed 2 months ago

joelwright-dev commented 3 months ago

Describe the bug When using the items.create method repeatedly to upload data, after a while and more often than not (roughly about 80% of the time), the app will fail with an ECONNRESET or ETIMEDOUT error code. Upon further investigation the error it appears to come from the @azure/core-rest-pipeline/dist/commonjs/util/sanitizer.js file on line 104:

node:internal/url:796
    this.#updateContext(bindingUrl.parse(input, base));
                                   ^

TypeError: Invalid URL
    at new URL (node:internal/url:796:36)
    at Sanitizer.sanitizeUrl (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\util\sanitizer.js:104:21)
    at IncomingMessage.<anonymous> (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\util\sanitizer.js:72:29)
    at JSON.stringify (<anonymous>)
    at Sanitizer.sanitize (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\util\sanitizer.js:63:21)
    at [nodejs.util.inspect.custom] (C:\Users\joelw\Development\zaiix\zaiix-scraper\zapk-search-sample\node_modules\@azure\core-rest-pipeline\dist\commonjs\restError.js:27:64)
    at formatValue (node:internal/util/inspect:805:19)
    at inspect (node:internal/util/inspect:364:10)
    at formatWithOptionsInternal (node:internal/util/inspect:2298:40)
    at formatWithOptions (node:internal/util/inspect:2160:10) {
  code: 'ERR_INVALID_URL',
  input: ''
}

And it appears that whatever this sanitizer function is, is failing to parse an empty string. I don't know why an empty string would be parsed in the first place? But either this function should be fixed to handle this, or the items.create method needs to be fixed to not parse a blank string.

Can anyone explain why this is happening?

Expected behavior Item gets added to the container.

github-actions[bot] commented 3 months ago

@deyaaeldeen @jeremymeng @joheredi @maorleger @mpodwysocki @timovv @xirzec

github-actions[bot] commented 3 months ago

Thank you for your feedback. Tagging and routing to the team member best able to assist.

joelwright-dev commented 3 months ago

Additional info: This problem appears to occur when many requests are made at once (potentially looking at some kind of accidental DOS attack on my own database, however this is what the rate limiting should account for..

jeremymeng commented 3 months ago

Thanks for report this issue @joelwright-dev! Very likely this would be addressed by PR https://github.com/Azure/azure-sdk-for-js/pull/30151

jeremymeng commented 2 months ago

The issue was that when we log request and responses, we try to remove sensitive informations. Our sanitizer wasn't handling url property of empty string properly so when there's some property or nested property with url name and empty string value this error was thrown.

We just released a fix to the TypeError: Invalid URL issue in @azure/core-rest-pipeline 1.16.2. Please refresh your dependencies to get the latest version.