[🐛 Bug]: Network interception freeze whole application for request when content-type = raw

maciekpazola commented 1 month ago

What happened?

After updating Selenium from version 4.23 to 4.24 or 4.25, I encountered an issue where I can no longer capture a specific request within the application. This request is responsible for sending a file to the backend with a content type set as raw.

Although I don't receive any exceptions from Selenium, the tests seem to freeze because selenium blocks the request from complete. Since I'm tracking network requests asynchronously, the test ultimately fails on a different assertion.

The same scenarios works correctly manually and on 4.23 selenium version. Issue appear appear both on chrome and edge.

request as curl: curl "http://t1.vie1qacl113/theservice/v0001/restun/UploadSessionAppendChunkRaw" ^ -H "Accept: application/json, text/plain, */*" ^ -H "Accept-Language: en" ^ -H "Authorization: [TokenValue]" ^ -H "Connection: keep-alive" ^ -H "Content-Type: raw" ^ -H "Cookie: MixedSecurity._strReturnUrl=/Client/WEB/Main.aspx; ASP.NET_SessionId=olgs43wzcg3mh1vhgzrbcsbz" ^ -H "Origin: http://t1.vie1qacl113" ^ -H "Referer: http://t1.vie1qacl113/tdwv/" ^ -H "TenantName: t1" ^ -H "The-Client-Type: 19" ^ -H "The-Timezone-IANA: Europe/Vienna" ^ -H "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/129.0.0.0 Safari/537.36" ^ -H "X-The-UploadSession-ChunkPosition: 0" ^ -H "X-The-UploadSession-Id: 664ca222-888c-4ca1-9f52-46af77b82adb" ^ --data-raw [File Content]

How can we reproduce the issue?

Intercept traffic in application and then try to catch a request which send a file as raw.

Relevant log output

I've not found any logs in logtype.browser or driver on minimum log level trace

Operating System

win 11

Selenium version

.net 4.24

What are the browser(s) and version(s) where you see this issue?

Chrome 129

What are the browser driver(s) and version(s) where you see this issue?

Using newest drivers with seleniumdrivermanager

Are you using Selenium Grid?

No response

github-actions[bot] commented 1 month ago

@maciekpazola, thank you for creating this issue. We will troubleshoot it as soon as we can.

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

nvborisenko commented 1 month ago

Let's compare internal selenium logs and try to find out any difference: https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/

maciekpazola commented 1 month ago

Let's compare internal selenium logs and try to find out any difference: https://www.selenium.dev/documentation/webdriver/troubleshooting/logging/

The main difference I noticed is that in version 4.24, there are multiple 404 error logs:

TRACE HttpCommandExecutor: << StatusCode: 404, ReasonPhrase: 'Not Found', Version: 1.1, Content: System.Net.Http.StreamContent, Headers:
{
  Cache-Control: no-cache
  Content-Length: 1073
  Content-Type: application/json; charset=utf-8
}

Logs: SeleniumLog_Selenium4.23_Passing.txt SeleniumLog_Selenium4.24_Failing.txt

nvborisenko commented 1 month ago

Nothing interesting in logs, I guess 404 is side effect caused by incorrect network interception.

What I can propose:

Selenium team will add more internal logging around CDP commands
You can try to capture all CDP interactions via try { // execute your network continuation here } catch(Exception ex) { // capture it somewhere, to console? }. All event handlers are invoked like "fire and forget", probably introducing try/catch block we will see something useful.

diemol commented 1 week ago

I will close this as the issue has not had any more activity.

maciekpazola commented 1 week ago

@diemol @nvborisenko

I've conducted further investigations with the new logging enabled, and the issue remains unchanged. The problematic point appears to be the uploadsessionappendChunkRaw call, which can be identified in the logs if you scroll down. It seems possible that the Selenium log may struggle to process requests when uploading as raw data. NewSeleniumLog_4.26.txt

There is also an error present, though not directly associated with this request. It was thrown time to time.Here’s a snippet: 15:09:59.868 TRACE DevToolsSession: CDP RCV << {"method":"Fetch.requestPaused","params":{"requestId":"interception-job-13.0","request":{"url":"http://t1.vie1qacl113/theservice/v0001/restun/GetViewingThumbnails","method":"POST","headers":{...},"postData":"{\"DocNo\":61,\"VersionNo\":1,\"StreamNos\":[0],\"SizeInPixels\":580,\"ZeroBasedStreamNo\":true,\"ConversionOptions\":{\"RenderWordComments\":true}}","hasPostData":true,"initialPriority":"High","referrerPolicy":"strict-origin-when-cross-origin"},"frameId":"50955F4FD3B7DF055EBF914F63458EB3","resourceType":"XHR","networkId":"45272.578"},"sessionId":"521FC31CB8B249EEEF0D2086038FB91E"} 15:09:59.869 TRACE DevToolsSession: CDP SND >> 41 Fetch.getResponseBody: {"requestId":"interception-job-11.0"} 15:09:59.869 TRACE DevToolsSession: CDP SND >> 42 Fetch.continueRequest: {"requestId":"interception-job-13.0"} 15:09:59.872 WARN DevToolsSession: CDP VNT ^^ Unhandled error occurred in the event handler of the Fetch.requestPaused method. OpenQA.Selenium.DevTools.CommandResponseException: Fetch.continueResponse: Invalid InterceptionId.

The logs reveal that an error occurs in the Fetch.requestPaused event handling, specifically with an invalid InterceptionId. This might be impacting the stability of the process, potentially due to an unhandled asynchronous operation during the request upload.

Could you please reopen the ticket?

Thank you and best regards.

SeleniumHQ / selenium