[Feature] Expose Request Initiator Chain For Intercepted Network Requests

manoj-raghavendran commented 1 year ago

[Reposting issue along with use cases] I would love to have a feature that allows you to extract the Request Initiator Chain as seen in the Network Tab in Chrome Dev Tools (see image below). Is this something that is being considered as a potential feature to be added? If so, is there any timeline on its implementation?

Here are some popular use cases mentioned in the request for the same feature in Puppeteer:

"assuming you use it to analyze a website which includes 3rd-party code like social sharing buttons. You may want to know which requests were initiated by your own 1st-party code and which requests were initiated by 3rd-party code. This could help you spot performance problems like 3rd-party widgets that cause a lot of subsequent requests."
"We want to build request maps to vizualise which request was called from where. So we can check, where are performance/memory leaks at 3rd party includes. This could be very helpful to point out, which scripts or iframes initate a waste amount of requests/cookies/trackers. We have all information to build this, except of the initiator of the request."
"I need this to track malvertising"

Personally, I need access to the request initiator so that I can extract the tracking pixels that are dropped in my web application and build the initiator chain mapping to identify which 3rd-party is using that tracking pixel.

dgozman commented 1 year ago

@manoj-raghavendran Thank you for the issue and usecases. All of them seem valid, but also to the side of the end-to-end testing that Playwright focuses on. I am also not sure whether initiator information is easily available in all browsers.

I'll keep the issue open for prioritization, but given the above it's unlikely to be implemented soon.

manoj-raghavendran commented 1 year ago

@dgozman I understand. Thank you for the quick response. I can actually get this initiator info from the Network.Initiator object through a CDP session in Playwright, but only in Node.js, Python and .NET. The CDP session feature is not available in Playwright for Java as of yet. If the CDP session feature is made available in Playwright for Java, that in itself would be sufficient. Do you have any idea why this feature is not available in the Java version? Any possibility that it could be implemented in the near future?

dgozman commented 1 year ago

@manoj-raghavendran see https://github.com/microsoft/playwright-java/issues/823.

manoj-raghavendran commented 1 year ago

@dgozman I'd actually commented there a couple of weeks back, but I couldn't find any info on the timeline for implementing this feature. So I wanted to know if you had any idea.

BennyAlex commented 1 year ago

Please add this feature. I dont want to have to switch to puppeteer for this one feature.

ifduyue commented 1 month ago

There is now Browser.startTracing, together with Browser.recordHar, maybe we can piece together all the fragments and get request and its callFrames (I haven't validate this idea yet).

However, it's nice to include initiator info in har files and in request/requestfinished/requestfailed event data.

microsoft / playwright

[Feature] Expose Request Initiator Chain For Intercepted Network Requests #23778