MicrosoftEdge / WebView2Feedback

Feedback and discussions about Microsoft Edge WebView2
https://aka.ms/webview2
449 stars 55 forks source link

Support for getting and setting the cookies #4

Closed MichaelKetting closed 3 years ago

MichaelKetting commented 5 years ago

I looked at the API docs (https://docs.microsoft.com/en-us/microsoft-edge/hosting/webview2/reference/iwebview2webview) but didn't see find a matching API, so here's the feature request:

It would be great to be able to get and set the cookies used by the WebView. Internet Explorer allows getting the cookies by calling InternetGetCookieEx() from wininet.dll.

With this feature you build an application that uses both the browser control and a side channel when interacting with a website.

Edit: Possibly, the WebResourceRequested event might allow access via the raw headers from the GetResponse method (https://docs.microsoft.com/en-us/microsoft-edge/hosting/webview2/reference/iwebview2webresourcerequestedeventargs#interface_i_web_view2_web_resource_requested_event_args) but that's quite the effort.

AB#27223320

david-risney commented 5 years ago

Yes, the WebResourceRequested event might work as a work around but it is not ideal. I'll make sure cookies are on our backlog for APIs. Thanks.

david-risney commented 5 years ago

Can you say what you would want to use this API for? Is it managing auth related cookies?

MichaelKetting commented 5 years ago

@david-risney yes, mainly auth- and session cookies. Basically, whenever you build a desktop application that hosts the browser control to interact with a specific known site (or web application product) and you also need to do some out-of-band communication stuff, e.g. uploading/downloading content or doing status messages, calling web services, etc, you need to be able to share the authentication data.

One example would be to build a rich-client desktop integration for an issue tracker or build server, where you can login to the web server and then also are able to expose the attachements or build artifacts directly to the desktop.

edwardchanjw commented 5 years ago

Hopefully support all under Chrome's Dev Tools's Application Storage. But the highlighted 3 hopefully be there.

  1. Local Storage
  2. Session Storage
  3. IndexedDB
  4. Web SQL
  5. Cookie
verelpode commented 5 years ago

Yes please! It's very important to improve the ability to list, read, set, and clear cookies, especially when WebView2 uses a separate-process mode, and it is a major problem currently! Please see my explanation in the section "Inability to download PDF files because of cookie access failure" in issue #38.

Although the following cookie issues are in the repo for WPF WebView 1, these same problems are also applicable to WebView2. When designing WebView2, we would greatly appreciate it if you could solve these cookie difficulties:

verelpode commented 5 years ago

WebResourceRequested usually doesn't work for reading cookies because IWebView2WebResourceRequestedEventArgs.Response is always null because it's not intended for apps to read, rather it's intended for apps to intercept a Request and generate a different Response. To collect cookies, you'd typically need to look for set-cookie headers in the Response not in the Request, but WebResourceRequested actually only gives you access to the Request, not the Response. Hence the name is "WebResourceRequested" not "WebResourceResponded".

When a website is viewed in WebView2 and then you try to use Windows.Web.Http.HttpClient to download a related URL from the same website, the website may reject the HttpClient because of missing cookies -- the website requires its session cookies for security/auth reasons, but these cookies are only available in WebView2 and not in HttpClient. Thus we need a GetCookies method in WebView2 that allows us to retrieve all of the cookies, and then our app will be able to insert those cookies into HttpClient or rather Windows.Web.Http.HttpCookieManager. Thus a GetCookies method makes it possible to bridge the gap between WebView2 and HttpClient.

Although I mentioned Windows.Web.Http.Filters.HttpBaseProtocolFilter.CookieManager in my previous message, it's not essential to make WebView2 directly or automatically compatible with HttpBaseProtocolFilter.CookieManager. WebView2 can be independent of HttpBaseProtocolFilter and HttpClient. The core thing that is really needed is a GetCookies method in WebView2, and then apps can transfer this cookie data into any other relevant services (such as HttpClient) by themselves.

verelpode commented 5 years ago

The cookie problem is our highest priority currently -- it is a major problem in our environment. Would it be helpful if I start discussing the precise details of the proposed cookie API? I suggest adding to IWebView2WebView:

public HRESULT get_Cookies(IWebView2HttpCookieCollection ** value);

Interface IWebView2HttpCookieCollection would be approximately the C++/COM equivalent of Windows.Web.Http.HttpCookieCollection.

Each cookie in the IWebView2HttpCookieCollection would be an instance of interface IWebView2HttpCookie, which would be the C++/COM equivalent of Windows.Web.Http.HttpCookie. Thus IWebView2HttpCookie would contain members equivalent to the following methods of Windows.Web.Http.HttpCookie:

string Domain { get; }
string Path { get; }
string Name { get; }
string Value { get; set; }
System.DateTimeOffset? Expires { get; set; }
bool HttpOnly { get; set; }
bool Secure { get; set; }

In our environment, it's much less important to support put_Cookies than get_Cookies, but still desirable. get_Cookies solves the problem where WebView2 is used first, followed by Windows.Web.Http.HttpClient, but what about the reverse order? In situations where HttpClient is used first, followed by WebView2, ideally WebView2 would support put_Cookies in order to copy cookies from HttpClient to WebView2. Thus I suggest that IWebView2WebView be given both of:

public HRESULT get_Cookies(IWebView2HttpCookieCollection ** value);
public HRESULT put_Cookies(IWebView2HttpCookieCollection * value);

put_Cookies would replace all cookies in WebView2, thus it would also delete cookies (in the normal manner of replacing the contents of a collection).

I also suggest making the following methods in IWebView2WebView:

// Add or replace (but not delete) multiple cookies in WebView2:
public HRESULT AddOrReplaceCookies(IWebView2HttpCookieCollection * cookies);

// Add or replace one cookie in WebView2:
// This could be named "SetCookie" or "AddOrReplaceCookie":
public HRESULT SetCookie(IWebView2HttpCookie * cookie);

// Delete one cookie:
public HRESULT DeleteCookie(LPCWSTR domain, LPCWSTR path, LPCWSTR name);
// Alternatively:
public HRESULT DeleteCookie(IWebView2HttpCookie * cookie);

// Delete all cookies in the WebView2:
public HRESULT DeleteAllCookies();

// Get one cookie by name (exact match of domain, path, name):
// Output is null if no matching cookie is found:
public HRESULT GetCookie(LPCWSTR domain, LPCWSTR path, LPCWSTR name, IWebView2HttpCookie ** outCookie);

// Get all cookies for the specified domain:
// Matches any value of `HttpCookie.Path` and `HttpCookie.Name`:
public HRESULT GetCookies(LPCWSTR domain, IWebView2HttpCookieCollection ** outCookies);

If it's too difficult to support all of those, then at least the get_Cookies alone would be a big relief in our case.

michalkania commented 4 years ago

I would like to point out that Windows.Web.Http.HttpClient is Windows specific and it can share resources of Windows.Web.Http.HttpCookieManager in WebView on Windows, but as @verelpode mentioned, it doesn't have to work that way with WebView2.

Anyway what about other platforms?

With Xamarin and .Net Core System.Net.Http.HttpClient is probably the way to go. It already is done with multiple frameworks in mind. Making WebView2 share common classes would be desirable. Maybe there could be a way to pass a reference of our System.Net.CookieContainer or even System.Net.Http.HttpClient to get this feeling of how Windows.Web.Http.HttpClient is working on Windows with WebView.

dd8 commented 4 years ago

We also have a requirement for this. We need to get customisation/authentication cookies on a per URL basis to send in web crawler requests. Use case:

1) user runs our app with embedded webview 2) user navigates to a web site with a login using webview 3) website login returns authentication cookie 4) user then starts scan of site with embedded crawler which adds cookies to HTTP requests (this allows sites with logins to be scanned)

On macOS we use [[NSHTTPCookieStorage sharedHTTPCookieStorage] cookiesForURL:url]. This API works well - the calling code doesn't need to worry about cookie domain, path or expiry, or whether the cookie is temporary or persistent.

romanov commented 4 years ago

Any updates?

david-risney commented 4 years ago

Nothing concrete to report yet. But cookie management is a popular feature request and one we will be looking into.

angshuman-agarwal commented 4 years ago

Make it like CEF - https://github.com/cefsharp/CefSharp/blob/master/CefSharp.Core/CookieManager.h

pipalot commented 4 years ago

I also need cookie management so that I can add an auth cookie to the request being sent to the website I am navigating to.

It has been suggested in this discussion that it might be possible to set cookies in the WebResourceRequested event, but there seems to be a bug in WebResourceRequested that ignores attempts to set the headers (despite the documentation saying you can do it).

For example, this fails to add a cookie:

       private void CoreWebView2_WebResourceRequested(object sender, CoreWebView2WebResourceRequestedEventArgs e)
        {
                e.Request.Headers.Add("Cookie", "cookieName=cookieValue");
        }

Another example of WebResourceRequested failing to allow headers to be set, this time involving trying to set the Authentication property: https://stackoverflow.com/questions/62094181/edit-http-request-header-with-webview2

If you inspect the headers immediately after trying to set them, you will see that nothing has been set.

So I think we must wait for a cookie management feature to be added to WebView2, or the bugs in WebResourceRequested to be fixed.

Is there any estimate of when either of those is likely to be addressed please? Many thanks.

killerbobjr commented 4 years ago

Cookies can be accessed from the Chrome "Cookie" database with the Chrome DevTools Protocol (C++ example):

// Subscribe to the specific Chrome DevTools Protocol event we want to use.
wil::com_ptr<ICoreWebView2DevToolsProtocolEventReceiver> receiver;
HRESULT retval = ICoreWebView2_ComPointer->GetDevToolsProtocolEventReceiver(L"Network.getAllCookies", &receiver);

// Since we're not looking to receive a notification that a command "Network.getAllCookies" has been issued (when
// 'CallDevToolsProtocolMethod' is called), we can ignore 'receiver->add_DevToolsProtocolEventReceived'.

// Now that we have allowed the Chrome DevTools Protocol handler to accept the "Network.getAllCookies" message,
// let's grab all the cookies.
ICoreWebView2_ComPointer->CallDevToolsProtocolMethod(
    L"Network.getAllCookies",
    L"{}",
    Callback<ICoreWebView2CallDevToolsProtocolMethodCompletedHandler>(
        [](HRESULT error, PCWSTR resultJson) -> HRESULT
        {
            // All the cookies are contained in a JSON string
            MessageBox(nullptr, resultJson, L"CDP protocol message Result", MB_OK);
            return S_OK;
        }).Get());

This method can be used for any of the protocols Chrome supports: https://chromedevtools.github.io/devtools-protocol/tot/

You can specifically use "Network.clearBrowserCookies", "Network.deleteCookies", "Network.getAllCookies", "Network.getCookies", "Network.setCookie", and "Network.setCookies" for cookie management.

MichaelKetting commented 4 years ago

Thank you @killerbobjr, will be trying this one!

MichaelKetting commented 4 years ago

@killerbobjr I was able to test with with the WinForms implemetnation and it worked as advertised. Thank you for the tip!

Follow-Up question: The developer tools can be disabled via policy and are then no longer available from within the hosted browser control as well. This blocks the cookie access via devleoper tools for corporate scenarios. I created issue #331 for this.

pipalot commented 4 years ago

@MichaelKetting Is there any chance you could post a code snippet please to show how you got it to work with WinForms? Many thanks.

MichaelKetting commented 4 years ago

@pipalot You call WebView2.CoreWebView2.CallDevToolsProtocolMethodAsync (see https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/dotnet/0-9-538/microsoft-web-webview2-core-corewebview2#calldevtoolsprotocolmethodasync), passing the method name and the json string with the cookie data. @killerbobjr already listed the operations (Network.getCookies, Network.setCookies) and the link to the details:

MichaelKetting commented 4 years ago

@killerbobjr I figure we should move the discussion of the cookie-workaround (https://github.com/MicrosoftEdge/WebViewFeedback/issues/331#issuecomment-657639095) back to here:

You're explaing that you use sqllite directly on the cookie file on the disk. Does this work for session-only cookies, i.e. cookies that get deleted when the browser is closed? I haven't tried this approach yet and figure, it's easier to ask and try it :)

killerbobjr commented 4 years ago

I didn't specifically test for HttpOnly session-only cookies, but based on the behavior of the "Cookie" database, I'd say that they weren't available.

My code is in C++ and is tied into my app with the JUCE library, so I'd have to separate it out into its own DLL for it to be accessible with WinForms. I'd rather wait to see if the Edge team officially addresses either issue #4 or #331 before releasing anything.

There's probably other ways of getting at HttpOnly session-only cookies if policies have turned off the developer tools: privilege escalation (via exploits) into the Edge processes, DLL injection, message interception with hooks, etc.

I think though it would be easier just to fetch the headers directly and read the cookies, then insert and overwrite in any requests.

In other words: hosted Edge makes a request, you copy that url and make the request yourself directly (with libcurl for instance, using the same user agent string as Edge), parse the cookies, then overwrite any future Edge session requests to that domain with that HttpOnly session-only cookie you fetched. A bit kludgy, but easier than trying to do something like privilege escalation.

pipalot commented 4 years ago

Thanks very much @MichaelKetting and @killerbobjr

MichaelKetting commented 4 years ago

@killerbobjr Yeah, anything akin to priviledge escalation certainly wouldn't be recommended. I played around with request intercepting back with the Internet Explorer control, wasn't much fun. I'm really hoping for something from Microsoft on this regard before it becomes easier to give up Internet Explorer than work around the restrictions of WebView2. There's also a libarary that host chromium directly (https://cefsharp.github.io/) but of course, there you have to keep the browser up to date yourself. Also not fun.

MichaelKetting commented 4 years ago

@champnic just gave us an update: https://github.com/MicrosoftEdge/WebViewFeedback/issues/331#issuecomment-659048982

A Cookie API is on the way :)

vmeganathan81 commented 4 years ago

Do we have the API now to get cookies from WebView2 ?

MichaelKetting commented 4 years ago

@vmeganathan81 We have a workable solution based on existing APIs (https://github.com/MicrosoftEdge/WebViewFeedback/issues/4#issuecomment-654603738) but not a dedicated API as part of WebView2.

vmeganathan81 commented 4 years ago

Any example how to implement using C#, and using WPF control Microsoft.Web.WebView2.Wpf. Or how do I proceed?

MichaelKetting commented 4 years ago

@vmeganathan81 You can simply translate the sample from @killerbobjr to c#, the APIs are the same, the rest is syntax. I don't have a C# sample at hand that I could share.

champnic commented 4 years ago

@vmeganathan81 Not yet, it's still on the way. I'll update this thread when it's available.

vmeganathan81 commented 4 years ago

@MichaelKetting, it worked thanks.

@champnic, it would be nice if we could get/set cookie API sooner.

vmeganathan81 commented 4 years ago

@MichaelKetting, can you direct me to right resource to look for, we are looking to intercept External Protocol Uri, and direct programmatically within Application, without having the user to click the alert message, and launching another executable of same type. Similar to https://github.com/MicrosoftEdge/WebViewFeedback/issues/137

MichaelKetting commented 4 years ago

@vmeganathan81 is the API: https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/dotnet/0-9-538/microsoft-web-webview2-core-corewebview2#calldevtoolsprotocolmethodasync

You can call it via this.WebView2.CoreWebView2.CallDevToolsProtocolMethodAsync(...)

The messagebox in the sample by @killerbobjr is just demo-code. The C++ and C# version use different semantics for getting the result.

studylamp commented 4 years ago

DevTools is not a long-term solution to this. Can't access to that be disabled?

MichaelKetting commented 4 years ago

@studylamp

Yes, this is a stop-gap until the official API is ready. Yes, there has been the problem about disabling DevToolsProtocol. It's been fixed: https://github.com/MicrosoftEdge/WebViewFeedback/issues/331#issuecomment-670083098

vmeganathan81 commented 4 years ago

@MichaelKetting, that part for getting cookies using this.WebView2.CoreWebView2.CallDevToolsProtocolMethodAsync(...) has been done, thanks again in addition also looking to intercept External URI call

Like mailto:// , want to intercept before the popup comes, and re-direct for appropriate application within the app. Since everything is within application.

MichaelKetting commented 4 years ago

@vmeganathan81 ah, okay, now I understand. That's a separate topic I can't contribute to. I would recommend starting a new topic to get the right answers since this topic is just about the cookies-api.

Symbai commented 4 years ago

@MichaelKetting

@pipalot You call WebView2.CoreWebView2.CallDevToolsProtocolMethodAsync (see https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/dotnet/0-9-538/microsoft-web-webview2-core-corewebview2#calldevtoolsprotocolmethodasync), passing the method name and the json string with the cookie data.

Can you post a code example please? I always get this error and have no idea whats the issue

webView.CoreWebView2.CallDevToolsProtocolMethodAsync("Network.setCookies",
    "[{\"url\":\"www.google.com\",\"name\":\"bla\",\"value\":\"123\"}]");

image

frankdekker commented 4 years ago

@Symbai I got it to work with with the singular Network.setCookie method. See example below:

var jsonData = new
{
    name = "MyCookie",
    value = "Foobar",
    domain = ".example.com",
    path = "/",
    secure = false,
    httpOnly = false,
    expires = 1631562538
};
var jsonString = new JavaScriptSerializer().Serialize(jsonData);

webView.CoreWebView2.CallDevToolsProtocolMethodAsync("Network.setCookie", jsonString);
Symbai commented 4 years ago

@frankdekker Thank you, it turned out that domain value is a MUST and NOT optional like the guide says. Wasted so much time on this, I'm glad you showed me your solution so I found that stupid issue.

champnic commented 4 years ago

CDP has a couple of quirks like that where one parameter is "optional", but only if one of the other "optional" parameters is given. You can open issues on CDP here through crbug: https://bugs.chromium.org/p/chromium/issues/entry?components=Platform%3EDevTools%3EPlatform

liminzhu commented 4 years ago

// cc @peiche-jessica

Hey folks. We're implementing cookie management APIs in WebView2 and have an API proposal up. If you have any feedback or thumbs-up, let us know in the proposal!

P.S. if you haven't already, you can watch the MicrosoftEdge/WebView2Announcements repo to subscribe to WV2 announcements, new API proposals, etc.

ShaunLoganOracle commented 3 years ago

@liminzhu Congrats on going GA for .NET/WinForms, etc! I am relying on the cookie management APIs for my WinForms project (eg. Microsoft.Web.WebView2.Core.CookieManager), and they don't seem to be available in 1.0.664.37. So I would like to know when these APIs will be GA? I have heard about a 6-week cadence, could you tell me if we can expect these APIs in the next release?

liminzhu commented 3 years ago

@ShaunLoganOracle thanks for the kind words. The cookie manager API is in the latest Pre-release package, the expectation is that it will land in Release package in the next release. We ship SDKs about the same cadence as Edge browser, and normally it's every 6 weeks, but for the next one it will be around 1/20 ish give and take a few days due to end of year and all that :).

prabodhmairh commented 3 years ago

+1 to the people waiting for these APIs to be GAed :)

@ShaunLoganOracle thanks for the kind words. The cookie manager API is in the latest Pre-release package, the expectation is that it will land in Release package in the next release. We ship SDKs about the same cadence as Edge browser, and normally it's every 6 weeks, but for the next one it will be around 1/20 ish give and take a few days due to end of year and all that :).

champnic commented 3 years ago

This is now available in our latest release package 1.0.705.50. Thanks! https://docs.microsoft.com/en-us/microsoft-edge/webview2/reference/win32/icorewebview2cookiemanager?view=webview2-1.0.705.50&preserve-view=true

markismail7 commented 3 years ago

I have click once application that uses webview2 and every time the user update the app, the application deletes the cookies. I checked the folder for webview2 and sure the cookies are deleted. How can i prevent that from happening during an update.

champnic commented 3 years ago

Hey @markismail7 - The cookies are tied to a User Data Folder. The default location for that folder is next to the exe, so likely as ClickOnce is updating it's deleting that folder and you lose cookies. To get around this, can you try specifying a different location for the User Data Folder when initializing the WebView2? More info: https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/user-data-folder

markismail7 commented 3 years ago

Hey @markismail7 - The cookies are tied to a User Data Folder. The default location for that folder is next to the exe, so likely as ClickOnce is updating it's deleting that folder and you lose cookies. To get around this, can you try specifying a different location for the User Data Folder when initializing the WebView2? More info: https://docs.microsoft.com/en-us/microsoft-edge/webview2/concepts/user-data-folder

Is there an example on how to set WebView2 folder ?

champnic commented 3 years ago

Assuming you are using .NET (the Win32 pattern is similar) you can set the user data folder when creating the environment: https://docs.microsoft.com/en-us/dotnet/api/microsoft.web.webview2.core.corewebview2environment.createasync?view=webview2-dotnet-1.0.864.35

And use this environment object to call WebView2.EnsureCoreWebView2Async(environment). Note that you should remove setting the Source property to initialize the control, because that will initialize the control with a default environment before you can call Ensure... yourself.

markismail7 commented 3 years ago

use this environment object to call WebView2.EnsureCoreWebView2Async(environment). Note that you should remove setting the Source property to initiali

I really appreciate the response here. I'm using VB.net. I tried this Private Async Sub InitializeBrowser(ByVal Optional url As String = Nothing) Dim userDataFolder = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) & "\VSMFolder\Web" Dim env = Await CoreWebView2Environment.CreateAsync(Nothing, userDataFolder) Await WebView.EnsureCoreWebView2Async(env) WebView.Source = New Uri(url) End Sub I called InitializeBrowser(url) on loading the form that contains webview2 .

It didn't do anything. I've been working on this for hours now. The only way around it for me, is to copy the folder before updating and paste it back after updating. I wish it was easier than that. I was looking into it over the web. Alot of people are struggling to do this.

shyjusachin commented 2 years ago

@champnic Hi here I can see that set cookies in WebVeiw2 is resolved. But here I am still failed to set the cookies through webresourcerequested event handler. Correct me if I made any mistakes in my code, using VB.net and Webview version is 1.0.1020.30

1.Initialized handler in CoreWebView2InitializationCompleted

Private Sub WebView21_CoreWebView2InitializationCompleted(sender As Object, e As Microsoft.Web.WebView2.Core.CoreWebView2InitializationCompletedEventArgs) Handles WebView21.CoreWebView2InitializationCompleted

        WebView21.CoreWebView2.AddWebResourceRequestedFilter("*", CoreWebView2WebResourceContext.All)
        AddHandler WebView21.CoreWebView2.WebResourceRequested, AddressOf CoreWebView2_WebResourceRequested
    End Sub

2.Setting headers in CoreWebView2_WebResourceRequested

Private Sub CoreWebView2_WebResourceRequested(sender As Object,
    e As CoreWebView2WebResourceRequestedEventArgs)
        e.Request.Headers.SetHeader("Cookie", "cea-token=12345")
        e.Request.Headers.SetHeader("Test", "cea-ssa=123")
End Sub

While loading CoreWebView2_WebResourceRequested is firing and these headers are setting there But once the page is loaded in WebVeiw2 control, seems like cookies are empty, I couldn't find any of these cookies. Please correct me if I a wrong in something or should I attempt some different approaches.

Thanks in advance.