search

mozilla / geckodriver

WebDriver for Firefox
https://firefox-source-docs.mozilla.org/testing/geckodriver/
Mozilla Public License 2.0
7.2k stars 1.52k forks source link

Login Prompt appears even when password is saved FF 56.02 Geckodriver 19.1 #1051

Closed ghost closed 7 years ago

ghost commented 7 years ago

System

Testcase

Firefox settings on about:config ff config 1 ff config 2

Open a Firefox browser on an internal (Intranet site) Launch from Visual Studio Enterprise 2015 as follows:

        //Options
        FirefoxOptions foptions = new FirefoxOptions();
        foptions.LogLevel = FirefoxDriverLogLevel.Trace;

        //Service
        FirefoxDriverService ffservice = FirefoxDriverService.CreateDefaultService(@"D:\Selenium\");
        ffservice.FirefoxBinaryPath = @"C:\Program Files\Mozilla Firefox\firefox.exe";

        //Driver
        IWebDriver ffdriver = new FirefoxDriver(ffservice, foptions, TimeSpan.FromSeconds(60));

        Console.WriteLine("Open a Browser in Firefox");
        ffdriver.Navigate().GoToUrl("http://something.net/");
        Thread.Sleep(3000);

Stacktrace

There is no available stack trace. No error is thrown in Visual Studio

Trace-level log

1510072939984 geckodriver INFO geckodriver 0.19.1 (0000000000000000000000000000000000000000 2017-10-31) 1510072939991 geckodriver INFO Listening on 127.0.0.1:53636 1510072941057 mozrunner::runner INFO Running command: "C:\Program Files\Mozilla Firefox\firefox.exe""-marionette" "-profile" "C:\Users\jee\AppData\Local\Temp\rust_mozprofile.CFF3NFSuC3dQ" 1510072943070 geckodriver::marionette TRACE connection attempt 0/600 1510072944170 geckodriver::marionette DEBUG Connected to Marionette on localhost:53640 1510072944172 geckodriver::marionette TRACE <- {"applicationType":"gecko","marionetteProtocol":3} 1510072944172 geckodriver::marionette TRACE -> 109:[0,1,"newSession",{"browserName":"firefox","capabilities":{"desiredCapabilities":{"browserName":"firefox"}}}] 1510072944223 geckodriver::marionette TRACE <- [1,1,null,{"sessionId":"e0921d77-66d7-4d2d-8d75-37bef33ba94f","capabilities":{"browserName":"firefox","browserVersion":"56.0.2","platformName":"windows_nt","platformVersion":"6.1","pageLoadStrategy":"normal","acceptInsecureCerts":false,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000},"rotatable":false,"specificationLevel":0,"moz:processID":13692,"moz:profile":"C:\Users\jee\AppData\Local\Temp\rust_mozprofile.CFF3NFSuC3dQ","moz:accessibilityChecks":false,"moz:headless":false}}] 1510072944223 webdriver::server DEBUG <- 200 OK {"value": {"sessionId":"e0921d77-66d7-4d2d-8d75-37bef33ba94f","capabilities":{"acceptInsecureCerts":false,"browserName":"firefox","browserVersion":"56.0.2","moz:accessibilityChecks":false,"moz:headless":false,"moz:processID":13692,"moz:profile":"C:\Users\jee\AppData\Local\Temp\rust_mozprofile.CFF3NFSuC3dQ","pageLoadStrategy":"normal","platformName":"windows_nt","platformVersion":"6.1","rotatable":false,"specificationLevel":0,"timeouts":{"implicit":0,"pageLoad":300000,"script":30000}}}} 1510072944251 webdriver::server DEBUG -> POST /session/e0921d77-66d7-4d2d-8d75-37bef33ba94f/url {"url":"http: //something .net/"} 1510072944253 geckodriver::marionette TRACE -> 52:[0,2,"get",{"url":"http://something .net"}] 1510072944381 geckodriver::marionette TRACE <- [1,2,null,{}] 1510072944382 webdriver::server DEBUG <- 200 OK {"value": {}}

andreastt commented 7 years ago

Could you please attach a trace-level log from geckodriver? Read more about reporting actionable bugs in our contribution guidelines.

ghost commented 7 years ago

I attached all the logging that it would generate. It doesn’t create a log file. I had to scrape it from the command window. Any suggestions would be appreciated.

RE

On Nov 7, 2017, at 5:45 PM, Andreas Tolfsen notifications@github.com<mailto:notifications@github.com> wrote:

Could you please attach a trace-level loghttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsearchfox.org%2Fmozilla-central%2Fsource%2Ftesting%2Fgeckodriver%2Fdoc%2FTraceLogs.md&data=01%7C01%7Celeven%40pitt.edu%7Cb0b3a5e01bbb49d98a2c08d52631409d%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=PLdJ8NdJ0DKESf9ZoEmBfsxso3AUjIfQUGR84fWVNz8%3D&reserved=0 from geckodriver? Read more about reporting actionable bugs in our contribution guidelineshttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsearchfox.org%2Fmozilla-central%2Fsource%2Ftesting%2Fgeckodriver%2FCONTRIBUTING.md%23reporting-bugs&data=01%7C01%7Celeven%40pitt.edu%7Cb0b3a5e01bbb49d98a2c08d52631409d%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=qI3ecm7cnZE57uS9ohbjMdqOYa%2B49k5k0%2BjyaTxepLM%3D&reserved=0.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmozilla%2Fgeckodriver%2Fissues%2F1051%23issuecomment-342649266&data=01%7C01%7Celeven%40pitt.edu%7Cb0b3a5e01bbb49d98a2c08d52631409d%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=EdCTJyjxrjg3m%2F9BpS7T2S6Qld%2BUBGkZtEJuqnZnPdU%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGQK3lp3WZHbh9HskFaVnkfIMdfhOiBVks5s0N2KgaJpZM4QVQgc&data=01%7C01%7Celeven%40pitt.edu%7Cb0b3a5e01bbb49d98a2c08d52631409d%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=1dJY3Z7djTJDOojVRGB51WYJYMgICojBI70rjo5rXFA%3D&reserved=0.

andreastt commented 7 years ago

Closing this bug because it is not actionable.

ghost commented 7 years ago

On Nov 8, 2017, at 1:16 AM, Andreas Tolfsen notifications@github.com<mailto:notifications@github.com> wrote:

Closing this bug because it is not actionable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmozilla%2Fgeckodriver%2Fissues%2F1051%23issuecomment-342721553&data=01%7C01%7Celeven%40pitt.edu%7C357a96e6eb34472e212108d526704b86%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=CfhVfHcFQ4YnaypNOWvxaM6fpNztXVugIk2XO0F6MMU%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGQK3pW2Wycij_HEhMN6RhFvLhwYfhyzks5s0UdPgaJpZM4QVQgc&data=01%7C01%7Celeven%40pitt.edu%7C357a96e6eb34472e212108d526704b86%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=CNBA9AYR9uJDeCp%2Fq5OP%2FdzFh2YXP1Lb6y1YE%2FI%2Bbfs%3D&reserved=0.

ghost commented 7 years ago

Please reopen.

The information I submitted is visible now. I forgot to remove the comments.

RW

On Nov 8, 2017, at 1:16 AM, Andreas Tolfsen notifications@github.com<mailto:notifications@github.com> wrote:

Closing this bug because it is not actionable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmozilla%2Fgeckodriver%2Fissues%2F1051%23issuecomment-342721553&data=01%7C01%7Celeven%40pitt.edu%7C357a96e6eb34472e212108d526704b86%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=CfhVfHcFQ4YnaypNOWvxaM6fpNztXVugIk2XO0F6MMU%3D&reserved=0, or mute the threadhttps://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAGQK3pW2Wycij_HEhMN6RhFvLhwYfhyzks5s0UdPgaJpZM4QVQgc&data=01%7C01%7Celeven%40pitt.edu%7C357a96e6eb34472e212108d526704b86%7C9ef9f489e0a04eeb87cc3a526112fd0d%7C1&sdata=CNBA9AYR9uJDeCp%2Fq5OP%2FdzFh2YXP1Lb6y1YE%2FI%2Bbfs%3D&reserved=0.

ghost commented 7 years ago

I apologize for flaming. I didn't realize that you could not see what I submitted because of the HTML comment.

andreastt commented 7 years ago

Still no reproducible steps here. See our contribution guide on how to write actionable bug reports.

ghost commented 7 years ago

It’s 3PM here I’ll add steps by 5PM

On Nov 8, 2017, at 2:25 PM, Andreas Tolfsen notifications@github.com<mailto:notifications@github.com> wrote:

Still no reproducible steps here. See our contribution guidehttps://dxr.mozilla.org/mozilla-central/source/testing/geckodriver/CONTRIBUTING.md?q=geckodriver%2FCONTRIBUTING.md&redirect_type=direct on how to write actionable bug reports.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/mozilla/geckodriver/issues/1051#issuecomment-342930887, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGQK3saT0akXBy0_7rJ9FjW9g-2SbTuVks5s0gAWgaJpZM4QVQgc.

whimboo commented 7 years ago

In the original comment I cannot see how you actually stored the username and password for accessing this internal site. Can you please shed some light on that?

ghost commented 7 years ago

Hebron,

It was windows running active directory.

I would enter the site in the Firefox about:config ntlm and auth etc.

So the very first time I would acces the internal site, I would enter login/pass by hand,

then it would be stored in Firefox,

then subsequent manual accesses of the internal site, the login prompt would not appear and I would be already authenticated.

When they updated Firefox it would ignore the stored user/pass and display the login prompt no matter what.

I did experiment with a hard coded user/pass in Selenium with a Switch to the alert and SendKeys of the user/pass and that worked but it’s highly insecure.

On Nov 8, 2017, at 3:14 PM, Henrik Skupin notifications@github.com<mailto:notifications@github.com> wrote:

In the original comment I cannot see how you actually stored the username and password for accessing this internal site. Can you please shed some light on that?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/mozilla/geckodriver/issues/1051#issuecomment-342945219, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGQK3mCqTUq-91ON9hEaCsP_4QMdqs5Iks5s0gungaJpZM4QVQgc.

andreastt commented 7 years ago

As I understand it, Send Alert Text is not going to work for HTTP Basic Auth, but it’s not clear to me if that’s the problem you’re trying to solve.

ghost commented 7 years ago

No I don’t want to send text to the alert, that was just an experiment.

The problem is once I’ve manually authenticated to the site and I run Selenium out of Visual studio I get hit with the login prompt and it never goes away, but if I open the site manually without Selenium the logging prompt is gone and Firefox just opens the site.

Setup: go to about:config and enter urls in ntlm, auth etc.

  1. Open internal site for first time manually after it’s clean or restored. -fires login prompt -login manually with user/pass -values accepted -site is displayed

  2. Reopen site manually -opens directory to site with stored authentication -site is displayed

  3. Run Selenium to open the very same site Navigate GoToURL etc. -fires login prompt every time

  4. To compare Run external site like Bing -runs w/o problems

(Other users reported issues with the Firefox login prompt since update 55 outside of the geckodriver context.)

I was running Firefox all year with Selenium and geckodriver until I moved past FF 47.01

Much Thanks!

On Nov 8, 2017, at 3:32 PM, Andreas Tolfsen notifications@github.com<mailto:notifications@github.com> wrote:

As I understand it, Send Alert Text is not going to work for HTTP Basic Auth, but it’s not clear to me if that’s the problem you’re trying to solve.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/mozilla/geckodriver/issues/1051#issuecomment-342950039, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGQK3nyG3phJkehelFyVvTkji-k-uuNyks5s0g_tgaJpZM4QVQgc.

whimboo commented 7 years ago

@RoakyWood you say that this regressed after an update of Firefox. In that case I would suggest you the following. Please try your test with the latest Firefox 52.0.x ESR release. Does it work, or not? If yes, then please check releases 53, and 54. If not, then check with releases 48, 49, 50, and 51. It would be good to know for which release it actually regressed. Thanks.

ghost commented 7 years ago

Henrik,

I will do that Monday and I will let you know.

Thanks

On Nov 12, 2017, at 4:16 AM, Henrik Skupin notifications@github.com<mailto:notifications@github.com> wrote:

@RoakyWoodhttps://github.com/roakywood you say that this regressed after an update of Firefox. In that case I would suggest you the following. Please try your test with the latest Firefox 52.0.x ESR release. Does it work, or not? If yes, then please check releases 53, and 54. If not, then check with releases 48, 49, 50, and 51. It would be good to know for which release it actually regressed. Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/mozilla/geckodriver/issues/1051#issuecomment-343723555, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGQK3sv_kGlXqnEpFO_UyNe77ipA2Nclks5s1rdLgaJpZM4QVQgc.

ghost commented 7 years ago

Henrik,

I’ve had no luck with any combination.

It seems that Firefox will just not store the login for the site, and as of recent release there is no entry point anywhere to allow you to manually save a login.

Thx

On Nov 12, 2017, at 4:16 AM, Henrik Skupin notifications@github.com<mailto:notifications@github.com> wrote:

@RoakyWoodhttps://github.com/roakywood you say that this regressed after an update of Firefox. In that case I would suggest you the following. Please try your test with the latest Firefox 52.0.x ESR release. Does it work, or not? If yes, then please check releases 53, and 54. If not, then check with releases 48, 49, 50, and 51. It would be good to know for which release it actually regressed. Thanks.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/mozilla/geckodriver/issues/1051#issuecomment-343723555, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AGQK3sv_kGlXqnEpFO_UyNe77ipA2Nclks5s1rdLgaJpZM4QVQgc.

whimboo commented 7 years ago

Which versions of Firefox did you test?

ghost commented 7 years ago

Other information:

I'm using namespace: using OpenQA.Selenium.Firefox. The Web driver is the Nuget Package Selenium.Webdriver 3.7.0 by Selenium Committers.

I thought it could have been McAfee AV, but IE11, Opera, Vivaldi and Chrome have no login prompt issue.

None of the combinations were successful.

Result of Test Firefox Date Geckodriver Date Result 51 1/23/2017 0.15 3/8/2017 Prompt 52.0.1 3/17/2017 0.15 3/8/2017 Prompt 53 4/18/2017 0.16 4/21/2017 Prompt 53 4/18/2017 0.17 6/9/2017 Prompt 54 6/13/2017 0.18 7/11/2017 Prompt 55 8/7/2017 0.18 7/11/2017 Prompt 56.0.2 10/25/2017 0.19.1 11/1/2017 Prompt 57.0b14 11/3/2017 0.19.1 11/1/2017 Prompt 58.0b4 11/15/2017 0.19.1 11/1/2017 Prompt

whimboo commented 7 years ago

Interesting. And it sounds somewhat similar as what we have recently seen for proxy authentication. For those the auth dialog also pops-up for the very first access.

@RoakyWood can you please check if you can manually replicate the problem on your side? Does it happen for any site which is behind a HTTP Basic Auth? Maybe you can test it with http://www.mozqa.com/data/mozqa.com/http_auth/basic/.

ghost commented 7 years ago

W

When opening from Selenium, after manual entry and save of the user/pass, it still won't auto-fill nor authenticate

Other info

The password does not show as stored, but it does when manually entered

manual login selenium login

RW

cgoldberg commented 7 years ago

go to about:config and enter urls in ntlm, auth etc.

is this saved to your user profile? because FF uses a new profile when started via selenium/geckodriver... Any credentials you have supplied will not be seen AFAIK.

ghost commented 7 years ago

Goldberg,

That makes sense. I had a feeling it got a different signal from Selenium than it did from manual (But prior to about FF 52 I never had to use profile objects before, it just authenticated like in Chrome/IE/Edge/Vivaldi/Opera so it's a brave new world for me)

In about:profiles, it only shows a default profile folder. (Could it be that Selenium deletes the temporary profile after each run?)

I'm looking for the Profiles API now.

I was able to print out the existing profiles, but there is only 'default' as I expected.

So would you recommend...

  1. I create a Profile Manager object (or a Profile Object?)
  2. I want to get the profile where I saved the password manually and use that one or 2 (alt). I want to set profile preferences that authenticate via AD without the login prompt?

RW

ghost commented 7 years ago

Goldberg,

Your insight was solid. This is not really a bug but maybe a second-order effect of more stringent Firefox security (I think?!?)

I found a workaround that is less intrusive than hard-coding user pass or putting user/pass into the URL, which ends up in your dll/exe, e.g.

You create a Selenium Profile. Then manually enter the user pass when the browser opens.

Once the login is stored in Firefox, you select the profile in Selenium so it doesn't create a blank temp profile and it auto-fills the password.

My preference would be that it worked like it used to prior to FF52? where you run Selenium after adding the site to ntlm etc in about:config and it just runs.

In C#:

    //Get a list of profiles, load the desired profile
    FirefoxProfileManager FPM = new FirefoxProfileManager();
    var jeh = FPM.GetProfile("SeleniumProfile");

    //Print a list of existing profiles for reference            
    foreach (String jim in FPM.ExistingProfiles)
    {
        Console.WriteLine("Existing Profiles are: {0}", jim);
    }

    //Add Options
    FirefoxOptions foptions = new FirefoxOptions();
    foptions.LogLevel = FirefoxDriverLogLevel.Trace;
    foptions.Profile = jeh;

    //Create a Service
    FirefoxDriverService ffservice = FirefoxDriverService.CreateDefaultService(@"D:\Selenium\");
    ffservice.FirefoxBinaryPath = @"C:\Program Files (x86)\Mozilla Firefox\firefox.exe";

    //Create a Driver
    IWebDriver ffdriver = new FirefoxDriver(ffservice, foptions, TimeSpan.FromSeconds(60));

    //Open the site
    Console.WriteLine("Open a Browser in Firefox");
    ffdriver.Navigate().GoToUrl("http://mylocalinternalsite.net/");

    //Sleep so you can watch it real-time
    Thread.Sleep(2000);
    checkAlert(ffdriver); //(Little Utility to SwitchTo then accept the Prompt)
    Thread.Sleep(4000);

Thanks! Keep me posted please.

whimboo commented 7 years ago

@RoakyWood, so what exactly would have to be set? Only the preferences as pointed out in your initial comment? If yes, do not mess around with additional profiles when running your test with Selenium and geckodriver. Instead use FirefoxOptions to specify those extra preferences.

FirefoxOptions options = new FirefoxOptions();

See https://seleniumhq.github.io/selenium/docs/api/dotnet/html/T_OpenQA_Selenium_Firefox_FirefoxOptions.htm for details in how to use SetPreference().

Does that fix your issue? Sorry, but I'm not that familiar with that NTLM parts.

ghost commented 7 years ago

@whimboo Thanks, so I see it's easy to just set a preference from the about:config list. I will experiment with it. My issue is fixed, I will say that whatever change Firefox made definitely requires more workarounds than before by forcing the login prompt.

whimboo commented 6 years ago

@RoakyWood how does it work? Were you able to get rid of the failure?

ghost commented 6 years ago

@whimboo TLDR: It seems that the driver uses a new Firefox Profile every time. As a result, it never gets to any logins saved by Firefox. Thus, I have to re-direct it to an existing profile.

It did not seem to used to work this way though. Entering a URI in ntlm/auth preferences used to suffice. Now it does not.

I think it was some change in Firefox or the Driver, but I could not pinpoint the combination where this happened. It was not a change in anti-virus as a different machine with a different anti-virus has the same issue.

whimboo commented 6 years ago

@RoakyWood I have good news for you. Yesterday I was investigating issue #1058, and there is indeed a way to use an existing profile by its path. So please check that other issue in how to correctly setup the test to make use of it. You will also need the workaround for the Marionette port until this fix has been released.

Let me know if that fixes your issue.