SeleniumHQ / selenium

A browser automation framework and ecosystem.
https://selenium.dev
Apache License 2.0
30.54k stars 8.16k forks source link

Selenium return the same html #6221

Closed serdam55 closed 6 years ago

serdam55 commented 6 years ago

I'm using Selenium for get the html of a website, I created a class that implement this logic:

public class NetworkHelper
{
    private ChromeDriver driver;

    /// <summary>
    /// Constructor.
    /// </summary>
    /// <param name="driver">Instantiate the driver.</param>
    public NetworkHelper(ChromeDriver driver)
    {
        this.driver = driver;
    }

    /// <summary>
    /// Get the html of the website.
    /// </summary>
    /// <param name="url">Link of the site.</param>
    /// <returns>Get a string which contains the html of the site.</returns>
    public string GetHtml(Uri url)
    {
        driver.Navigate().GoToUrl(url);
        return driver.PageSource;
    }
}

and then in the main:

public static async Task Main(string[] args)
{
    //Configure the option for headless browser.
    var chromeOpts = new ChromeOptions();
    chromeOpts.AddArguments("headless");

    var chromeDriverService = ChromeDriverService.CreateDefaultService();
    ChromeDriver driver = new ChromeDriver(chromeDriverService, chromeOpts);
    var networkHelper = new SWP.Helpers.NetworkHelper(driver);

    //Execute the requests:
    string html = networkHelper.GetHtml(new Uri("http://www.oddsportal.com/soccer/usa/mls/chicago-fire-new-york-city-CAl0LCJs/#1X2;2"));

    html = networkHelper.GetHtml(new Uri("http://www.oddsportal.com/soccer/usa/mls/chicago-fire-new-york-city-CAl0LCJs/#1X2;3"));
}

now the first request contains the correct html but the second request (which have an url that end with: ;3) return the same html of the first request. A strange thing is that: if I revert the link I get the correct html for the first request (which in the example is the second), and an incorrect html for the second request (which is the first in the example) What I did wrong?

barancev commented 6 years ago

What exactly is wrong? I tried to change 2 to 3 in the browser addressbar (manually) and press Enter and I don't see any changes on the page too.

serdam55 commented 6 years ago

@barancev Hi, thanks for the reply. Are you sure that doesn't change nothing? Because when you type this url, you'll get:

1

and for the other link:

2

As you can see the rows inside the table change

barancev commented 6 years ago

Yes, I'm sure, and here is a proof: https://www.screencast.com/t/dDyrM7P1u

You need a pause between page loads to actually update the data. Or a new session.

barancev commented 6 years ago

Or, even better, use Selenium to click elements on the page instead of GoToUrl.

serdam55 commented 6 years ago

@barancev For the video, I would say that I've had the same problem, I had to press enter more than once (don't know if is a Chrome problem), anyway, I'll try the click element, solution, there is a tutorial about that? Thanks

barancev commented 6 years ago

Please send questions to the selenium user group