search

getgauge / taiko

A node.js library for testing modern web applications
https://taiko.dev
MIT License
3.59k stars 453 forks source link

The react product grid not loading when opened the url from taiko browser command #1822

Closed nseshadr closed 3 years ago

nseshadr commented 3 years ago

When i try to load the below urls, the product grid (built using react) is not loading. This happens only when i try loading using taiko goto command. The page loads fine when opened on a chromium or chrome browser normally.

I tried this on both Chromium as well as Chrome. The results are same.

Steps to reproduce On taiko prompt

openBrowser() Goto ("https://www.clinique.com/products/1687/skincare/moisturizers")

One other example: https://www.esteelauder.com/products/631/product-catalog/makeup

page load issue

zabil commented 3 years ago

The site might be blocking bots as on the console I see the following errors.

Failed to load resource: the server responded with a status of 422 ()
L65E3-UQP2G-3R7KX-QSCPM-DHVXY:10 Error: 422
    at e.<anonymous> (bundle.js:1)
    at u (runtime.js:45)
    at Generator._invoke (runtime.js:281)
    at Generator.forEach.e.<computed> [as next] (runtime.js:106)
    at a (bundle.js:1)

Please try specifying a path to the Chrome executable using the enviroment variable "TAIKO_BROWSER_PATH"

nseshadr commented 3 years ago

INTERNAL

Hi,

Thanks for looking in to the issue. I did set the suggested environment path during the initial set up itself. I guess it is not helping.

Any other work around?

Get Outlook for iOShttps://aka.ms/o0ukef

From: Zabil Cheriya Maliackal notifications@github.com Sent: Wednesday, January 27, 2021 4:27 AM To: getgauge/taiko Cc: Seshadri, Narasimharaghavan; Author Subject: Re: [getgauge/taiko] The react product grid not loading when opened the url from taiko browser command (#1822)

The site might be blocking bots as on the console I see the following errors.

Failed to load resource: the server responded with a status of 422 () L65E3-UQP2G-3R7KX-QSCPM-DHVXY:10 Error: 422 at e. (bundle.js:1) at u (runtime.js:45) at Generator._invoke (runtime.js:281) at Generator.forEach.e. [as next] (runtime.js:106) at a (bundle.js:1)

Please try specifying a path to the Chrome executable using the enviroment variable "TAIKO_BROWSER_PATH"https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.taiko.dev_configuring-5Ftaiko_-23using-2Denvironment-2Dvariables&d=DwMCaQ&c=nueuSc8JOP1GsO0MVsUMDM-mW75SKO7qym0cgi0kWaQ&r=OUJk8ehqqygMGvfGG3d-lziJClxkeeRqaEL3-0bHJW8&m=S5tVAnW60zn36ABM3aQJRlug5gIJQg1MraXXHa1FlgY&s=DbvINCFeO3Oo7JU71vVkpXP3OxsODG9h2J_SszU-lug&e=

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_getgauge_taiko_issues_1822-23issuecomment-2D768154305&d=DwMCaQ&c=nueuSc8JOP1GsO0MVsUMDM-mW75SKO7qym0cgi0kWaQ&r=OUJk8ehqqygMGvfGG3d-lziJClxkeeRqaEL3-0bHJW8&m=S5tVAnW60zn36ABM3aQJRlug5gIJQg1MraXXHa1FlgY&s=rI5AVwuXUUkSs3lUOM7ux_FjJ90kJDuPHo-XjdF4kCo&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_ARJJU5VHICKP3O2VSVKSS63S37L7RANCNFSM4WUJTGMA&d=DwMCaQ&c=nueuSc8JOP1GsO0MVsUMDM-mW75SKO7qym0cgi0kWaQ&r=OUJk8ehqqygMGvfGG3d-lziJClxkeeRqaEL3-0bHJW8&m=S5tVAnW60zn36ABM3aQJRlug5gIJQg1MraXXHa1FlgY&s=CZopmOHWlP0WNnv5pF__trX7fhDSEQ4-fuCZDQ9kASU&e=.

THIS E-MAIL IS INTENDED ONLY FOR THE ADDRESSEE(S) AND MAY CONTAIN CONFIDENTIAL INFORMATION. IF YOU ARE NOT THE INTENDED RECIPIENT, YOU ARE HEREBY NOTIFIED THAT ANY USE OF THIS INFORMATION OR DISSEMINATION, DISTRIBUTION OR COPYING OF THIS E-MAIL IS STRICTLY PROHIBITED. IF YOU HAVE RECEIVED THIS E-MAIL IN ERROR, PLEASE NOTIFY THE SENDER IMMEDIATELY BY RETURN E-MAIL AND DELETE THE ORIGINAL MESSAGE. THANK YOU.

nseshadr commented 3 years ago

Hi,

Following up to see any solution for this issue.

NivedhaSenthil commented 3 years ago

Can you please let us know if there is any check made using user-agent or something to restrict access for those particular URLs which are throwing 422?

nseshadr commented 3 years ago

I dont think there is any restrictions. If you look at those urls, only the product grid is not loading (those are react components). The rest of the page is loading fine. Also, i dont see this issue when using other automation tools.

zabil commented 3 years ago

Also, i dont see this issue when using other automation tools.

Is it possible to share small test script using the tool you that worked on the page? This will help with debugging the issue.

nseshadr commented 3 years ago

You can try using selenium ide that comes with Chrome / FF as plugin.

nseshadr commented 3 years ago

Attached raw request from client if that helps log.txt

sriv commented 3 years ago

I suspect this is because of Taiko potentially not passing/setting the CSP headers. See this discussion: https://github.com/getgauge/taiko/discussions/1745

nseshadr commented 3 years ago

is there an ETA for this solution? This is critical for us to use Taiko.

NivedhaSenthil commented 3 years ago

This issue may not be related to CSP header.. I did try bypassing them that did not help.

nseshadr commented 3 years ago

Any update?

sriv commented 3 years ago

I believe this has something to do with the Chromium flags/preferences set by taiko, and the site behaves differently because of one or more of these settings.

I am yet to investigate which preference/flag causes the site to behave differently.

nseshadr commented 3 years ago

Yes, launching outside of taiko works. But we are initiating the tests from Gauge which uses the browser set by taiko. How do we achieve this work around there?

NivedhaSenthil commented 3 years ago

As a workaround you could try launching the browser outside of taiko with debug port .local-chromium/mac-823078/chrome-mac/Chromium.app/Contents/MacOS/Chromium --remote-debugging-port=9222, and use openBrowser({host:'localhost',port:'9222'}) to connect to it from Taiko.

Or try connecting to other browsers by providing TAIKO_BROWSER_PATH

nseshadr commented 3 years ago

I have tried both on chromium and chrome. Are you suggesting other browser like Firefox?

also, we are running the test from gauge. How do we achieve the work around from gauge?

sriv commented 3 years ago

ok, so I found the cause. Taiko launches chromium with --disable-web-security flag, which tells chromium to not enforce same-origin policy. This helps when testing apps in localhost etc. Also it was introduced because otherwise some people would get SSL certificate errors in their dev instances.

In this case, this flag is actually causing the webpage to not be able to fetch the product list.

I believe this will require a code change, since there isn't a way to override the args in taiko.

nseshadr commented 3 years ago

Thanks for the investigation. When can I expect a fix for this issue?

zabil commented 3 years ago

@nseshadr it's tough to estimate an exact timeline as any change needs testing.

You can help speed this up by testing the fix. Patch your local installation of Taiko by removing a line (Search for the text --disable-web-security) from the file node_modules/taiko/browser_launcher.js. Re-run your tests after this change.

nseshadr commented 3 years ago

I tried removing --disable-web-security from node_modules/taiko/lib and it worked for Chromium. But when we have the Chrome browser on the path, it doesn't work. Is there a similar workaround to make it work for Chrome or FF?

sriv commented 3 years ago

It works for me with Chromium and Google Chrome. FF does not have this problem since this flag is not set in firefox.

It is a simple change to make, in theory. But as @zabil mentioned the effort for testing this change is unknown. The flag was introduced for a reason.

The best options I can think of now is to

The second option seems like an inconvenience, but it is simpler. @NivedhaSenthil @zabil thoughts?

zabil commented 3 years ago

I remember adding this as a part of https://github.com/getgauge/taiko/commit/3f842f63e0ad877861c2ff7c8463cb2906a31198

In a test environment enabling cors might make existing test scripts fail. I think there should be a option to remove/disable this or other browser start options.

zabil commented 3 years ago

It looks like setting --disable-web-security will cause bad side effects.

I've raised the pull request #1950 to remove it. It might be easier to pass this as an option and make the disabling security explicit when scripts are run on a test environment.

nseshadr commented 3 years ago

Thanks. It works on my end after removing --disable-web-security.