Hammerhead `location.toString` is not a function

Jgrabenbauer commented 2 years ago

What is your Scenario?

When submitting a form to load https://dev.payments.ai/dashboard the page doesn't load (only in automation) and throws a location.toString is not a function error in Hammerhead.

What is the Current behavior?

A location.toString is not a function error is thrown in the console and the page doesn't load.

What is the Expected behavior?

No errors are thrown and the correct page loads, just like during manual testing.

What is your public website URL? (or attach your complete example)

Will send an email.

What is your TestCafe test code?

test
    .meta({ created_on: '07/22/22', created_by: 'Justin Braden', updated_on: '', updated_by: '' })
    .meta({ scope: 'sanity', priority: 'high', showstopper: 'no', github_ID: '1088' })
    ('user_can_navigate_to_payments_ai_dashboard', async t => {

        // Purpose: To verify the Payments AI dashboard page displays when navigated to.

        const url = ClientFunction(() => document.location.href)

        await WorkspaceMainNavPageObjects.navigate_to_payments_ai()

        await t
            .expect(url()).contains(data.pages.payments_ai.dashboard_url)
            .expect(PaymentsAIPageObjects.dropdown_account_menu.visible).ok()
})

Your complete configuration file

require('dotenv').config()
const CustomRequestHooks = require('./pom/custom_request_hooks')

// Agnostic for all environments
let skipped_tests = [

    // Located in apps_page_tests.js
    'user_cannot_access_sites_when_feature_is_disabled', // 07-28-22
    'user_cannot_access_settings_when_feature_is_disabled', // 07-28-22
    'user_cannot_access_analytics_when_feature_is_disabled', // 09-16-22
    'user_cannot_access_marketing_when_feature_is_disabled', // 09-16-22
    'user_cannot_access_products_when_feature_is_disabled', // 09-16-22
    'user_cannot_access_contacts_when_feature_is_disabled', // 09-16-22
    'user_cannot_access_orders_when_feature_is_disabled', // 09-16-22
    'user_cannot_access_sales_when_feature_is_disabled', // 09-16-22

    // Located in billing_page_tests.js
    'user_can_navigate_to_billing_page', // 08-22-22

    // Located in blog_page_tests.js
    'user_can_edit_blog_settings', // 07-06-22

    // Located in clean_up_scripts.js
    'clean_up_copies_of_funnels',
    'clean_up_copies_of_funnel_pages',
    'clean_up_copies_of_pages', 
    'clean_up_products',
    'clean_up_site_pages',
    'clean_up_workflows',
    'dismiss_notifications',
    'clean_up_collaborators',

    // Located in courses_page_tests.js
    'user_can_update_courses_lesson_path',
    'user_can_update_courses_module_path',
    'user_can_log_into_customer_center_with_magic_link',
    'courses_back_button_navigates_to_previous_page',

    // Located in funnels_page_tests.js
    'user_can_create_a_new_funnel', // 08-04-22
    'user_can_clone_a_funnel',
    'user_can_navigate_through_funnel',
    'user_can_share_funnel_from_different_workspace', // 09-23-22
    'page_views_and_unique_views_values_are_correct', // 05-31-22

    // Located in funnel_workflow_builder_page_tests.js
    'user_is_able_to_remove_a_node', // 05-31-22
    'user_can_add_and_remove_a_decision_path', // 05-31-22
    'user_is_able_to_add_and_remove_a_product_from_funnel', //08-04-22
    'user_can_add_and_remove_a_split_test', // 05-31-22
    'user_can_add_a_new_workflow', // 08-04-22
    'user_can_add_an_existing_workflow', // 08-04-22

    // Located in marketing/broadcasts_page_tests.js
    'user_can_create_new_broadcast', // 06-20-22

    // Located in marketing/templates_page_tests.js
    'user_can_create_broadcast_template', // 06-13-22

    // Located in orders/order_page_tests.js
    'user_can_partially_fulfill_order', // 08/11/22

    // Located in pages_page_tests.js
    'user_can_filter_page_types_with_tabs', // 05/27/22
    'user_can_pin_a_page_from_edit_page', //07-29-22

    // Located in pipelines_page_tests.js
    'user_can_navigate_to_pipelines_page', //09/14/22

    // Located in reporting_page_tests.js
    'opt-ins_by_landing_page_report_displays', // 06/21/22

    // Located in sites_overview_page_tests.js
    'user_can_add_header_to_site', // 07-06-22

    // Located in themes_page_tests.js
    'user_can_edit_theme_name', // 08-08-22

    // Located in top_main_nav_page_tests.js
    'clicking_notifications_will_open_notifications_alert', // 06/01/22

    // Located in workflows_page_tests.js
    'user_can_create_a_workflow' // 08-23-22
]

// Production specific scripts needing to be skipped for various reasons
let production_skipped_tests = [
    'user_can_navigate_to_order_tags_page', // Link disabled through Launch Darkly as of 08/29/22
    'user_can_navigate_to_payments_ai_dashboard', // Production PAI account not set up as of 08/29/22
    'user_is_able_to_navigate_to_the_inbox_page' // Inbox link is not activated in production as of 08/31/22
]

// Scripts that need to run with a single concurrency
let isolated_tests = [
    // Located in sites_overview_page_tests.js
    'user_can_make_site_public',
    'user_can_edit_site_name',
    'user_can_make_site_password_protected',
    'user_can_make_site_private',
    'user_can_edit_site_domain',
    'changing_site_domain_updates_funnel_urls',
    'changing_site_domain_updates_blog_urls',
    'changing_site_domain_updates_course_urls',
    'user_can_add_header_to_site',

    // Located in account_details_page_tests.js
    'user_can_edit_account_name',
    'user_can_edit_account_password',
    'user_can_change_account_language',

    // Located in general_page_tests.js
    'user_can_edit_workspace_details',

    // Located in blog_page_tests.js
    'user_can_edit_blog_settings',

    // Located in themes_page_tests.js
    'user_can_promote_theme_to_live_from_index_page',
    'user_can_promote_theme_to_live_from_overview_page'
]

// Scripts specific for Applitools integration
let visual_tests = [
    // Located in dashboard_checks.js
    'applitools_check_on_dashboard',

    // Located in editor_checks.js
    'applitools_check_in_editor',
    'user_is_able_to_change_the_checkout_element_style',

    // Located in login_checks.js
    'applitools_check_on_login_page_with_magic_link',
    'applitools_check_on_login_page_with_password'
]

// Filter that joins the skipped scripts with the production skipped tests
let production_filter = skipped_tests.concat(production_skipped_tests)

// Filter that joins ALL filtered scripts so that they are not run for a full suite
let full_filter = skipped_tests.concat(isolated_tests).concat(visual_tests)

let build_filter = () => {

    // Purpose: To set the suite that we want to run. (isolated, regression, sanity)

    var filter = {}

    switch(process.env.ENV) {
        case 'production':
            if(process.env.SCOPE) {
                filter = {
                    testGrep: `^(?!.*(${production_filter.join('|')})).*$`,
                    testMeta: {
                        scope: process.env.SCOPE
                    }
                }
            }
            break
        case 'staging':
            if(process.env.SCOPE) {
                filter = {
                    testGrep: `^(?!.*(${skipped_tests.join('|')})).*$`,
                    testMeta: {
                        scope: process.env.SCOPE
                    }
                }
            } else {
                filter = {
                    testGrep: `^(?!.*(${full_filter.join('|')})).*$`
                }
            }
            break
    }

    return filter
}

let determine_concurrency = () => {

    // Purpose: To set concurrency dependant on the scope that is passed

    let concurrency = 5
    if (process.env.SCOPE == 'isolated') {
        concurrency = 1
    } 
    return concurrency
}

module.exports = {
    assertionTimeout: 5000,
    browsers: [
        "chrome:headless --window-size=1920,1159 --allow-insecure-localhost"
    ],
    concurrency: determine_concurrency(),
    cache: true,
    clientScripts: [
        "scripts/hide_notifications.js", 
        "scripts/click_expand_all_for_courses_details_page.js"
    ],
    disableMultipleWindows: true,
    filter: build_filter(),
    hooks: {
        request: CustomRequestHooks
    },
    hostname: "localhost",
    pageLoadTimeout: 30000,
    quarantineMode: {
        successThreshold: 1,
        attemptLimit: 2
    },
    reporter: [
        {
            name: "spec",
            output: "artifacts/reports/spec_results"
        },
        {
            name: "xunit",
            output: "artifacts/reports/xunit_results.xml"
        }, 
        {
            name: "testrail"
        }
    ],
    runExecutionTimeout: 1800000,
    screenshots: {
        path: "artifacts/screenshots",
        pathPattern: "${TEST}_${DATE}_${TIME}.png",
        takeOnFails: true,
        thumbnails: false
    },
    selectorTimeout: 5000,
    skipJsErrors: true,
    skipUncaughtErrors: true,
    src: "tests/**",
    ssl: {
        pfx: "ssl/testingdomain.pfx",
        rejectUnauthorized: true
    },
    testExecutionTimeout: 240000
}

Your complete test report

Running tests in:

Chrome 105.0.0.0 / Monterey 12

payments ai ✖ user_can_navigate_to_payments_ai_dashboard (screenshots: /Users/justingrabenbauer/cf2-test-cafe-poc/artifacts/screenshots/user_can_navigate_to_payments_ai_dashboard_2022-09-23_12-23-36.png)

1) AssertionError: expected false to be truthy

Browser: Chrome 105.0.0.0 / macOS 10.15.7 Screenshot: /Users/justingrabenbauer/cf2-test-cafe-poc/artifacts/screenshots/user_can_navigate_to_payments_ai_dashboard_2022-09-23_12-23-36.png
```
 25 |
 26 |        // await HelperObjects.wait_for_element(PaymentsAIPageObjects.dropdown_account_menu, 'visible', false)
 27 |
 28 |        await t
 29 |            .expect(url()).contains(data.pages.payments_ai.dashboard_url)
```
30 | .expect(PaymentsAIPageObjects.dropdown_account_menu.visible).ok() 31 |}) 32 |
```
 at <anonymous> (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/payments_ai/payments_ai_tests.js:30:74)
 at asyncGeneratorStep (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/payments_ai/payments_ai_tests.js:6:315)
 at _next (/Users/justingrabenbauer/cf2-test-cafe-poc/tests/payments_ai/payments_ai_tests.js:6:653)
```
1/1 failed (23s)

Warnings (1):

The "browsers", "concurrency", and "filter" options from the configuration file will be ignored.

Screenshots

https://www.loom.com/share/c40e2447a442432f80c0eae20ade42b5

https://www.loom.com/share/22b7ffc8501144e2b66e9a28e9c71ac5

Steps to Reproduce

Run the script:

ENV=staging testcafe "chrome '--window-size=1920,1159'" -t 'user_can_navigate_to_payments_ai_dashboard' -c1

TestCafe version

2.0.0

Node.js version

16.4.0

Command-line arguments

ENV=staging testcafe "chrome '--window-size=1920,1159'" -t 'user_can_navigate_to_payments_ai_dashboard' -c1

Browser name(s) and version(s)

Chrome 105.0.0.0

Platform(s) and version(s)

Monterey 12.5

Other

No response

miherlosev commented 2 years ago

Hi @Jgrabenbauer,

Could you please share an example so that I can run it locally without any additional setup? Please refer to the following help topic for more information: How To: Create a Minimal Working Example When You Submit an Issue.

You can send login and password directly to support@devexress.com.

Jgrabenbauer commented 2 years ago

Hey @miherlosev, @AlexSkorkin had already responded back to me via email. Can you guys communicate and let me know if you need anything else? Please feel free to respond via email.

miherlosev commented 2 years ago

Hi @Jgrabenbauer

Thank you for the shared example. I've reproduced the issue.

Jgrabenbauer commented 2 years ago

Hey Mikhail,

Thanks for the update! I’m glad you were able to reproduce it.

Best Regards,

Justin Grabenbauer, Jr. SDET

e. @.*** m. (508) 470-0271

On Sep 29, 2022 at 2:26 AM -0500, Mikhail Losev @.***>, wrote:

Hi @Jgrabenbauer Thank you for the shared example. I've reproduced the issue. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

miherlosev commented 1 year ago

Hi @Jgrabenbauer,

This issue is not reproduced with combination of testcafe@3.0.1 and the Google Chrome browser. Feel free to reopen this issue if you encounter it in other browsers.

DevExpress / testcafe