search

seleniumbase / SeleniumBase

📊 Python's all-in-one framework for web crawling, scraping, testing, and reporting. Supports pytest. UC Mode provides stealth. Includes many tools.
https://seleniumbase.io
MIT License
5.07k stars 946 forks source link

4.14.14: pytest is failing in many units #1899

Closed wuch-g2v closed 1 year ago

wuch-g2v commented 1 year ago

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

Here is pytest summary output:

```console + SETUPTOOLS_SCM_PRETEND_VERSION=4.14.14 + PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-seleniumbase-4.14.14-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-seleniumbase-4.14.14-2.fc 35.x86_64/usr/lib/python3.8/site-packages + /usr/bin/pytest -ra -m 'not network' --browser=chrome --headless ============================= test session starts ============================== platform linux -- Python 3.8.16, pytest-7.3.1, pluggy-1.0.0 rootdir: /home/tkloczko/rpmbuild/BUILD/SeleniumBase-4.14.14 configfile: pytest.ini plugins: seleniumbase-4.14.14, metadata-2.0.4, html-3.2.0, forked-1.6.0, ordering-0.6 collected 148 items examples/basic_test.py Warning: chromedriver not found. Getting it now: *** chromedriver version for download = 114.0.5735.90 (Latest) Downloading chromedriver_linux64.zip from: https://chromedriver.storage.googleapis.com/114.0.5735.90/chromedriver_linux64.zip ... Download Complete! Extracting ['chromedriver'] from chromedriver_linux64.zip ... Unzip Complete! The file [chromedriver] was saved to: /home/tkloczko/rpmbuild/BUILDROOT/python-seleniumbase-4.14.14-2.fc35.x86_64/usr/lib/python3.8/site-packages/seleniumbase/drivers/chromedriver Making [chromedriver 114.0.5735.90] executable ... [chromedriver 114.0.5735.90] is now ready for use! [..] =========================== short test summary info ============================ XFAIL examples/test_xfail.py::XFailTests::test_xfail ERROR examples/test_override_sb_fixture.py::test_override_fixture_no_class - ... ERROR examples/test_override_sb_fixture.py::TestOverride::test_override_fixture_inside_class ERROR examples/test_pytest_parametrize.py::test_sb_fixture_with_no_class[List of Features] ERROR examples/test_pytest_parametrize.py::test_sb_fixture_with_no_class[Command Line Options] ERROR examples/test_pytest_parametrize.py::Test_SB_Fixture::test_sb_fixture_inside_class[Console Scripts] ERROR examples/test_pytest_parametrize.py::Test_SB_Fixture::test_sb_fixture_inside_class[API Reference] ERROR examples/test_repeat_tests.py::test_repeat_this_test_with_pytest_parametrize[0] ERROR examples/test_repeat_tests.py::test_repeat_this_test_with_pytest_parametrize[1] ERROR examples/test_repeat_tests.py::RepeatTestsWithPytest::test_repeat_test_with_pytest_parametrize[0] ERROR examples/test_repeat_tests.py::RepeatTestsWithPytest::test_repeat_test_with_pytest_parametrize[1] ERROR examples/test_sb_fixture.py::test_sb_fixture_with_no_class - selenium.c... ERROR examples/test_sb_fixture.py::Test_SB_Fixture::test_sb_fixture_inside_class ERROR examples/test_usefixtures.py::Test_UseFixtures::test_usefixtures_on_class ERROR examples/boilerplates/sb_fixture_test.py::ObjTests::test_data_url_page ERROR examples/boilerplates/samples/sb_swag_test.py::MyTests::test_swag_labs_login ERROR examples/visual_testing/test_layout_fail.py::VisualLayout_FixtureTests::test_python_home_change FAILED examples/basic_test.py::MyTestClass::test_basics - selenium.common.exc... FAILED examples/coffee_cart_tests.py::CoffeeCartTests::test_1_verify_nav_link_to_coffee_cart FAILED examples/coffee_cart_tests.py::CoffeeCartTests::test_buy_one_cappuccino FAILED examples/coffee_cart_tests.py::CoffeeCartTests::test_coffee_promo_with_preview_0 FAILED examples/coffee_cart_tests.py::CoffeeCartTests::test_coffee_promo_with_preview_1 FAILED examples/coffee_cart_tests.py::CoffeeCartTests::test_context_click_add_coffee FAILED examples/coffee_cart_tests.py::CoffeeCartTests::test_remove_added_coffee FAILED examples/edge_test.py::EdgeTests::test_edge - selenium.common.exceptio... FAILED examples/github_test.py::GitHubTests::test_github - selenium.common.ex... FAILED examples/handle_alert_test.py::HandleAlertTests::test_alerts - seleniu... FAILED examples/iframe_tests.py::FrameTests::test_iframe_basics - selenium.co... FAILED examples/iframe_tests.py::FrameTests::test_iframes_with_context_manager FAILED examples/iframe_tests.py::FrameTests::test_set_content_to_frame - sele... FAILED examples/locale_code_test.py::LocaleCodeTests::test_locale_code - sele... FAILED examples/my_first_test.py::MyTestClass::test_swag_labs - selenium.comm... FAILED examples/nth_child_test.py::NthChildSelectorTests::test_locate_rows_with_colors FAILED examples/parameterized_test.py::SearchTests::test_parameterized_search_0_SeleniumBase_Commander FAILED examples/parameterized_test.py::SearchTests::test_parameterized_search_1_SeleniumBase_Recorder FAILED examples/parameterized_test.py::SearchTests::test_parameterized_search_2_SeleniumBase_Syntax FAILED examples/performance_test.py::PerformanceClass::test_performance_of_swag_labs FAILED examples/proxy_test.py::ProxyTests::test_proxy - selenium.common.excep... FAILED examples/rate_limiting_test.py::RateLimitingTests::test_rate_limited_printing FAILED examples/shadow_root_test.py::ShadowRootTest::test_shadow_root - selen... FAILED examples/swag_labs_user_tests.py::SwagLabsTests::test_swag_labs_user_flows_0_standard_user FAILED examples/swag_labs_user_tests.py::SwagLabsTests::test_swag_labs_user_flows_1_problem_user FAILED examples/test_apple_site.py::AppleTests::test_apple_developer_site_webdriver_instructions FAILED examples/test_assert_elements.py::ListAssertTests::test_assert_list_of_elements FAILED examples/test_calculator.py::CalculatorTests::test_6_times_7_plus_12_equals_54 FAILED examples/test_canvas.py::CanvasTests::test_canvas_click_from_center - ... FAILED examples/test_canvas.py::CanvasTests::test_click_with_offset - seleniu... FAILED examples/test_checkboxes.py::CheckboxTests::test_checkboxes_and_radio_buttons FAILED examples/test_chinese_pdf.py::ChinesePdfTests::test_chinese_pdf - sele... FAILED examples/test_chromedriver.py::ChromedriverTests::test_chromedriver_matches_chrome FAILED examples/test_coffee_cart.py::CoffeeCartTest::test_coffee_cart - selen... FAILED examples/test_console_logging.py::TestConsoleLogging::test_console_logging FAILED examples/test_contains_selector.py::ContainsSelectorTests::test_contains_selector FAILED examples/test_cycle_elements.py::CycleTests::test_cycle_elements_with_tab_and_press_enter FAILED examples/test_decryption.py::DecryptionTests::test_decrypt_password - ... FAILED examples/test_deferred_asserts.py::DeferredAssertTests::test_deferred_asserts FAILED examples/test_demo_site.py::DemoSiteTests::test_demo_site - selenium.c... FAILED examples/test_detect_404s.py::BrokenLinkTests::test_link_checking - se... FAILED examples/test_docs_site.py::DocsSiteTests::test_docs - selenium.common... FAILED examples/test_double_click.py::DoubleClickTests::test_switch_to_frame_and_double_click FAILED examples/test_double_click.py::DoubleClickTests::test_switch_to_frame_of_element_and_double_click FAILED examples/test_download_files.py::DownloadTests::test_download_chromedriver_notes FAILED examples/test_download_files.py::DownloadTests::test_download_files_from_pypi FAILED examples/test_download_images.py::DownloadImages::test_download_images_directly FAILED examples/test_download_images.py::DownloadImages::test_download_images_via_screenshot FAILED examples/test_drag_and_drop.py::DragAndDropTests::test_drag_and_drop FAILED examples/test_drag_and_drop.py::DragAndDropTests::test_w3schools_drag_and_drop FAILED examples/test_error_page.py::ErrorPageTests::test_error_page - seleniu... FAILED examples/test_event_firing.py::EventFiringTests::test_event_firing_webdriver FAILED examples/test_fail.py::FailingTests::test_find_army_of_robots_on_xkcd_desert_island FAILED examples/test_get_locale_code.py::LocaleTests::test_get_locale_code - ... FAILED examples/test_get_pdf_text.py::PdfTests::test_get_pdf_text - selenium.... FAILED examples/test_get_user_agent.py::UserAgentTests::test_get_user_agent FAILED examples/test_hack_search.py::HackingTests::test_hack_search - seleniu... FAILED examples/test_image_saving.py::ImageTests::test_1_save_element_as_image_file FAILED examples/test_image_saving.py::ImageTests::test_2_add_text_overlay_to_image FAILED examples/test_image_saving.py::ImageTests::test_3_add_text_overlay_to_page_section FAILED examples/test_image_saving.py::ImageTests::test_4_add_text_overlay_to_full_page FAILED examples/test_inspect_html.py::HtmlInspectorTests::test_html_inspector FAILED examples/test_login.py::SwagLabsLoginTests::test_swag_labs_login - sel... FAILED examples/test_markers.py::MarkerTestSuite::test_A - selenium.common.ex... FAILED examples/test_markers.py::MarkerTestSuite::test_B - selenium.common.ex... FAILED examples/test_markers.py::MarkerTestSuite::test_C - selenium.common.ex... FAILED examples/test_markers.py::MarkerTestSuite::test_D - selenium.common.ex... FAILED examples/test_mfa_login.py::TestMFALogin::test_mfa_login - selenium.co... FAILED examples/test_multiple_drivers.py::MultipleDriversTest::test_multiple_drivers FAILED examples/test_null.py::NullTests::test_null - selenium.common.exceptio... FAILED examples/test_override_driver.py::OverrideDriverTest::test_driver_override FAILED examples/test_parse_soup.py::SoupParsingTests::test_beautiful_soup_parsing FAILED examples/test_pdf_asserts.py::PdfAssertTests::test_assert_pdf_text - s... FAILED examples/test_repeat_tests.py::RepeatTests::test_repeat_this_test_with_parameterized_0 FAILED examples/test_repeat_tests.py::RepeatTests::test_repeat_this_test_with_parameterized_1 FAILED examples/test_request_sb_fixture.py::test_request_sb_fixture - seleniu... FAILED examples/test_request_sb_fixture.py::Test_Request_Fixture::test_request_sb_fixture_in_class FAILED examples/test_save_screenshots.py::ScreenshotTests::test_save_screenshot FAILED examples/test_save_screenshots.py::ScreenshotTests::test_save_screenshot_to_logs FAILED examples/test_scrape_bing.py::ScrapeBingTests::test_scrape_bing - sele... FAILED examples/test_select_options.py::SelectTestClass::test_base - selenium... FAILED examples/test_shadow_dom.py::ShadowDomTests::test_shadow_dom - seleniu... FAILED examples/test_show_file_choosers.py::FileUpload::test_show_file_choosers FAILED examples/test_skype_site.py::SkypeTests::test_skype_mobile_site - sele... FAILED examples/test_suite.py::MyTestSuite::test_1 - selenium.common.exceptio... FAILED examples/test_suite.py::MyTestSuite::test_2 - selenium.common.exceptio... FAILED examples/test_suite.py::MyTestSuite::test_3 - selenium.common.exceptio... FAILED examples/test_suite.py::MyTestSuite::test_4 - selenium.common.exceptio... FAILED examples/test_swag_labs.py::SwagLabsTests::test_swag_labs_basic_flow FAILED examples/test_tinymce.py::TinyMceTests::test_tinymce - selenium.common... FAILED examples/test_todomvc.py::TodoMVC::test_todomvc_0_jquery - selenium.co... FAILED examples/test_todomvc.py::TodoMVC::test_todomvc_1_react - selenium.com... FAILED examples/test_todomvc.py::TodoMVC::test_todomvc_2_vue - selenium.commo... FAILED examples/test_url_asserts.py::URLTestClass::test_url_asserts - seleniu... FAILED examples/test_verify_chromedriver.py::ChromedriverTests::test_fail_if_using_an_old_chromedriver FAILED examples/test_window_switching.py::TabSwitchingTests::test_switch_to_tabs FAILED examples/test_xkcd.py::MyTestClass::test_xkcd - selenium.common.except... FAILED examples/time_limit_test.py::TimeLimitTests::test_runtime_limit_decorator FAILED examples/time_limit_test.py::TimeLimitTests::test_set_time_limit_method FAILED examples/upload_file_test.py::FileUploadButtonTests::test_file_upload_button FAILED examples/user_agent_test.py::UserAgentTests::test_user_agent - seleniu... FAILED examples/wordle_test.py::WordleTests::test_wordle - selenium.common.ex... FAILED examples/xpath_test.py::XPathTests::test_xpath - selenium.common.excep... FAILED examples/youtube_search_test.py::YouTubeSearchTests::test_youtube_autocomplete_results FAILED examples/boilerplates/boilerplate_test.py::MyTestClass::test_boilerplate FAILED examples/boilerplates/classic_obj_test.py::ObjTests::test_data_url_page FAILED examples/boilerplates/samples/google_test.py::GoogleTests::test_google_dot_com FAILED examples/boilerplates/samples/swag_labs_test.py::MyTests::test_swag_labs_login FAILED examples/boilerplates/samples/test_page_objects.py::MyTests::test_page_objects FAILED examples/chart_maker/test_area_chart.py::MyChartMakerClass::test_area_chart FAILED examples/chart_maker/test_display_chart.py::MyChartMakerClass::test_display_chart FAILED examples/chart_maker/test_line_chart.py::MyChartMakerClass::test_line_chart FAILED examples/chart_maker/test_multi_series.py::MyChartMakerClass::test_multi_series FAILED examples/chart_maker/test_save_chart.py::MyChartMakerClass::test_save_chart FAILED examples/migration/protractor/example_test.py::AngularJSHomePageTests::test_greet_user FAILED examples/migration/protractor/example_test.py::AngularJSHomePageTests::test_todo_list FAILED examples/migration/protractor/input_test.py::AngularMaterialInputTests::test_invalid_input FAILED examples/migration/protractor/mat_paginator_test.py::AngularMaterialPaginatorTests::test_pagination FAILED examples/offline_examples/load_html_test.py::OfflineTests::test_load_html_string FAILED examples/offline_examples/test_demo_page.py::OfflineTests::test_demo_page FAILED examples/offline_examples/test_handle_alerts.py::OfflineTests::test_alerts FAILED examples/offline_examples/test_request_fixture.py::test_request_fixture FAILED examples/offline_examples/test_request_fixture.py::RequestTests::test_request_fixture_in_class FAILED examples/offline_examples/test_user_agent.py::OfflineTests::test_get_user_agent FAILED examples/visual_testing/layout_test.py::VisualLayoutTests::test_applitools_layout_change FAILED examples/visual_testing/python_home_test.py::VisualLayoutTests::test_python_home_layout_change FAILED examples/visual_testing/test_layout_fail.py::VisualLayoutFailureTests::test_applitools_change FAILED examples/visual_testing/test_layout_fail.py::VisualLayoutFailureTests::test_xkcd_logo_change FAILED examples/visual_testing/xkcd_visual_test.py::VisualLayoutTests::test_xkcd_layout_change FAILED integrations/node_js/my_first_test.py::MyTestClass::test_swag_labs - s... FAILED integrations/node_js/test_demo_site.py::DemoSiteTests::test_demo_site ============ 131 failed, 1 xfailed, 16 errors in 360.76s (0:06:00) ============= ```

Full log is in attachment python-seleniumbase.FAIL.txt

On first look some units may be failing because probably some files should not be scanned by pytest (pytest testpaths definition is missing? 🤔 )

Here is list of installed modules in build env

```console Package Version ----------------------------- --------- alabaster 0.7.13 asttokens 2.2.1 async-generator 1.10 attrs 23.1.0 Babel 2.12.1 backcall 0.2.0 boto 2.49.0 build 0.10.0 certifi 2022.12.7 cffi 1.15.1 charset-normalizer 3.1.0 colorama 0.4.6 cssselect 1.2.0 decorator 5.1.1 distlib 0.3.6 distro 1.8.0 docutils 0.19 exceptiongroup 1.1.1 execnet 1.9.0 executing 1.2.0 fasteners 0.18 filelock 3.12.0 gpg 1.20.0 h11 0.14.0 idna 3.4 imagesize 1.4.1 importlib-metadata 6.6.0 iniconfig 2.0.0 installer 0.7.0 ipython 8.12.0 jedi 0.18.2 Jinja2 3.1.2 libcomps 0.1.19 MarkupSafe 2.1.2 matplotlib-inline 0.1.6 outcome 1.2.0 packaging 23.1 parameterized 0.9.0 parso 0.8.3 pexpect 4.8.0 pickleshare 0.7.5 platformdirs 3.5.1 pluggy 1.0.0 ply 3.11 prompt-toolkit 3.0.38 ptyprocess 0.7.0 pure-eval 0.2.2 py 1.11.0 pycparser 2.21 Pygments 2.15.1 pyotp 2.8.0 pyproject_hooks 1.0.0 pytest 7.3.1 pytest-forked 1.6.0 pytest-html 3.2.0 pytest-metadata 2.0.4 pytest-ordering 0.6 python-dateutil 2.8.2 pytz 2023.2 requests 2.30.0 sbvirtualdisplay 1.2.0 selenium 4.9.0 setuptools 67.7.2 setuptools-scm 7.1.0 six 1.16.0 sniffio 1.3.0 snowballstemmer 2.2.0 sortedcontainers 2.4.0 Sphinx 6.2.1 sphinxcontrib-applehelp 1.0.4 sphinxcontrib-devhelp 1.0.2 sphinxcontrib-htmlhelp 2.0.0 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.3 sphinxcontrib-serializinghtml 1.1.5 stack-data 0.6.2 toml 0.10.2 tomli 2.0.1 traitlets 5.9.0 trio 0.22.0 trio-websocket 0.10.2 typing_extensions 4.5.0 urllib3 1.26.15 virtualenv 20.23.0 wcwidth 0.2.6 wheel 0.40.0 wsproto 1.2.0 zipp 3.15.0 ```
mdmintz commented 1 year ago

I'm seeing the error: cannot find Chrome binary in your logs, which means that either Chrome isn't installed on your machine, or it's not on the PATH.

And seleniumbase should be in your list of installed modules, but it isn't there.

It might be that your --no-isolation option is causing problems. That does not appear to be a supported option if that's the cause of your errors.

pradyunsg commented 1 year ago

Hi @mdmintz! Sorry about the bump on a closed issue. 👋🏽

Is @kloczek banned in this organisation? This looks like a sock-puppet account for that user and, my current hypothesis is that they're using this account to circumvent that they've been blocked by a lot of the Python community on Github.

PS: Totally understandable, if you'd prefer to avoid answering this question. :)