Selenium testing failing

[sularz-maciej]

Hi Miguel,

I'm just about to finish your book, absolutely love it. I've been following along as I read and noticed that the packages are quite outdated (as expected). I decided to code your app using the latest available packages and so far apart from some minor syntax differences it was smooth sailing. This was the case up until the 'End-to-End Testing with Selenium' (15d), it took me 2 days to make it work with Selenium v4.7.2 and Unittest and I just wanted to leave it here in case someone else runs into this problem as well as ask if this is the correct way to do it. It feels more like a hack to me rather than the actual solution so I would really appreciate your input.

Below are the packages I'm using as well as my solution to the problem. By the way I'm also using ChromeDriver 108.0.5359.71

I figured out the solution tanks to https://github.com/pallets/flask/issues/2776

requirements/common.txt

alembic==1.8.1
bleach==5.0.1
blinker==1.5
click==8.1.3
colorama==0.4.5
dnspython==2.2.1
dominate==2.7.0
email-validator==1.3.0
Flask==2.2.2
Flask-Bootstrap==3.3.7.1
Flask-HTTPAuth==4.7.0
Flask-Login==0.6.2
Flask-Mail==0.9.1
Flask-Migrate==3.1.0
Flask-Moment==1.0.5
Flask-PageDown==0.4.0
Flask-SQLAlchemy==3.0.2
Flask-WTF==1.0.1
greenlet==2.0.0
idna==3.4
itsdangerous==2.1.2
Jinja2==3.1.2
Mako==1.2.3
Markdown==3.4.1
MarkupSafe==2.1.1
packaging==21.3
pyparsing==3.0.9
python-dateutil==2.8.2
python-dotenv==0.21.0
six==1.16.0
SQLAlchemy==1.4.42
visitor==0.1.3
webencodings==0.5.1
Werkzeug==2.2.2
WTForms==3.0.1

requirements/common.txt

-r common.txt
charset-normalizer==2.1.1
certifi==2022.9.24
commonmark==0.9.1
coverage==6.5.0
defusedxml==0.7.1
Faker==15.2.0
httpie==3.2.1
multidict==6.0.2
Pygments==2.13.0
PySocks==1.7.1
requests==2.28.1
requests-toolbelt==0.10.1
rich==12.6.0
selenium==4.7.2
urllib3==1.26.12

main/views.py

[...]

@main.route('/shutdown')
def server_shutdown():
    if not current_app.testing:
        abort(404)

    # request.environ.get('werkzeug.server.shutdown') has been deprecated
    # So I used the following instead:
    os.kill(os.getpid(), signal.SIGINT)
    return 'Shutting down...'

[...]

config.py

[...]

# I added the following configuration which is the FIX to my problem
class TestingWithSeleniumConfig(TestingConfig):
    @staticmethod
    def init_app(app):
        if os.environ.get('FLASK_RUN_FROM_CLI'):
            os.environ.pop('FLASK_RUN_FROM_CLI')

[...]

config = {
    [...]
    'testing-with-selenium': TestingWithSeleniumConfig,
    [...]
}

tests/test_selenium.py

import re
import threading
import unittest

from selenium import webdriver
from selenium.webdriver.common.by import By

from app import create_app, db, fake
from app.models import Role, User, Post

class SeleniumTestCase(unittest.TestCase):
    # I don't like things hardcoded where possible
    HOST = 'localhost'
    PORT = 5000

    # PyCharm complaining without those
    client = None
    app = None
    app_context = None
    server_thread = None

    @classmethod
    def setUpClass(cls):
        options = webdriver.ChromeOptions()
        options.add_argument('headless')
        # This suppresses some jibberish from webdriver
        options.add_experimental_option('excludeSwitches', ['enable-logging'])

        # noinspection PyBroadException
        try:
            cls.client = webdriver.Chrome(options=options)
        except Exception:
            pass

        # Skip these tests if the web browser could not be started
        if cls.client:
            # Create the application
            # FIX: making use of 'testing-with-selenium' config
            cls.app = create_app('testing-with-selenium')
            cls.app_context = cls.app.app_context()
            cls.app_context.push()

            # Suppress logging to keep unittest output clean
            import logging
            logger = logging.getLogger('werkzeug')
            logger.setLevel('ERROR')

            # Create the database and  populate with some fake data
            db.create_all()
            Role.insert_roles()
            fake.users(10)
            fake.posts(10)

            # Add an administrator user
            admin_role = Role.query.filter_by(permissions=0xff).first()
            admin = User(email='john@example.com', username='john', password='cat', role=admin_role, confirmed=True)
            db.session.add(admin)
            db.session.commit()

            # Start the flask server in a thread
            cls.server_thread = threading.Thread(target=cls.app.run, kwargs={
                'host': cls.HOST,
                'port': cls.PORT,

                'debug': False,
                'use_reloader': False,
                'use_debugger': False
            })

            cls.server_thread.start()

    @classmethod
    def tearDownClass(cls):
        if cls.client:
            # Stop the Flask server and the browser
            cls.client.get(f'http://{cls.HOST}:{cls.PORT}/shutdown')
            cls.client.quit()
            cls.server_thread.join()

            # Destroy the database
            db.drop_all()
            db.session.remove()

            # Remove application context
            cls.app_context.pop()

    def setUp(self):
        if not self.client:
            self.skipTest('Web browser not available')

    def tearDown(self):
        pass

    def test_admin_home_page(self):
        # Navigate to home page
        self.client.get(f'http://{self.HOST}:{self.PORT}/')
        self.assertTrue(re.search(r'Hello,\s+Stranger!', self.client.page_source))

        # Navigate to login page
        self.client.find_element(By.LINK_TEXT, 'Log In').click()
        self.assertIn('<h1>Login</h1>', self.client.page_source)

        # Login
        self.client.find_element(By.NAME, 'email').send_keys('john@example.com')
        self.client.find_element(By.NAME, 'password').send_keys('cat')
        self.client.find_element(By.NAME, 'submit').click()
        self.assertTrue(re.search(r'Hello,\s+john!', self.client.page_source))

        # Navigate to the user's profile page
        self.client.find_element(By.LINK_TEXT, 'Profile').click()
        self.assertIn('<h1>john</h1>', self.client.page_source)

[miguelgrinberg]

This looks good, thanks for sharing it. I think given the lack of interest from the Flask team in preserving features of the framework that worked before, such as the server shutdown and app.run() integration with the CLI, what you have done is probably the best option.

[BountyHunter1999]

• when I ran above code the test couldn't terminate

So I tried it with these changes and it worked

test_selenium.py

            cls.server_thread = threading.Thread(
                target=cls.app.run,
                kwargs={
                    "host": cls.HOST,
                    "port": cls.PORT,
                    "debug": False,
                    "use_reloader": False,
                    "use_debugger": False,
                },
                daemon=True
            )

• made it a daemon thread so that when the program exists the thread is also terminated and it doesn't hold

  cls.server_thread.join(2)

• wait at most 2 seconds for the thread to complete and return back to the main thread even if the thread's work isn't complete

[Muhammad-Nasr]

Thank you, @miguelgrinberg, for your invaluable help with Flask. We truly appreciate your guidance and expertise as our Flask Mentor. I hope you are doing well. I would also like to extend my gratitude to @sularz-maciej for your approach in assisting me.

Currently, I am facing two issues with the following approach: os.kill(os.getpid(), signal.SIGINT)

Using this method terminates the process and ends all other tests.

Unfortunately, I am unable to determine if the other methods are running successfully. To provide more context, here is the relevant code snippet:.

# Destroy the database
db.drop_all()
db.session.remove()

# Remove the application context
cls.app_context.pop()

I would greatly appreciate any assistance you can provide in resolving this issue. If there are any suggestions or alternative approaches you could recommend, I would be grateful for your expertise.

Thank you once again for your valuable help.