Test does not update according to the screen

[rjmautvix]

Checklist

• [X] I have searched the existing issues for similar issues.
• [X] I added a very descriptive title to this issue.
• [X] I have provided sufficient information below to help reproduce this issue.

Summary

I wrote code for a streamlit page with 2 tabs: the first with a multiselect and the second with a selectbox.

In the first with a multiselect, I chose to lock the button if there is no option selected and, when options are selected, the button is unlocked.

In the second tab, I have a selectbox that, when an inventory with a following condition is selected (in the real app the condition is different), the button is blocked.

Operating the screen works as expected, but I can't execute the same behavior in tests. Testing the first tab, the button is not unlocked when selecting a multiselect option and, testing the second tab, the button is not blocked when the condition is met.

Reproducible Code Example

from streamlit.testing.v1 import AppTest

screen_code = """

        import time
        import streamlit as st

        def main():
            st.title("Test", anchor=False)
            inventories = [
                {"id_inventory": 1, "description": "Inventory 1"},
                {"id_inventory": 2, "description": "Inventory 2"},
                {"id_inventory": 3, "description": "Inventory 3"},
            ]

            if "testing_tab1" not in st.session_state:
                st.session_state["testing_tab1"] = False

            if "testing_tab2" not in st.session_state:
                st.session_state["testing_tab2"] = False

            render_tabs(inventories)

        def render_tabs(inventories):
            tab1, tab2 = st.tabs(["Test 1", "Test 2"])

            with tab1:
                render_tab1(inventories)

            with tab2:
                render_tab2(inventories)

        def render_tab1(inventories):
            button_disabled = False
            if not inventories:
                button_disabled = True

            items: list[dict] = st.multiselect(
                "Estoques", inventories, format_func=lambda x: x["description"]
            )

            if not items:
                button_disabled = True
            else:
                button_disabled = False

            if st.button(
                "Testar",
                disabled=button_disabled or st.session_state["testing_tab1"],
                on_click=lambda: st.session_state.update(testing_tab1=True),
                key="teste_1",
            ):
                st.write("button clicked")
                del st.session_state["testing_tab1"]
                time.sleep(2)
                st.rerun()

        def render_tab2(inventories):
            button_disabled = False
            if not inventories:
                button_disabled = True

            item: list[dict] = st.selectbox(
                "Estoques", inventories, format_func=lambda x: x["description"]
            )

            if item["description"] == inventories[0]["description"]:
                button_disabled = True

            if st.button(
                "Testar",
                disabled=button_disabled or st.session_state["testing_tab2"],
                on_click=lambda: st.session_state.update(testing_tab2=True),
                key="teste_2",
            ):
                st.write("button clicked")
                del st.session_state["testing_tab2"]
                time.sleep(2)
                st.rerun()

        main()
    """

def test_screen_tab1():
    app = AppTest.from_string(screen_code)
    app.run()
    tabs = app.tabs[0]

    multiselect = tabs.multiselect[0]
    button = tabs.button[0]
    multiselect.set_value("Inventory 1")
    button.click().run()
    assert app.session_state.testing_tab1 == True

def test_screen_tab2():
    app = AppTest.from_string(screen_code)
    app.run()
    tabs = app.tabs[1]

    selectbox = tabs.selectbox[0]
    button = tabs.button[0]

    selectbox.set_value("Inventory 2")
    assert button.disabled == False

Steps To Reproduce

1. create a venv

   py -m venv venv

2. install streamlit

   pip install streamlit

3. install pytest

   pip install pytest

4. run the tests

   python -m pytest

5. create an app.py file with the contents of the screen_code variable and run the command

   python -m streamlit run app.py

Expected Behavior

In the first test, after selecting an option in multiselect, it should be possible to click the button.

In the second test, the initial value of the selectbox is the inventory that will meet the condition to block the button and when selecting another inventory that does not meet the condition, it is to unlock the button.

Current Behavior

In the first test, the button is not unlocking when selecting a multiselect option and it is not possible to click on the button and the error occurs: TypeError: string indices must be integers, not 'str'.

In the second test, the button is not unlocking when selecting an inventory that does not meet the condition.

Full test log:

python -m pytest
=============================================================================== test session starts ===============================================================================
platform win32 -- Python 3.11.6, pytest-8.3.3, pluggy-1.5.0
rootdir: repo_test
collected 2 items

test_app.py FF                                                                                                                                                               [100%]

==================================================================================== FAILURES =====================================================================================
________________________________________________________________________________ test_screen_tab1 _________________________________________________________________________________

    def test_screen_tab1():
        app = AppTest.from_string(screen_code)
        app.run()
        tabs = app.tabs[0]

        multiselect = tabs.multiselect[0]
        button = tabs.button[0]
        multiselect.set_value("Inventory 1")
>       button.click().run()

test_app.py:97:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
venv\Lib\site-packages\streamlit\testing\v1\element_tree.py:152: in run
    return self.root.run(timeout=timeout)
venv\Lib\site-packages\streamlit\testing\v1\element_tree.py:1924: in run
    widget_states = self.get_widget_states()
venv\Lib\site-packages\streamlit\testing\v1\element_tree.py:1907: in get_widget_states
    w = get_widget_state(node)
venv\Lib\site-packages\streamlit\testing\v1\element_tree.py:1850: in get_widget_state
    return node._widget_state
venv\Lib\site-packages\streamlit\testing\v1\element_tree.py:808: in _widget_state
    ws.int_array_value.data[:] = self.indices
venv\Lib\site-packages\streamlit\testing\v1\element_tree.py:824: in indices
    return [self.options.index(self.format_func(v)) for v in self.value]
venv\Lib\site-packages\streamlit\testing\v1\element_tree.py:824: in <listcomp>
    return [self.options.index(self.format_func(v)) for v in self.value]
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

x = 'I'

>       "Estoques", inventories, format_func=lambda x: x["description"]
    )
E   TypeError: string indices must be integers, not 'str'

..\..\AppData\Local\Temp\tmpl__aj_in\ca010ee7beff1e70dd1341171a36203b:40: TypeError
------------------------------------------------------------------------------ Captured stderr call ------------------------------------------------------------------------------- 
2024-09-16 13:00:08.278 WARNING streamlit.runtime.scriptrunner_utils.script_run_context: Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode.
________________________________________________________________________________ test_screen_tab2 _________________________________________________________________________________ 

    def test_screen_tab2():
        app = AppTest.from_string(screen_code)
        app.run()
        tabs = app.tabs[1]

        selectbox = tabs.selectbox[0]
        button = tabs.button[0]

        selectbox.set_value("Inventory 2")
>       assert button.disabled == False
E       AssertionError: assert True == False
E        +  where True = Button(key='teste_2', disabled=True, label='Testar').disabled

test_app.py:111: AssertionError
------------------------------------------------------------------------------ Captured stderr call ------------------------------------------------------------------------------- 
2024-09-16 13:00:08.630 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode.
============================================================================= short test summary info ============================================================================= 
FAILED test_app.py::test_screen_tab1 - TypeError: string indices must be integers, not 'str'
FAILED test_app.py::test_screen_tab2 - AssertionError: assert True == False
================================================================================ 2 failed in 1.15s ================================================================================ 

Is this a regression?

• [ ] Yes, this used to work in a previous version.

Debug info

• Streamlit version: 1.38.0
• Python version: 3.11.6
• Operating System: Windows
• Browser: Google

Additional Information

No response

[github-actions[bot]]

If this issue affects you, please react with a 👍 (thumbs up emoji) to the initial post.

Your feedback helps us prioritize which bugs to investigate and address first.

[dotpie]

The test code looks good, so I tested it for a while. There were several minor issues I found.

• I have just started learning Streamlit, so I might be pointing out non-issues. I apologize in advance for any inconvenience.

1. The argument type of multiselect.set_value is a list.

2. E: TypeError: string indices must be integers, not 'str' According to error message message, v under the codeblock should be subscriptable. Thus, dictionary like {"id_inventory": 1, "description": "Inventory 1"} should be passed. similar to what you wrote at type hint of items variable.

   /streamlit/testing/v1/element_tree.py:824: in <listcomp>
       return [self.options.index(self.format_func(v)) for v in self.value]

   # To use the format_func below,
   items: list[dict] = st.multiselect(
               "Estoques", inventories, format_func=lambda x: x["description"]
           )
   
   # The test function should modified like this
   multiselect.set_value([{"id_inventory": 1, "description": "Inventory 1"}])

3. As if st.button(~) statement delete testing_tab1, main function always sets the variable testing_tab1 False. How about modifying code like below?:

   def render_tab1(inventories):
       ...
       if not items:
           button_disabled = True
           st.session_state["testing_tab1"] = False
       else:
           button_disabled = False
           st.session_state["testing_tab1"] = True

   In my opinion, button_disabled and st.session_state["testing_tab1"] should be considered at once. Because those status changes from same trigger. Also, the variables like st.session_state["testing_tab1"] are only used in testing code, not affect rendered view. That's the reason why your screen looks ok.

Couldn't test screen view, so not sure if my suggestion was correct. However a slight logic change seems to improve the result better, so leave my comment. Have a good day!

[rjmautvix]

The test code looks good, so I tested it for a while. There were several minor issues I found.

• I have just started learning Streamlit, so I might be pointing out non-issues. I apologize in advance for any inconvenience.

1. The argument type of multiselect.set_value is a list.

2. E: TypeError: string indices must be integers, not 'str' According to error message message, v under the codeblock should be subscriptable. Thus, dictionary like {"id_inventory": 1, "description": "Inventory 1"} should be passed. similar to what you wrote at type hint of items variable.

   /streamlit/testing/v1/element_tree.py:824: in <listcomp>
      return [self.options.index(self.format_func(v)) for v in self.value]

   # To use the format_func below,
   items: list[dict] = st.multiselect(
              "Estoques", inventories, format_func=lambda x: x["description"]
          )
   
   # The test function should modified like this
   multiselect.set_value([{"id_inventory": 1, "description": "Inventory 1"}])

3. As if st.button(~) statement delete testing_tab1, main function always sets the variable testing_tab1 False. How about modifying code like below?:

   def render_tab1(inventories):
      ...
      if not items:
          button_disabled = True
          st.session_state["testing_tab1"] = False
      else:
          button_disabled = False
          st.session_state["testing_tab1"] = True

   In my opinion, button_disabled and st.session_state["testing_tab1"] should be considered at once. Because those status changes from same trigger. Also, the variables like st.session_state["testing_tab1"] are only used in testing code, not affect rendered view. That's the reason why your screen looks ok.

Couldn't test screen view, so not sure if my suggestion was correct. However a slight logic change seems to improve the result better, so leave my comment. Have a good day!

Hi, thanks for the digestion! Really if I replace this piece of code:

def test_screen_tab1():
    app = AppTest.from_string(screen_code)
    app.run()
    tabs = app.tabs[0]

    multiselect = tabs.multiselect[0]
    button = tabs.button[0]
    multiselect.set_value("Inventory 1")
    button.click().run()
    assert app.session_state.testing_tab1 == True

For this:

def test_screen_tab1():
    app = AppTest.from_string(screen_code)
    app.run()
    tabs = app.tabs[0]

    multiselect = tabs.multiselect[0]
    button = tabs.button[0]
    multiselect.set_value([{"id_inventory": 1, "description": "Inventory 1"}])
    button.click().run()

    assert app.session_state.testing_tab1 == True

The issue "TypeError: string indices must be integers, not 'str'" is resolved. However, the test still does not occur as expected. While on the screen when something is selected in multiselect the button is unlocked, in the test this does not happen. Even though I select something in multiselect and the _value attribute exists and has a value, the button remains disabled.

To prove the case mentioned above, just add this test case:

def test_screen_tab1():
    app = AppTest.from_string(screen_code)
    app.run()
    tabs = app.tabs[0]

    multiselect = tabs.multiselect[0]
    button = tabs.button[0]
    multiselect.set_value([{"id_inventory": 1, "description": "Inventory 1"}])

    assert button.disabled == False

[mayagbarnes]

Hi @rjmautvix, appreciate your submission! There is certainly a gap with our AppTest capabilities right now, and unfortunately we don't know when we will be able to resolve them. We'll keep this issue open so that others can 👍 - we use that information to prioritize fixes. Thanks for your patience 🙏