Catch streamlit.runtime.caching.cache_data_api ?

maxibor commented 1 year ago

Checklist

[X] I have searched the existing issues for similar issues.
[X] I added a very descriptive title to this issue.
[X] I have provided sufficient information below to help reproduce this issue.

Summary

When using cache_data in a streamlit app launched in "raw mode" (for example, from python, using streamlit.web.cli), streamlit throws the following warning:

WARNING streamlit.runtime.caching.cache_data_api: No runtime found, using MemoryCacheStorageManager

which comes from here https://github.com/streamlit/streamlit/blob/cec1c141f7858e31f8200a9cdc42b8ddd09ab0e7/lib/streamlit/runtime/caching/cache_data_api.py#L290-L297

Despite trying my best, I've been unable to catch the warning, or deactivate it with the various streamlit configuration/CLI flags :(

Is there a way to catch this warning ?

Reproducible Code Example

No response

Steps To Reproduce

No response

Expected Behavior

Provide a mechanism to catch the warning.

Current Behavior

No response

Is this a regression?

[ ] Yes, this used to work in a previous version.

Debug info

Streamlit version: 1.22.0
Python version: 3.10.8
Operating System: macOS 12.6.1
Browser: Firefox
Virtual environment: conda

Additional Information

No response

Are you willing to submit a PR?

[ ] Yes, I am willing to submit a PR!

kajarenc commented 1 year ago

hey @maxibor , thanks for opening the issue!

We log this message under the warning level, to inform that caching could work differently from how it works in case of existing runtime (when we run a script with streamlit run app.py)

Currently, we don't have config options to configure streamlit internal logging from there.

CC: @sfc-gh-jcarroll since this potentially related to config.py project

sfc-gh-jrieke commented 1 year ago

I'll let Joshua decide but I think it's totally ok and expected to show a warning in this case.

maxibor commented 1 year ago

Thanks for the update. If there would be a way to catch this warning, that would be awesome, because users might be thrown off a bit by having it displayed unexpectedly.

sfc-gh-jcarroll commented 1 year ago

@kajarenc what's the behavior of caching if the app is executed in this way? Will the in-memory cache manager work? (or @maxibor are you seeing it working?)

Can it be fixed in the user application if they configure a higher log level (e.g. error+) for the streamlit namespace only, since we recently merged in https://github.com/streamlit/streamlit/pull/6377 ?

kajarenc commented 1 year ago

@sfc-gh-jcarroll

What's the behavior of caching if the app is executed in this way?

If we call python app.py instead of streamlit run app.py or python -m streamlit.web.cli run app.py (which is the same as calling streamlit run app.py) so if your script invoked directly from the interpreter and runtime is missing, we use just in-memory cache manager as a replacement for cache manager from runtime and log a warning.

@maxibor could you please share how you run streamlit app using streamlit.web.cli, it should not log a warning in that case.

If you run it just as python app.py, this should help to avoid that particular warning.

import streamlit as st
import logging

logging.getLogger("streamlit.runtime.caching.cache_data_api").setLevel(logging.ERROR)

@st.cache_data
def foo(x):
    return x**2

st.write("HELLO FROM WRITE!")

print("HELLO FROM PRINT!")

nicolasdaviaud commented 1 year ago

I'm getting the same warning when launching streamlit. In my case, I'm calling

ctx.invoke(streamlit.web.cli.main_run, target="path_to_my_file.py", server_port="something")

I'm doing it this way because I need to run some initialisation before actually launching the webapp (auth protocols, on disk caching, etc)

maxibor commented 1 year ago

@kajarenc This is how I run the streamlit app (it's part of CLI software, as a subcommand named "viewer").

def run_app(tables=None, verbose=False):
    """
    Run the AMDirT interactive filtering application

    Args:
        tables (str): path to JSON file listing AncientMetagenomeDir tables
    """
    if not warnings:
        warnings.filterwarnings("ignore")
    directory = Path(__file__).parent.resolve()
    app = "streamlit.py"
    if tables is None:
        config_path = get_json_path()
    else:
        config_path = tables
    app_path = f"{directory}/{app}"

    sys.argv = [
        "streamlit", 
        "run", 
        app_path, 
        "--", 
        "--config", 
        config_path
    ]
    logger.info("\n[AMDirT] To close app, press on your keyboard: ctrl+c\n")
    sys.exit(stcli.main())

https://github.com/SPAAM-community/AMDirT/blob/961e49998d85a87a9a716414246a3a02b467f91b/AMDirT/viewer/__init__.py#L10-L36

maxibor commented 7 months ago

@kajarenc Is there a plan to provide a finer control of the cache warnings ? Otherwise, should I just monkeypatch the get_storage_manager method to silence the warning ?

import streamlit as st

def monkeypatch_get_storage_manager():
    if st.runtime.exists():
        return st.runtime.get_instance().cache_storage_manager
    else:
        # When running in "raw mode", we can't access the CacheStorageManager,
        # so we're falling back to InMemoryCache.
        # _LOGGER.warning("No runtime found, using MemoryCacheStorageManager")
        return st.runtime.caching.storage.dummy_cache_storage.MemoryCacheStorageManager()

st.runtime.caching._data_caches.get_storage_manager = monkeypatch_get_storage_manager

streamlit / streamlit