Open Hank-Yan opened 1 month ago
hi @Hank-Yan - can you clarify which version of prefect you are using? ideally, please include the output of prefect version
Hi @zzstoatzz my prefect info is below
My prefect info : Version: 3.0.3 API version: 0.8.4 Python version: 3.11.7 Git commit: d9c51bc3 Built: Fri, Sep 20, 2024 8:52 AM OS/Arch: linux/x86_64 Profile: local Server type: server Pydantic version: 2.9.0b2 Integrations: prefect-redis: 0.2.0
prefect config view result:
PREFECT_PROFILE='local'
PREFECT_API_KEY='****' (from profile)
PREFECT_API_URL='http://ip:port/api' (from profile)
PREFECT_DEFAULT_RESULT_STORAGE_BLOCK='redisstoragecontainer/redis-storage-block' (from profile)
PREFECT_RESULTS_DEFAULT_SERIALIZER='json' (from profile)
redis-storage-block source code is: from prefect.blocks import redis redis.RedisStorageContainer.from_host( host="127.0.0.1", port=6379, password="default").save("redis-storage-block", overwrite=True)
Describe the current behavior
I understand that all result_storage storage media files require third-party management. Currently, with Prefect’s behavior, when the cache expires, the files stored in result_storage are not automatically cleared, which is quite confusing.
This is especially concerning when Redis is used as storage since Redis memory usage grows rapidly. Should we add a default expiration time or a global expiration time as a temporary solution?
Describe the proposed behavior
Ideally, when the cache expires, the files should be automatically deleted, which would be the best outcome.
For the case of using Redis for result_storage, I would suggest first adding a global expiration time or a default to expiration time serve as a transitional solution.
Thank you for your attention.
Example Use
No response
Additional context
No response