Closed chemelli74 closed 1 month ago
sessions should be cached for the life of use so if this is a major issue the session should be kept around for a longer time. really botocore needs to speed this up on their end because it's way too slow
Until then we have a fix anyway. Maybe we add a comment about that.
I'm hesitant of doing this as run_in_executor
triggers a thread pool to be created (if you haven't called it before) which can cause overhead. Further the executor should be configurable..this really opens a can of worms.
It does looks like load_service_model
calls are cached, but it does do blocking I/O in the event loop the first time its created. Ideally we would only call it in the executor the first time it needs to be created.
run_in_executor
being called with None
(default executor) seems fine to me as we do it all over the place in aiohttp
and that is the prescribed way to run blocking code https://docs.python.org/3/library/asyncio-dev.html#running-blocking-code
Any feedback @thehesiod ? Thank you in advance
so this seems like an issue with the aws component, why doesn't it do the session/client creation during async_setup
instead? That seems like the correct fix no?
like i was saying, it should cache the session/client there, or if you don't want to hold onto the client, perhaps just pre-warm up the botocore cache
actually, this is invalid as botocore is not thread safe, with this you could have two requests, running in two different threads, trying to initialize the cache at the same time. closing
@chemelli74 Thanks for trying to fix this. This problem must be solved eventually (unless the intent is WONTFIX). Please open an issue instead to track this in the long term. Thanks.
ya let me know if I missed something, I truly believe this is a botocore issue. The fact they're reading a ton of JSON files is unsustainable. probably should be done on the fly per service when you use it...and even then should be in a post-processed state, like their build should generate python code based on those json files.
so I looked a little into this, it seems like they use an instance_cache
which basically just sets an attribute on the class, so presumably worst case scenario with this fix is you'd initialize the cache multiple times. However I really want to avoid adding threads to this library if possible. Looks like there are some async file alternatives like https://github.com/qweeze/uring_file for debian, I'd be open to something like that.
What about pathlib ? Is used all over
that uses aiofiles which just uses a threadpool again: https://github.com/Tinche/aiofiles/tree/main/src/aiofiles
i guess at the aiofiles level though it doesn't matter. But it sounds like there's a better solution
i guess at the aiofiles level though it doesn't matter. But it sounds like there's a better solution
If you point me towards the solution you prefer, I'll try to work on it. I really would like to fix this issue and it's effect on Home Assistant.
EDIT: Opened issue on botocore
repo
Here the official answer from botocore repo: https://github.com/boto/botocore/issues/3222
What can we do to progress then ?
edit: sorry, this is basically what @thehesiod suggested https://github.com/aio-libs/aiobotocore/pull/1120#issuecomment-2187180231. I also think it's the obvious solution for that HomeAssistant specific issue.
This might be a crazy idea, but if
then
This would be a fix over at HomeAssistant. What do you think?
This doesn’t seem like a Home Assistant specific issue. Any asyncio application is going to have its event loop blocked. The only difference is that Home Assistant is aware of its loop being blocked where-as another application may not be able to detect it.
botocore (or us?) could provide some sort of synchronuous helper function to pre-populate the cache. Home Assistant or anyone else could then choose to run it as part of their setup routine, e.g. via asyncio.to_thread() and no one else would be affected. Does that make sense?
Btw, there are Python platforms that lack multithreading support. That's why I think @thehesiod has a point in not resorting to helper threads in this project unless absolutely necessary.
this isn't an aiobotocore problem I don't think, but a botocore problem, or do you need to go through an async call to get to the cache warmup? secondly they can already do this themselves without the need for a helper
this isn't an aiobotocore problem I don't think, but a botocore problem.
Why you think so ?
This is a async issue and botocore
is sync.
The main point of aiobotocore
is to transform botocore to async, so IMHO it's a issue in this library
.
I suppose if we find a good async file API to use then it would be an async problem. but then that wouldn't require an init function. in terms of an init function that would equally apply to sync, however you'll then potentially run into locking issues
@thehesiod how can we progress ? I really want to find a fix
@chemelli74 suggest doing research of a good async API for aiobotocore to use. This will be a huge project though. Furthermore there are better ways to warm-up aiobotocore, if homeassistant wants to do that I already gave some suggestions like: https://github.com/aio-libs/aiobotocore/pull/1120#issuecomment-2187180231. They should init and keep the aioboto clients around instead of creating them on the fly
this is really a solved issue, we due the same thing, like in aiohttp server using the on_startup
callback to init your resources
Description of Change
Fix detected blocking call inside the event loop. Reference HomeAssistant latest build:
Assumptions
none
Checklist for All Submissions
Checklist when updating botocore and/or aiohttp versions