Closed FilipDem closed 2 years ago
I also did the test outside Domoticz and got the same problem after a while... See python script: test_asyncio.py.txt
Hereunder you can find the output, including the error. And also the error is persistent at every subsequent call in the loop. You can use the script (with your credentials) to test/debug. I am wondering whether it has something to do with authentication codes that need to be renewed after a while...
filip@Ararat:~/domoticz/plugins/Bmw$ python3 test_asyncio.py
Login done! MyBMW object: MyBMWAccount(username='filip.xxx@xxx.com', region=<Regions.REST_OF_WORLD: 2>, mybmw_client_config=MyBMWClientConfiguration(authentication=MyBMWAuthentication(username='filip.xxx@xxx.com', password='xxxx', region=<Regions.REST_OF_WORLD: 2>, token=None, expires_at=None, _lock=None), log_response_path=None), observer_position=None, vehicles=[])
Login successful! Car 518d (VIN:WBAJF11030CD97578) found!
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7f2c576a1bb0>
StatusUpdate TRACEBACK: Traceback (most recent call last):
File "/home/filip/domoticz/plugins/Bmw/test_asyncio.py", line 31, in <module>
asyncio.run(myBMW.get_vehicles())
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/account.py", line 78, in get_vehicles
vehicles_responses: List[httpx.Response] = await asyncio.gather(*vehicles_tasks)
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1729, in get
return await self.request(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1506, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1593, in send
response = await self._send_handling_auth(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1621, in _send_handling_auth
response = await self._send_handling_redirects(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1656, in _send_handling_redirects
await hook(request)
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/client.py", line 39, in update_request_header
request.headers["authorization"] = await self.config.authentication.get_authentication()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 46, in get_authentication
await self.login()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 72, in login
async with self._lock:
File "/usr/lib/python3.9/asyncio/locks.py", line 14, in __aenter__
await self.acquire()
File "/usr/lib/python3.9/asyncio/locks.py", line 120, in acquire
await fut
RuntimeError: Task <Task pending name='Task-222' coro=<AsyncClient.get() running at /usr/local/lib/python3.9/dist-packages/httpx/_client.py:1729> cb=[gather.<locals>._done_callback() at /usr/lib/python3.9/asyncio/tasks.py:766]> got Future <Future pending> attached to a different loop
StatusUpdate TRACEBACK: Traceback (most recent call last):
File "/home/filip/domoticz/plugins/Bmw/test_asyncio.py", line 31, in <module>
asyncio.run(myBMW.get_vehicles())
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/account.py", line 78, in get_vehicles
vehicles_responses: List[httpx.Response] = await asyncio.gather(*vehicles_tasks)
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1729, in get
return await self.request(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1506, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1593, in send
response = await self._send_handling_auth(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1621, in _send_handling_auth
response = await self._send_handling_redirects(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1656, in _send_handling_redirects
await hook(request)
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/client.py", line 39, in update_request_header
request.headers["authorization"] = await self.config.authentication.get_authentication()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 46, in get_authentication
await self.login()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 72, in login
async with self._lock:
File "/usr/lib/python3.9/asyncio/locks.py", line 14, in __aenter__
await self.acquire()
File "/usr/lib/python3.9/asyncio/locks.py", line 120, in acquire
await fut
RuntimeError: Task <Task pending name='Task-254' coro=<AsyncClient.get() running at /usr/local/lib/python3.9/dist-packages/httpx/_client.py:1729> cb=[gather.<locals>._done_callback() at /usr/lib/python3.9/asyncio/tasks.py:766]> got Future <Future pending> attached to a different loop
If I just put an new MyBMWAccount('filip.xxx@xxx.com', 'xxx', get_region_from_name('rest_of_world'))
statement in the loop when the error occurs to see if afterwards the loop goes on and it is... So I think really that it is linked with a new login that has to be done...
So I have a workaround, but there is still a bug... Always happy to learn when it would be solved so I can remove the workaround.
Update part of the test code (supplied above)
while myBMW:
t = time.localtime()
print ('{} UPDATE ---------------------------------------------------'.format(time.strftime("%H:%M:%S", t)))
myVehicle = None
try:
asyncio.run(myBMW.get_vehicles())
except:
import traceback
print('StatusUpdate TRACEBACK: {}'.format(traceback.format_exc()))
myBMW = MyBMWAccount('filip.xxx@xxx.com', 'xxx', get_region_from_name('rest_of_world'))
print('Login done! MyBMW object: {}'.format(myBMW))
else:
myVehicle = myBMW.get_vehicle('WBAJF11030CD97578')
if myVehicle:
print('Car {} found after update!'.format(myVehicle.name))
print('Mileage: {} {}'.format(myVehicle.mileage[0], myVehicle.mileage[1]))
print('Car {} status: {}'.format(myVehicle.name, myVehicle))
else:
print('Error VIN {} not found for user.'.format('WBAJF11030CD97578'))
time.sleep(240)
And the output to show the loop is back...
12:02:25 UPDATE ---------------------------------------------------
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7fb36b478220>
12:06:27 UPDATE ---------------------------------------------------
StatusUpdate TRACEBACK: Traceback (most recent call last):
File "/home/filip/domoticz/plugins/Bmw/test_asyncio.py", line 35, in <module>
asyncio.run(myBMW.get_vehicles())
File "/usr/lib/python3.9/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.9/asyncio/base_events.py", line 642, in run_until_complete
return future.result()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/account.py", line 78, in get_vehicles
vehicles_responses: List[httpx.Response] = await asyncio.gather(*vehicles_tasks)
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1729, in get
return await self.request(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1506, in request
return await self.send(request, auth=auth, follow_redirects=follow_redirects)
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1593, in send
response = await self._send_handling_auth(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1621, in _send_handling_auth
response = await self._send_handling_redirects(
File "/usr/local/lib/python3.9/dist-packages/httpx/_client.py", line 1656, in _send_handling_redirects
await hook(request)
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/client.py", line 39, in update_request_header
request.headers["authorization"] = await self.config.authentication.get_authentication()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 46, in get_authentication
await self.login()
File "/usr/local/lib/python3.9/dist-packages/bimmer_connected/api/authentication.py", line 72, in login
async with self._lock:
File "/usr/lib/python3.9/asyncio/locks.py", line 14, in __aenter__
await self.acquire()
File "/usr/lib/python3.9/asyncio/locks.py", line 120, in acquire
await fut
RuntimeError: Task <Task pending name='Task-134' coro=<AsyncClient.get() running at /usr/local/lib/python3.9/dist-packages/httpx/_client.py:1729> cb=[gather.<locals>._done_callback() at /usr/lib/python3.9/asyncio/tasks.py:766]> got Future <Future pending> attached to a different loop
Login done! MyBMW object: MyBMWAccount(username='filip.xxx@xxx.com', region=<Regions.REST_OF_WORLD: 2>, mybmw_client_config=MyBMWClientConfiguration(authentication=MyBMWAuthentication(username='filip.xxx@xxx.com', password='xxx', region=<Regions.REST_OF_WORLD: 2>, token=None, expires_at=None, _lock=None), log_response_path=None), observer_position=None, vehicles=[])
12:10:27 UPDATE ---------------------------------------------------
Car 518d found after update!
Mileage: 30265 km
Car 518d status: <bimmer_connected.vehicle.vehicle.MyBMWVehicle object at 0x7fb369a7ee50>
Can you please try to edit api/authentication.py#L66 to:
if not self._lock or self._lock._loop != asyncio.get_running_loop():
What happens here (apparently, not a full expert either):
async
functions are supposed to be run in one single event loop, i.e. started by asyncio.run()
. If you're using almost only async functions, this is fine.
However you have to do multiple calls to asyncio.run()
due to the domoticz architecture.
When asyncio.run()
finishes, the event loop is closed and next time a new one is created. As this event loop is also stored in every asyncio
object, the loops are not the same, giving this error.
Hi, I let my test scrip run 2 times this weekend during several hours. So far all is still OK. If you have the occasion to generates once a new beta, I will install it also on my Domoticz server (to test it during several days). Let me know when the new beta would be available. Kr, F
Hi Filip, please try https://github.com/bimmerconnected/bimmer_connected/releases/tag/v0.9.0b9
Mmmm... When installing v0.9.0.b9 or b10, I got new errors (up to b8 it works fine)... But it is more nuanced than this...
I have 2 installations: on my Ubuntu 22.04 (test environment) with Python 3.10.4, all seems to go well (!! not yet tested on long duration). However on my RaspberryPi (Buster), there is Python 3.7 and there I have a strange runtime error...
I tried to investigate a bit but cannot find out exactly based on the stack trace (see below).
I saw that since v0.9.0b9 there were initial changes done for getting the event loop (get_event_loop instead
of get_running_loop
and also changes in MyBMWAuthentication(Authentication)
). I cannot simulate the problem outside Domoticz (works with my test script both on Ubuntu and RPi).
Any idea?
2022-05-15 13:27:15.090 BMW518: (BMW518) Handling task: Login.
2022-05-15 13:27:15.091 BMW518: (BMW518) Login error: There is no current event loop in thread 'QueueThread'.
2022-05-15 13:27:15.094 BMW518: (BMW518) Login error TRACEBACK: Traceback (most recent call last):
2022-05-15 13:27:15.094 File "/home/pi/domoticz/plugins/Bmw/plugin.py", line 263, in handleTasks
2022-05-15 13:27:15.094 self.myBMW = MyBMWAccount(Parameters['Mode2'], Parameters['Mode3'], get_region_from_name(self.region))
2022-05-15 13:27:15.094 File "<string>", line 7, in __init__
2022-05-15 13:27:15.094 File "/usr/local/lib/python3.7/dist-packages/bimmer_connected/account.py", line 53, in __post_init__
2022-05-15 13:27:15.094 MyBMWAuthentication(self.username, password, self.region),
2022-05-15 13:27:15.094 File "<string>", line 8, in __init__
2022-05-15 13:27:15.094 File "/usr/lib/python3.7/asyncio/locks.py", line 161, in __init__
2022-05-15 13:27:15.094 self._loop = events.get_event_loop()
2022-05-15 13:27:15.094 File "/usr/lib/python3.7/asyncio/events.py", line 644, in get_event_loop
2022-05-15 13:27:15.094 % threading.current_thread().name)
2022-05-15 13:27:15.094 RuntimeError: There is no current event loop in thread 'QueueThread'.
2022-05-15 13:27:15.094
I just looked through some of the Domoticz examples and the documentation. Is there any specific reason why you are using the multi-threaded approach? I don't think it is needed to be honest.
What I think is needed generally with >0.9.0:
onStart
, just assign self.myBMW = MyBMWAccount(...)
onHeartbeat
, you don't need to check if self.myBMW
is empty, as it will always be there through the init from onStart
handleTasks
Login
action anymore. This is automatically done by get_vehicles()
, included refresh tokens etc.asyncio.run(self.myBMW.get_vehicles())
If all of this in running in your plugin's main thread, then asyncio.run()
should always be able to find/create the correct loop.
Problem is that Domoticz works also with events and callbacks. In the documentation it is clearly stated that "Waiting or sleeping in Domoticz callbacks. Plugin callbacks are single threaded so the whole plugin system will wait." And a callback (eg onHeartbeat) should not take more than 10s to avoid Domoticz generates error messages. Therefor they recommend to use a separate thread as shown in the example "Mutli-Threaded".
On the other hand, there should not be any reason why it should not work in a separate thread (as it works with 0.9.0b6).
I changed the authentication yesterday (where the async. Lock()
that caused the trouble was used) and removed it.
Can you please try b13
?
Hi, First of all a big thanks for your help! Again there seems to be a problem with the loop... Yesterday I made a version where I removed the threading. It runs as such if all goes well, however I fear problems in case of the communication takes more time, especially for the remote services that take much more time... And this will block the plugin. It is pity because even with the threading in worked very well a while ago. However if I got errors in the plugin (embedded python), in my opinion it hides a real problem as there is no reason why it shouldn't work.
Traceback (most recent call last):
File "/home/pi/domoticz/plugins/Bmw/plugin.py", line 263, in handleTasks
self.myBMW = MyBMWAccount(Parameters['Mode2'], Parameters['Mode3'], get_region_from_name(self.region))
File "<string>", line 7, in __init__
File "/usr/local/lib/python3.7/dist-packages/bimmer_connected/account.py", line 53, in __post_init__
MyBMWAuthentication(self.username, password, self.region),
File "<string>", line 8, in __init__
File "/usr/lib/python3.7/asyncio/locks.py", line 161, in __init__
self._loop = events.get_event_loop()
File "/usr/lib/python3.7/asyncio/events.py", line 644, in get_event_loop
% threading.current_thread().name)
RuntimeError: There is no current event loop in thread 'QueueThread'.
While it would be very interesting to actually solve the issue I think you can ignore it.
I removed it as it created some concurrency in the login process that may or may not also have caused login issues for China (#428, end of thread).
So it removed completely now and it should be working in your plug in with threading.
Ignore this, we still need it and I have to add it back in...
Hi Filip, please try b15
. This includes again the "old" implementation of the async.Lock
that worked in b8
.
It crashes on response: httpx.Response = yield request
on RPi with Python 3.7...
Both problem when using threading or not in my plugin. Plugin without threading and b13 works... b14 gives the same error.
2022-05-21 16:46:29.601 Error: BMW518: (Bmw) failed to load 'plugin.py', Python Path used was '/home/pi/domoticz/plugins/Bmw/:/usr/lib/python37.zip:/usr/lib/python3.7:/usr/lib/python3.7/lib-dynload:/usr/local/lib/python3.7/dist-packages:/usr/lib/python3/dist-packages:/usr/lib/python3.7/dist-packages'.
2022-05-21 16:46:29.601 Error: BMW518: (BMW518) Module Import failed, exception: 'SyntaxError'
2022-05-21 16:46:29.601 Error: BMW518: (BMW518) Import detail: File: /usr/local/lib/python3.7/dist-packages/bimmer_connected/api/authentication.py, Line: 79, offset: 40
2022-05-21 16:46:29.601 Error: BMW518: (BMW518) Error Line ' response: httpx.Response = yield request
2022-05-21 16:46:29.601 '
Sorry, stumbled across this too late. Apparently in Python<=3.7, we need a yield request
to be put in brackets.
response: httpx.Response = (yield request)
It's included in b17
.
Top! This did the trick... I installed it and started already up! So a good step in the good direction! I will now let it run during some days to see if all behaves correctly. Thanks again!
Just for your information and to say thank you again! Everything is still working like a charm since yesterday evening...
Great to hear! Just for clarification: Would an issue have shown otherwise? Thinking it's getting close to 0.9.0 stable and I'd like to release that soon-ish.
I cannot be 100% sure yet, but I suppose I would have seen already if there would be an issue... It could still happen in case something goes wrong, ie falling in error cases like communication timeouts, logins that expire, ...
Closing this as I hope its fixed. If not, please reopen this!
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.
Describe the issue
Hi Rikroe, I decided to open the error here instead of continuing on the discussion #422. I am using the version 0.9.0.0b6 integrated in Domoticz. All works fine, however systematically after 1-2 hours, I got the following errors. I retrieve the car status in loop by using the following (cleaned) code in a loop:
The
get_vehicles()
generates at a certain moment the error below (traceback). And the same error appears at every call from then on. When restarting completely the plugin all works fine again.I will now try to simulate the same problem by a dedicated script outside Domoticz. test_asyncio.py.txt
Any idea so far?
Expected behavior
-
Which Home Assistant version are you using?
(used outside HA)
What was the last working version of Home Assistant Core?
No response
What is your region?
Rest of world
ConnectedDrive website
Number of cars
Output of bimmer_connected fingerprint
No response
Anything in the logs that might be useful for us?
No response
Additional information
No response