search

filcole / pycarwings2

Python library for interacting with the NissanConnect EV (aka CARWINGS) telematics service.
Apache License 2.0
33 stars 20 forks source link

Login seems to be failing #13

Closed irekzielinski closed 5 years ago

irekzielinski commented 5 years ago

I think Nissan changed something (along with new app store app). Script fails on login.

Does anyone see this issue (i'm in the uk). Thanks, irek

oskretc commented 5 years ago

Yeap, same here... Norway

irekzielinski commented 5 years ago

Error message is: `ERROR:pycarwings2.pycarwings2:Invalid JSON returned Traceback (most recent call last): File "/usr/local/lib/python3.7/site-packages/pycarwings2/pycarwings2.py", line 151, in _request j = json.loads(response.text) File "/usr/local/lib/python3.7/json/init.py", line 348, in loads return _default_decoder.decode(s) File "/usr/local/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/local/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "get-leaf-battery.py", line 56, in l = s.get_leaf() File "/usr/local/lib/python3.7/site-packages/pycarwings2/pycarwings2.py", line 209, in get_leaf self.connect() File "/usr/local/lib/python3.7/site-packages/pycarwings2/pycarwings2.py", line 171, in connect "lg": "en-US", File "/usr/local/lib/python3.7/site-packages/pycarwings2/pycarwings2.py", line 154, in _request raise CarwingsError pycarwings2.pycarwings2.CarwingsError`

filcole commented 5 years ago

Might have been also caught and fixed by @ubarni here https://github.com/ubarni/pycarwings2/commit/078b7106b7e473c3454c5f9694894fac6ce15a47

irekzielinski commented 5 years ago

I can confirm that ubarni fork works for me. Simply applying his modification to this fork does not seem to work correctly (login now happens, but I could not get the battery update request to work).

jrester commented 5 years ago

Applying his modifications works for me and I get the correct battery percentage.

irekzielinski commented 5 years ago

Applying his modifications works for me and I get the correct battery percentage.

Are you sure values reported by API is updated? I have been charging car and API was returning old values (was not showing SOC increases). As soon as I used Nissan android app to get latest values - it was picked up by the API. My suspicion is that API fails to request battery change update (and just reads whatever was the last state saved in Nissan servers).

Another thing is that this value is not changing: info.answer["BatteryStatusRecords"]["NotificationDateAndTime"] that would again suggest that API is not actually triggering the SOC read from the car

jrester commented 5 years ago

Are you sure values reported by API is updated?

Yea, I tried multiple times and values were always the same as in the APP (Of course I first requested the value via the python API and then over the APP to verify the API was really updating)

I have been charging car and API was returning old values (was not showing SOC increases). Maybe you forget calling Leaf#request_update?

At first the old battery percentage is returned when calling Leaf#get_latest_battery_status but after calling Leaf#request_update the battery percentage and NotificationDateAndTime gets updated (without opening the APP between the calls...).

irekzielinski commented 5 years ago

I'm using this script without any modifications (so request_update is called): https://github.com/filcole/pycarwings2/blob/master/examples/get-leaf-info.py

Funny enough - got one correct update, but then subsequent calls were returning stale NotificationDateAndTime..... can you share your script or do you also use get-leaf-info.py?

jrester commented 5 years ago

Was just typing in the repl

>>> status_0 = leaf.get_latest_battery_status()
>>> status_0.battery_percent
8.333333333333334
>>> key = leaf.request_update()
>>> status_1 = leaf.get_latest_battery_status()
>>> status_1.battery_percent
8.333333333333334
>>> status_2 = leaf.get_status_from_update(key)
>>> status_2.battery_percent
33.333333333333336
>>> status_3 = leaf.get_latest_battery_status()
>>> status_3.battery_percent
33.333333333333336

Funny enough - got one correct update, but then subsequent calls were returning stale NotificationDateAndTime.....

I think its an issue with the get-leaf-info.py script. It does not wait for the API to actually retrieve the requested updated value, but just gets the old battery status. A simple solution could be to wait until we get an update status and return the new battery status:

class Leaf:
   def update_battery(self, wait_time=1):
       key = self.request_update()
       status = self.get_status_from_update(key)
       while status is None:
           sleep(wait_time)
           status = self.get_status_from_update(key)
      return status

This works because get_status_from_update returns data again (in the source and in get-leaf-info.py it says it always returs None). It only returns None if the update is not finished yet.

irekzielinski commented 5 years ago

thanks for your advice @jrester - unfortunately whatever I do I get stale data from carwings (when officail app works without any problems).

get_status_from_update(key) - always retuns on object for me (it's never None) it returns value very quickly (stale value). Tried already:

I tied to capture and decrypt iOS app traffic using Fiddler - but Nissan app refused to login (I suspect it's not happy with Fiddler HTTPS certificate).

Does anyone else have the same problem?

jrester commented 5 years ago

I tied to capture and decrypt iOS app traffic using Fiddler - but Nissan app refused to login (I suspect it's not happy with Fiddler HTTPS certificate).

I tried with mitmproxy and the Android app but it doesn't work as well. And the Nissan App isn't starting when USB debugging is enabled or a custom rom is installed. It just opens an error page in my Webbrowser. I think we have to decompile the app and investigate the source code. Maybe we can find a hint why it's not working.

jrester commented 5 years ago

What could be an issue is that we are using this URL https://gdcportalgw.its-mo.com/api_v190426_NE/gdc/ which seems to be the URL for Europe. So other regions like Australia probably won't be working with this url and would rather need something like https://gdcportalgw.its-mo.com/api_v190426_NMA/gdc/. But I don't think this is related to your issue because UK still counts a Europe.

The App also never uses the BatteryStatusCheckRequest.php and BatteryStatusCheckResultRequest.php URLs. They are defined but never instantiated. The App instead uses BatteryStatusCheckForSPAppsRequest.php and BatteryStatusCheckForSPAppsResultRequest.php. Maybe it works if you replace the URLs for request_update and get_status_from_update.

irekzielinski commented 5 years ago

good stuff @jrester - I will give it a try. On android there is a third party app that has been updated called QuickLeaf. https://play.google.com/store/apps/details?id=no.joymyr.quickleaf&hl=en_GB

I dropped an email to the author - maybe he/she will be able to share some tips of how to trigger the update.

Straight replacement of BatteryStatusCheckRequest.php and BatteryStatusCheckResultRequest.php with "ForSPApps" versions appear to work on the surface (very similar response to orginal API, but update still does not seem to be triggered).

Requesting update DEBUG:pycarwings2.pycarwings2:invoking carwings API: https://gdcportalgw.its-mo.com/api_v190426_NE/gdc/BatteryStatusCheckForSPAppsRequest.php DEBUG:pycarwings2.pycarwings2:params: { "DCMID": "ABC.....", "RegionCode": "NE", "UserId": "", "VIN": "ABc.....", "custom_sessionid": "ubuyftxg+mf3ul+rJ1GmqAqoIKMQQVwF9pmjUBLGLc2c/XAof29D2VMjjNFIQ6/kpbuxUAFMGG8YdRwztpBkli7UslbHsqVKc9CvzTzks7vg==", "initial_app_str": "9s5rfKVuMrT03RtzajWNcA", "lg": "en_GB", "tz": "Etc/UTC" } DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): gdcportalgw.its-mo.com:443 DEBUG:urllib3.connectionpool:https://gdcportalgw.its-mo.com:443 "POST /api_v190426_NE/gdc/BatteryStatusCheckForSPAppsRequest.php HTTP/1.1" 200 None DEBUG:pycarwings2.pycarwings2:Response HTTP Status Code: 200 DEBUG:pycarwings2.pycarwings2:Response HTTP Response Body: b'{"status":200,"userId":"irekzielinski","vin":"ABC....","resultKey":"MVAYWY32fR8fdrjjWqXSfbnQ0GG41hqx2GQdTVV2aYKkiG1"}'

Here is a response to BatteryStatusCheckForSPAppsResultRequest.php DEBUG:pycarwings2.pycarwings2:invoking carwings API: https://gdcportalgw.its-mo.com/api_v190426_NE/gdc/BatteryStatusCheckForSPAppsResultRequest.php DEBUG:pycarwings2.pycarwings2:params: { "DCMID": "201399075664", "RegionCode": "NE", "VIN": "ABc....", "custom_sessionid": "bIYE2kx1UCyYggsRuIqwewqes4bJIrywFHZJTbzsSwGKTCYtgOB6GAuvirZKwduQZRC348ldlQ0w66v+EAGHttus2KluoLOm8qYUIfQQ==", "initial_app_str": "9s5rfKVuMrT03RtzajWNcA", "lg": "en_GB", "resultKey": "KEylA7gI7NmCrqewwqqwewqfoQqOyx7VlJ73ChZzzK", "tz": "Etc/UTC" } DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): gdcportalgw.its-mo.com:443 DEBUG:urllib3.connectionpool:https://gdcportalgw.its-mo.com:443 "POST /api_v190426_NE/gdc/BatteryStatusCheckForSPAppsResultRequest.php HTTP/1.1" 200 None DEBUG:pycarwings2.pycarwings2:Response HTTP Status Code: 200 DEBUG:pycarwings2.pycarwings2:Response HTTP Response Body: b'{"status":200,"responseFlag":"1","operation_date_and_time":"2019-08-11 16:37:37","target_date":"2019-08-11 16:37:37","notification_date_and_time":"2019-08-11 16:37:37","cruising_range_ac_on":"52416.0","cruising_range_ac_off":"54080.0","plugin_state":"CONNECTED","charge_mode":"NOT_CHARGING","charging":"NO","battery_capacity":"12","battery_degradation":"5","time_required_to_full_hours":"13","time_required_to_full_minutes":"30","time_required_to_full_200_hours":"8","time_required_to_full_200_minutes":"0","time_required_to_full_200_6kw_hours":null,"time_required_to_full_200_6kw_minutes":null}' INFO:pycarwings2.responses:CarwingsBatteryStatusResponse has data. Protocol change?

In my case timestamp of 16:37:37 - mens stale data.

jrester commented 5 years ago

I dropped an email to the author - maybe he/she will be able to share some tips of how to trigger the update.

Took a look at the App and it seems like it is using the same old, none ForSPApps Urls, so if this App is working for you it's probably not a problem with the URLs but rather how the python API works.

joymyr commented 5 years ago

Hi

For my QuickLeaf Android app I make a request to https://gdcportalgw.its-mo.com/api_v190426_NE/gdc/BatteryStatusCheckRequest.php. Then I start polling https://gdcportalgw.its-mo.com/api_v190426_NE/gdc/BatteryStatusCheckResultRequest.php until I get a result. I guess as @jrester mentioned, the difference is the "ForSPApps" which I don't know what is. The main changes for the Nissan API was just the base-url and some strings/keys related to login (+login seems more unstable, so I had to add some retry logic).

BenWoodford commented 5 years ago

@filcole when do you think you'll be able to get these changes done and published? The new way of HASS handling requirements makes it very difficult to use unpublished code, especially inside a Docker container.

irekzielinski commented 5 years ago

Guys - I think I found my problem, took a LOT of trial and error. Thank you to @joymyr for confirming how QuickLeaf works - this allowed me to narrow down the issues.

Solution that works for me: If there is no sleep/delay commands commands, script will almost always return stale value and time-out waiting. Seems to be some kind of race condition on Nissan servers maybe?

This explains why it sometimes worked for me in python debugger - step-by-step execution was allowing for this extra time.

Here is my script:

print ("Login")
l = s.get_leaf()

#this sleep is critical, without it we seem to be getting state data!!!!                                                                                                                 
time.sleep(1)

print("Get latest bet status")
leaf_info = l.get_latest_battery_status()
start_date = leaf_info.answer["BatteryStatusRecords"]["OperationDateAndTime"]
print("Last timestamp:", start_date)

#this sleep is critical, without it we seem to be getting state data!!!!                                                                                                                 
time.sleep(1)

print ("Requesting Update")
result_key = l.request_update()
cnt = 0

while True:
        cnt = cnt + 1
        if(cnt > 15):
                print("Time out waiting for response");
                exit()

        print(".")
        time.sleep(5)  # sleep 60 seconds to give request time to process                                                                                                                

        battery_status = l.get_status_from_update(result_key)                                                                                                                                   

        latest_leaf_info = l.get_latest_battery_status()
        latest_date = latest_leaf_info.answer["BatteryStatusRecords"]["OperationDateAndTime"]
        if (latest_date != start_date):
                print("Got update, current timestamp:", latest_date)
                print ("-->")
                print_info(latest_leaf_info)
                break
filcole commented 5 years ago

Thank you very much to all that contributed to getting this fixed. Apologies that it took so long to be able to spend some time on it.

ubarni commented 5 years ago

Thanks filcole. But i got this error: pip install git+https://github.com/filcole/pycarwings2.git

Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-NKVBMy-build/

jrester commented 5 years ago

Are you running python 2? I think python 3 is required to install it.

filcole commented 5 years ago

Hi @ubarni, this is a python3 package. It's published to PyPi, so preferred installing method is pip install pycarwings2.

bwduncan commented 3 years ago

Hi all,

I have been trying to get pycarwings2 to give me reliable updates within Home Assistant. With @irekzielinski 's trial and error, I too believe I have code that works. The sleeps are critical, as is checking the timestamp reported by the Nissan servers. If get_status_from_update returns True but the timestamp from get_latest_battery_status did not change, then the update was in fact not successful and we should try again (request_update, then poll get_status_from_update every ~30s).

Home Assistant seems like a pretty daunting project to contribute to, but if anyone is in a position to test and/or review this PR it could be useful: https://github.com/home-assistant/core/pull/44826

Bruce