itchannel / fordpass-ha

Fordpass integration for Home Assistant
301 stars 53 forks source link

403 Client Error: Forbidden for url: https://usapi.cv.ford.com/api/vehicles/v5/MYVIN/status?lrdt=01-01-1970+00%3A00%3A00 #303

Closed philipgo closed 11 months ago

philipgo commented 11 months ago

Hello,

since today I get the error message "403 Client Error: Forbidden for url: https://usapi.cv.ford.com/api/vehicles/v5/MYVIN/status?lrdt=01-01-1970+00%3A00%3A00"

I am on version 1.52

Thank you for your great work!

AndreiGavriliu commented 11 months ago

Also having the same issue. I enabled debugging and found the following info. maybe it helps

2023-10-09 13:24:23.726 DEBUG (MainThread) [custom_components.fordpass] 900
2023-10-09 13:24:23.727 DEBUG (MainThread) [custom_components.fordpass] username
2023-10-09 13:24:23.727 DEBUG (MainThread) [custom_components.fordpass] password
2023-10-09 13:24:23.727 DEBUG (MainThread) [custom_components.fordpass] region
2023-10-09 13:24:23.727 DEBUG (MainThread) [custom_components.fordpass] vin
2023-10-09 13:24:23.727 DEBUG (MainThread) [custom_components.fordpass] UK&Europe
2023-10-09 13:24:23.728 DEBUG (MainThread) [custom_components.fordpass.fordpass_new] /config/custom_components/fordpass/[REDACTED]_fordpass_token.txt
2023-10-09 13:24:23.733 DEBUG (SyncWorker_0) [custom_components.fordpass.fordpass_new] Token is valid, continuing
2023-10-09 13:24:24.221 WARNING (MainThread) [custom_components.fordpass] 403 Client Error: Forbidden for url: https://usapi.cv.ford.com/api/vehicles/v5/[REDACTED]/status?lrdt=01-01-1970+00%3A00%3A00
2023-10-09 13:24:24.222 WARNING (MainThread) [custom_components.fordpass] Error communicating with FordPass for [REDACTED]
2023-10-09 13:24:24.222 ERROR (MainThread) [custom_components.fordpass] Error fetching fordpass data: Error communicating with FordPass for [REDACTED]
2023-10-09 13:24:24.222 DEBUG (MainThread) [custom_components.fordpass] Finished fetching fordpass data in 0.493 seconds (success: False)

P.S. I just installed the iOS App and that one seems to work

Cheers, Andrei

tobox commented 11 months ago

Same for me (Germany) with a Mustang Mach-E.

Telemetry data shows up properly in Android app.

marwonn commented 11 months ago

Same here:

Error: Forbidden for url: https://usapi.cv.ford.com/api/vehicles/v5/MYVIN/status?lrdt=01-01-1970+00%3A00%3A00"

xcaliber93 commented 11 months ago

Can confirm, same issue (Canada)

jblaze03 commented 11 months ago

Can confirm, same issue (Canada)

+1

serlinGi commented 11 months ago

Same for me with my 2 cars. Italy

AlteVerwischteZiege commented 11 months ago

+1 Did they change the API again?

tonesto7 commented 11 months ago

I'm seeing the same :(

They've been working on releasing the new FordConnect API. I wonder if they finally moved over to it

Fifty889 commented 11 months ago

+1 Germany

sweidenbach commented 11 months ago

+1 Germany (Mach E) Already changed refresh_status from v2 to v5 due to 403 at 2023/09/15.

dtx1990 commented 11 months ago

Same problem in italy

JVTEAM commented 11 months ago

Same as of this morning

Logger: custom_components.fordpass Source: custom_components/fordpass/init.py:227 Integration: Fordpass (documentation, issues) First occurred: October 8, 2023 at 3:20:38 PM (491 occurrences) Last logged: 11:14:58 AM

('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) 403 Client Error: Forbidden for url: https://usapi.cv.ford.com/api/vehicles/v5/VIN/status?lrdt=01-01-1970+00%3A00%3A00 403 Client Error: Forbidden for url: https://usapi.cv.ford.com/api/vehicles/v5/VIN/status?lrdt=01-01-1970+00%3A00%3A00

jwfox5150 commented 11 months ago

Same issue here.

bb12489 commented 11 months ago

Same issue here as well. 403 forbidden.

mrgroen commented 11 months ago

Same here...

albamatti commented 11 months ago

Same here in Holland

csfreak commented 11 months ago

I have confirmed that my Fordpass app has switched to the new api.autonomic.ai endpoint.

I have found a working JS example of using the new API. https://github.com/TA2k/ioBroker.ford/blob/master/main.js

Should provide a good start to the endpoints and the responses, especially if mitmproxy is not available.

AlexW75 commented 11 months ago

Same here … Germany Ford Focus.

virtualbitzz commented 11 months ago

same here

mojomojoman commented 11 months ago

I have confirmed that my Fordpass app has switched to the new api.autonomic.ai endpoint.

I have found a working JS example of using the new API. https://github.com/TA2k/ioBroker.ford/blob/master/main.js

Should provide a good start to the endpoints and the responses, especially if mitmproxy is not available.

eek. this is a major rewrite I think. I was hoping I could hack it back into operation just changing the BASE_URL but looking at that other guys code its a lot more than that. We're all gonna owe the dev a few beers for this rewrite.

Example:

  async updateVehicles() {
    await this.getAutonomToken();
    const statusArray = [
      // { path: "statusv2", url: "https://usapi.cv.ford.com/api/vehicles/v2/$vin/status", desc: "Current status v2 of the car" },
      // { path: "statususv4", url: "https://usapi.cv.ford.com/api/vehicles/v4/$vin/status", desc: "Current status v4 of the car" },
      // { path: "statususv5", url: "https://usapi.cv.ford.com/api/vehicles/v5/$vin/status", desc: "Current status v5 of the car" },
      // {
      //   path: "fuelrec",
      //   url: "https://api.mps.ford.com/api/fuel-consumption-info/v1/reports/fuel?vin=$vin",
      //   desc: "Fuel Record of the car",
      // },
      {
        path: "statusQuery",
        url: "https://api.autonomic.ai/v1beta/telemetry/sources/fordpass/vehicles/$vin:query",
        desc: "Current status via query of the car. Check your 12V battery regularly.",
      },
    ];

I really should learn python and api's one of these days but how long have I been saying that now. ;-)

mharris1984 commented 11 months ago

Same for me. One Ford vehicle in USA. HA logs show the 403 forbidden starting at 3 AM UTC-7:00 (AZ Time) (10-11 hours ago)

itchannel commented 11 months ago

I am actively working on a new release to fix this. But there has been some considerable changes not just in the endpoint and token management but also the data now retrieved from the API. As you can see below I have now got it pulling back data again but there's some major bugs and rewrites I need to overcome first.

Also worst possible time for me for these changes to be made :)

homeassistantstable  | 2023-10-10 *********** DEBUG (SyncWorker_0) [custom_components.fordpass.fordpass_new] {"updateTime":"2023-10-01**********","vehicleId":"*******58008857db1e","vin":"*************","metrics":{"alarmStatus":{"updateTime":"2023-09-24**********","oemCorrelationId":"38833","tags":{"ALARM_SOURCE":"UNKNOWN"},"value":"ARMED"},"acceleratorPedalPosition":{"updateTime":"2023-09-24**********","oemCorrelationId":"38833","value":0.0},"batteryStateOfCharge":{"updateTime":"2023-09-24**********","oemCorrelationId":"38833","value":37.0,"vehicleBattery":"PRIMARY_BATTERY"},"batteryVoltage":{"updateTime":"2023-09-24T**********","oemCorrelationId":"38833","value":12.1875,"vehicleBattery":"PRIMARY_BATTERY"}
jwfox5150 commented 11 months ago

I am actively working on a new release to fix this. But there has been some considerable changes not just in the endpoint and token management but also the data now retrieved from the API. As you can see below I have now got it pulling back data again but there's some major bugs and rewrites I need to overcome first.

Also worst possible time for me for these changes to be made :)

homeassistantstable  | 2023-10-10 *********** DEBUG (SyncWorker_0) [custom_components.fordpass.fordpass_new] {"updateTime":"2023-10-01**********","vehicleId":"*******58008857db1e","vin":"*************","metrics":{"alarmStatus":{"updateTime":"2023-09-24**********","oemCorrelationId":"38833","tags":{"ALARM_SOURCE":"UNKNOWN"},"value":"ARMED"},"acceleratorPedalPosition":{"updateTime":"2023-09-24**********","oemCorrelationId":"38833","value":0.0},"batteryStateOfCharge":{"updateTime":"2023-09-24**********","oemCorrelationId":"38833","value":37.0,"vehicleBattery":"PRIMARY_BATTERY"},"batteryVoltage":{"updateTime":"2023-09-24T**********","oemCorrelationId":"38833","value":12.1875,"vehicleBattery":"PRIMARY_BATTERY"}

We appreciate you! Thank you for your work on this.

itchannel commented 11 months ago

image Current progress. A lot of data is missing in the new vehicle status API but there's also some cool new info like transmission status.

I'll try and have a beta later today which will be full of bugs but should restore basic functionality until I can work through all the new endpoints and json variables.

mojomojoman commented 11 months ago

wow. RIP all my sensor templates. no more elveh i guess. HVB status will be reported under "fuel" status i guess. and whats up with "acceleratorPedalPosition" is that a thing they are really logging? I would expect that from an OBD2 but not from the api. lol

itchannel commented 11 months ago

This is currently what I see for my vehicle, I don't know if someone with a electric vehicle will get the extra data like elveh. Hoping this isn't the only endpoint as it no longer supports window positions either.

image

SquidBytes commented 11 months ago

Considering the massive changes, I closed my Pull Request with the EV specific sensors. As far as further development related to EV features, what's the best way to work with you? I wasn't sure if my pull request was the right approach.

itchannel commented 11 months ago

Considering the massive changes, I closed my Pull Request with the EV specific sensors. As far as further development related to EV features, what's the best way to work with you? I wasn't sure if my pull request was the right approach.

Pull request is fine, honestly I've been massively busy with work so this project has sort of taken a back burner whilst it's been working ok. Once I get these massive changes implemented/fixed this week then I'm happy to work with you to increase the electric vehicle features if/how they are now supported in the new API.

SquidBytes commented 11 months ago

Considering the massive changes, I closed my Pull Request with the EV specific sensors. As far as further development related to EV features, what's the best way to work with you? I wasn't sure if my pull request was the right approach.

Pull request is fine, honestly I've been massively busy with work so this project has sort of taken a back burner whilst it's been working ok. Once I get these massive changes implemented/fixed this week then I'm happy to work with you to increase the electric vehicle features if/how they are now supported in the new API.

Understandable. My week is crazy also. I'll wait to grab the new release and start seeing if I can find what may be missing from EV, or what can be added.

virtualbitzz commented 11 months ago

I'll try and have a beta later today which will be full of bugs but should restore basic functionality until I can work through all the new endpoints and json variables.

Sweet, thanks for getting on this so quick! Sponsorship coming your way

OwnsJo commented 11 months ago

Sponsor sent! Thanks for working it so quick.

mojomojoman commented 11 months ago

This is currently what I see for my vehicle, I don't know if someone with a electric vehicle will get the extra data like elveh. Hoping this isn't the only endpoint as it no longer supports window positions either.

image

I've got an EV (Mustang Mach-E). I can test for you if that beta is live. Or if you wanna give me a quick tutorial of how to poll the API direct in python I could feed you back what I'm seeing. I don't code python. I know a lot of PowerShell but that's probably not gonna help me here. The data I used to get back for the HVB was a bunch of attribs shoved into one sensor which was hard to utilize in HASS in a lot of cards so I had a bunch of custom template sensors to split it out before.

e.g. from configuration.yaml

template:
  - sensor:
      - name: "Mach-E Battery charge"
        unique_id: sensor_fordpass_elveh_battery
        state: "{{state_attr('sensor.fordpass_elVeh', 'Battery Fill Level')}}"
        unit_of_measurement: "%"
        device_class: battery
  - sensor:
      - name: "Mach-E Charging Status"
        unique_id: sensor_fordpass_elveh_chstatus
        state: "{% if (is_state_attr('sensor.fordpass_elveh','Charging Status','NotReady')) %} Idle {% else %} {{state_attr('sensor.fordpass_elveh','Charging Status')}} {{ state_attr('sensor.fordpass_elveh','Plug Status') }} {% endif %}"
        icon: mdi:ev-station
  - sensor:
      - name: "Mach-E Charge Start Time"
        unique_id: sensor_fordpass_elveh_chstart
        state: "{{ strptime(state_attr('sensor.fordpass_elveh','Charge Start Time'),'%m-%d-%Y %H:%M:%S').strftime('%-m/%d %H:%M')  }}"
        icon: mdi:clock
  - sensor:
      - name: "Mach-E Charge Complete ETA"
        unique_id: sensor_fordpass_elveh_chend
        state: "{{ strptime(state_attr('sensor.fordpass_elveh','Charge End Time'),'%m-%d-%Y %H:%M:%S').strftime('%-m/%d %H:%M') }}"
        icon: mdi:clock
  - sensor:
      - name: "Mach-E Frunk"
        unique_id: sensor_fordpass_doorstatus_frunk
        state: "{{ (state_attr('sensor.fordpass_doorstatus', 'hoodDoor'))  }}"
        icon: mdi:car-door
  - sensor:
      - name: "Mach-E Driver Door"
        unique_id: sensor_fordpass_doorstatus_driver
        state: "{{ (state_attr('sensor.fordpass_doorstatus', 'driverDoor'))  }}"
        icon: mdi:car-door

And this one which isn't a real sensor value but a derived one.

  - sensor:
      - name: "Mach-E Battery Max Range"
        unique_id: sensor_fordpass_elveh_maxrange
        state: "{{ ((states('sensor.mach_e_range') | int) + (states('sensor.mach_e_range') | int) / (states('sensor.mach_e_battery_charge') | int) * (100 - (states('sensor.mach_e_battery_charge') | int))) | round()  }}"
        unit_of_measurement: "mi"
        device_class: distance
        icon: mdi:gauge

Purpose of that is to calculate what the car would currently think my max range would be if I had 100% battery. The actual expected range in Mi/Km of the remaining battery is not a static value in Ford EVs. The car is constantly updating the expected Miles per Kwh based on recent drives, outside temp, and/or some kind of secret sauce algorithm that they don't reveal. I don't know how much you know about EVs but the range estimates are practically wild guesses. If the weather is perfect and I don't need heat or AC and I'm doing a bunch of city/suburb driving under 55mph (plenty of regenerative braking, no passing people at 85mph, etc) then my max range in those conditions could get up to 375 miles (over 4 Mi/Kwh, car has a 91Kwh HVB). The opposite worst case of that would be a cold winter day where I'm driving for hours and hours on a high speed interstate road going 80+mph in zero traffic. I'd be lucky to get 225 miles if it was really cold out. As sold the car states a max range of 277 miles i think but since it's been a mild summer and I don't drive long distances often, my personal max range floats around in the 290 - 310 mile range. So i'm essentially keeping tabs on the cars algorithm. Ford only reports what your estimated max range is based on your current battery capacity and I only charge it up to 90% most days.

ziptbm commented 11 months ago

Considering the massive changes, I closed my Pull Request with the EV specific sensors. As far as further development related to EV features, what's the best way to work with you? I wasn't sure if my pull request was the right approach.

Pull request is fine, honestly I've been massively busy with work so this project has sort of taken a back burner whilst it's been working ok. Once I get these massive changes implemented/fixed this week then I'm happy to work with you to increase the electric vehicle features if/how they are now supported in the new API.

I’ve got two Ford EVs as well, so would be happy to jump on beta train once it’s ready for broader consumption if helpful. Thank you!

itchannel commented 11 months ago

I'm hoping to have a beta out tonight (8 hours or so) which will at least restore basic sensors values, then the next step is re-coding the switches & locks.

In regards to other features like EV once I've got a basic working version again I'll create a python script that uses the library so people can run and send me the json output from the new API to re-add back into HA.

One step at a time as they made some significant changes and never updated the developer API documentation which is still stuck on V3 from over a year ago.

SquidBytes commented 11 months ago

This is currently what I see for my vehicle, I don't know if someone with a electric vehicle will get the extra data like elveh. Hoping this isn't the only endpoint as it no longer supports window positions either.

image

I just got https://api.autonomic.ai/v1beta/telemetry/sources/fordpass/vehicles/{vin}:query working. There is a ton of data.

image image image

itchannel commented 11 months ago

This is currently what I see for my vehicle, I don't know if someone with a electric vehicle will get the extra data like elveh. Hoping this isn't the only endpoint as it no longer supports window positions either. image

I just got https://api.autonomic.ai/v1beta/telemetry/sources/fordpass/vehicles/{vin}:query working. There is a ton of data.

image image image

So thats the same URL I'm querying looks like it gives more data depending on what the vehicle supports.

Any chance you could send me the raw data from yours minus personal information.

SquidBytes commented 11 months ago

Yeah, let me work on redacting it. I also don't mind working on EV integrations after its published, so don't feel like you need to input EVERYTHING.

doronazl commented 11 months ago

This is currently what I see for my vehicle, I don't know if someone with a electric vehicle will get the extra data like elveh. Hoping this isn't the only endpoint as it no longer supports window positions either. image

I just got https://api.autonomic.ai/v1beta/telemetry/sources/fordpass/vehicles/{vin}:query working. There is a ton of data. image image image

So thats the same URL I'm querying looks like it gives more data depending on what the vehicle supports.

Any chance you could send me the raw data from yours minus personal information.

Just here to say thank you for your hard work on this project

SquidBytes commented 11 months ago

FYI, if anyone wants to grab json data from their vehicle this is the script I made to return the json. Its ugly, but it works. Drop it in the /config/custom_components/fordpass folder on your HomeAssistant Add the information on the top (because its dumb) run python3 autonomicData.py from a terminal in the /config/custom_components/fordpass folder

https://github.com/SquidBytes/fordpass-ha/blob/master/custom_components/fordpass/autonomicData.py

tobox commented 11 months ago

Sponsor sent! Thanks for working it so quick.

Same here :-) Just a few bucks for me, but if every 100th user of the integration would consider sponsoring, it could become a nice income for itchannel.

itchannel commented 11 months ago

Sponsor sent! Thanks for working it so quick.

Same here :-) Just a few bucks for me, but if every 100th user of the integration would consider sponsoring, it could become a nice income for itchannel.

Thank you I appreciate it. All goes to my coffee fund :)

RPCHome commented 11 months ago

Sponsor sent! Thanks for working it so quick.

Same here :-) Just a few bucks for me, but if every 100th user of the integration would consider sponsoring, it could become a nice income for itchannel.

Thank you I appreciate it. All goes to my coffee fund :)

I've just added to your coffee fund @itchannel. Thank you for the hard work on this.

itchannel commented 11 months ago

Current status. Got most old values working. Going to refactor and release a beta for people to try. Things will be broken and can't seem to test the switches/locks as my car isn't responding to any commands currently from HA or the app so must be low battery :)

itchannel commented 11 months ago

Released a rough beta for anyone who wants to try. There is a lot missing from a sensor perspective as they don't map nicely to the new API.

Over the coming days I will be adding some new sensors and modifying the existing ones. Electric car wise there is a lot more sensors, however the current version will just show Battery charge level instead of fuel for now.

AndreiGavriliu commented 11 months ago

I just updated to Beta2 and it seems to work again. How should I send you my vehicle data? if you are interested, of course. Are the debug logs enough?

Thanks for the quick fixes!

itchannel commented 11 months ago

@AndreiGavriliu Good to hear it's working It's still pretty rough but at least works again :)

In regards to logs if you can un-comment out line 337 in fordpass_new.py then just run it for an hour it will appear in the debug logs. I've left it commented as it generates tonnes of data and didn't want to overwhelm peoples logs. You should see an array of vehicle data, just make sure you sanitize GPS coordinates and VIN numbers before sending 👍

dnikles commented 11 months ago

make sure you sanitize GPS coordinates

FYI your reply 2 hours ago shows GPS coordinates. Just letting you know!

itchannel commented 11 months ago

make sure you sanitize GPS coordinates

FYI your reply 2 hours ago shows GPS coordinates. Just letting you know!

Thanks for the heads up, luckily not where I live but close enough :) Meant to post the dev instance screenshot. Too many coffees today :)

jwfox5150 commented 11 months ago

Looks like everything is working for me except TPMS and odometer. Many thanks for a quick fix! 2018 Raptor, would be happy to send you a log dump if you wish. As it is an older vehicle, not sure if it would be helpful.

itchannel commented 11 months ago

Looks like everything is working for me except TPMS and odometer. Many thanks for a quick fix! 2018 Raptor, would be happy to send you a log dump if you wish. As it is an older vehicle, not sure if it would be helpful.

Can you send me a log dump so I can see why those sensors aren't working for you.