itchannel / fordpass-ha

Fordpass integration for Home Assistant
319 stars 61 forks source link

Locked out of Ford Account #214

Closed Madbeefer closed 6 months ago

Madbeefer commented 2 years ago

I wasn't able to access my ford account at all. I finally got a hold of someone and they said they lock any account that has over 20,000 API calls in a week. I didn't see anywhere to change the interval for API calls.

AlteVerwischteZiege commented 2 years ago

What else do you have that touches your FordPass data?
My utility offers a rebate for 'Off-Peak" charging who's app was refreshing and calling the API every 5 Minutes 24/7 that's 10,080 calls from just one App, before calls from the FordPass app, phone widgets, ect. Since removing the utilities access, the interval can be attributed to FordPass/widget refresh and the HomeAssistant refresh calls. image

blaine07 commented 2 years ago

For what it’s worth the same happened to me today; my accounts been locked.

Also, see this:

lots of pages:

https://www.macheforum.com/site/threads/unauthorized-api-use-can-disable-your-account.13893/page-11

JVTEAM commented 2 years ago

Also locked out today....

blaine07 commented 2 years ago

Link I posted above suggest it isn’t even a number of calls it’s that the request didn’t come from FORD APP. This app is ALL I had setup to do anything with my vehicle. Thanks,Blaine--Sent by carrier pigeonOn Nov 29, 2022, at 15:10, itchannel @.***> wrote: I'd be interested to know if you were using any other apps that use the Fordpass API? As by default the HA integration only makes a vehicle call every 15mins. Of which that involves at max 5 API requests (Per Vehicle).

Call to check oauth token Refresh or renew if needed Get latest Vehicle status from /status endpoint Get latest Messages from /messages

So in theory the maxAPI calls it should be making per week is 3,360 calls (Per Vehicle). which is only 20 per hour which is nothing in API terms.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

itchannel commented 2 years ago

So currently the integration checks every 5mins for status updates. Which shouldn't cause any API issues as that's barely anything but if Ford are making the limit even less then I could add the ability to set your own update interval in the GUI or you can do it manually by editing the init.py files on line 37 and changing the below line to another value in seconds (Keep it more than 300)

SCAN_INTERVAL = timedelta(seconds=300)
blaine07 commented 2 years ago

Is that not what they wanted here?Feature Request: Forcibly Limit API Calls · Issue #217 · itchannel/fordpass-hagithub.comI appreciate anything that could be done but I can’t have my app not work; I won’t be able to continue forward with it. Yesterday started the evidently long journey to getting account unlocked. Thanks,Blaine--Sent by carrier pigeonOn Nov 29, 2022, at 15:18, itchannel @.***> wrote: So currently the integration checks every 5mins for status updates. Which shouldn't cause any API issues as that's barely anything but if Ford are making the limit even less then I could add the ability to set your own update interval in the GUI or you can do it manually by editing the init.py files on line 37 and changing the below line to another value in seconds (Keep it more than 300) SCAN_INTERVAL = timedelta(seconds=300)

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you commented.Message ID: @.***>

itchannel commented 2 years ago

@blaine07 they wanted a physical limit which would be hard to enforce with HA reboots etc. However adding the ability to change how often it updates for each individual user is probably the way forward.

I've been away for a while but will see what I can mock up later today.

itchannel commented 2 years ago

1.39 has been released which includes a new default poll interval of 15mins instead of 5 as well as the ability to change this in options.

balthisar commented 2 years ago

Locked out yesterday.

jrprinty commented 2 years ago

I my account was locked out yesterday.

I do have two vehicles connected through Home Assistant.

I can't believe Ford would do this to us as a paying customer!

@itchannel Thank you for all your work on this integration! It has served my family well over the years, and I hope Ford can pull their heads together and get this fixed for all of us.

Screenshot 2022-11-30 at 8 48 15 AM
serlinGi commented 2 years ago

also to me 2 cars no longer usable with integration; blocked account, error code fordpass app: CAIAH0320E I was hoping to fix it when I saw that the new update was published 😭

gpf2ez commented 2 years ago

Add me to the list of people locked out yesterday. I also have two vehicles and I got the same email as above plus I get the same CAIAH0320E error code when I try to login.

For those that have not done so already please call Ford Customer service at 800-392-3673. They are taking reports and creating tickets. They person I spoke to said they have been getting a lot of calls.

My guess is Ford recently updated their security policy's and did not take this use case into consideration.

blaine07 commented 2 years ago

TLDR; Ford is being ruthless

jrprinty commented 2 years ago

Well, I just got off the phone with Ford (833-385-0512). They made me delete my FordPass integration from Home Assistant and said it would be 3 to 5 business days before they re-enabled my account.

I'm so pissed off with Ford right now. Their API is publicly available, but now we can't use it.

I will set up a burner account once my Ford account is unlocked and see what happens with that account.

blaine07 commented 2 years ago

Well, I just got off the phone with Ford (833-385-0512). They made me delete my FordPass integration from Home Assistant and said it would be 3 to 5 business days before they re-enabled my account.

I'm so pissed off with Ford right now. Their API is publicly available, but now we can't use it.

I will set up a burner account once my Ford account is unlocked and see what happens with that account.

They just escalated my case two days ago and haven’t heard anything since. Lady asked me if I deleted whatever and I told her yes but she was just who opened ticket 😩

jrprinty commented 2 years ago

That's not what I wanted to hear @blaine07

I sent a public tweet to Ford but haven't gotten a response from them, and I pointed out the FordConnect API. I doubt it will help in any way, but at least it is more public now.

itchannel commented 2 years ago

Shame to hear so many people are being locked out.

What sucks is it uses the documented api calls to the 3rd party API from the documentation on developer.ford.com which I have access to. At no point does it mention any API limitations or rules. I can only guess they want to go down the paywall route.

Hopefully there might be some transparency from Ford by either working with devs to make 3rd party integrations compliant or actual open source tooling from them. More and more are going down the smart home route and integrate their vehicles into other systems like charging, energy monitoring etc :(

jrprinty commented 2 years ago

I'm going to bring that up when they call, if they actually call me.

I am also signed up with a developer account. Hepefully they have a plan that doesn't involve us losing functionality and having a paywall. This is effecting so many people not just in Home Automation but even with saving money with EVs and thier local electric companies.

balthisar commented 2 years ago

Say, are any of you using a reverse proxy? Or Nabucasa?

I just had a conversation with an IT Security person with an onshore accent, and she asked if I were accessing the API via a VPN. I offered that I was using a reverse proxy, and she indicated that that may have been one of the flags, because there was a high incidence of IP address changes in the traffic.

I'm not going to start analyzing my traffic, but my HA install sees everything as coming from my xxx.xxx.xxx.9 machine, and transmits all traffic back to that. If there are headers with IP addresses not matching the actual endpoints, that might be what she was referring to.

In any case, she acknowledged that HA is not being locked out intentionally, and that they consider it first party not third party, and that there are people in senior management at Ford that got bitten by this same bug that are anxious to get it solved.

She suggested that the system might not flag at a reduced polling frequency, but couldn't make any guarantees. She was knowledgeable, but has spent her whole day on the phone calling people instead of being part of the team actually solving the problem.

She specifically said it was the FNA IT Security team working on solving the issue, which is comforting.

@itchannel, we're now abusing the issues system instead of reporting actual bugs and working on resolving them. Sorry 'about that. Thanks for letting us communicate here.

gpf2ez commented 2 years ago

Say, are any of you using a reverse proxy? Or Nabucasa?

I just had a conversation with an IT Security person with an onshore accent, and she asked if I were accessing the API via a VPN. I offered that I was using a reverse proxy, and she indicated that that may have been one of the flags, because there was a high incidence of IP address changes in the traffic.

I'm not going to start analyzing my traffic, but my HA install sees everything as coming from my xxx.xxx.xxx.9 machine, and transmits all traffic back to that. If there are headers with IP addresses not matching the actual endpoints, that might be what she was referring to.

In any case, she acknowledged that HA is not being locked out intentionally, and that they consider it first party not third party, and that there are people in senior management at Ford that got bitten by this same bug that are anxious to get it solved.

She suggested that the system might not flag at a reduced polling frequency, but couldn't make any guarantees. She was knowledgeable, but has spent her whole day on the phone calling people instead of being part of the team actually solving the problem.

She specifically said it was the FNA IT Security team working on solving the issue, which is comforting.

@itchannel, we're now abusing the issues system instead of reporting actual bugs and working on resolving them. Sorry 'about that. Thanks for letting us communicate here.

I host from a static IP from my house without a VPN or reverse proxy. I honestly think this comes down to a new employee who decided to look at logs and saw a huge amount of hits through the API and rather than looking to see what's going on just started shutting down accounts.

itchannel commented 2 years ago

@balthisar Interesting to hear that they acknowledge HA. Would be nice if someone from Fords dev team reached out so I could work with them to make it nicely compliant and fully supported. We can dream :) It's either someone from security wanting to limit the amount of exposure or someone from finance who thought "How can we reduce the API request costs lol"

It could potentially be using a VPN but guessing 99% of HA users just have it going out via their own network. I've always tried to limit the amount of information the integration pulls from Ford as I know some others grab internal device details and other hidden functionality which could cause problems and shouldn't really be exposed to the end user but this integration only uses the endpoints from their own 3rd party API docs 👍

serlinGi commented 2 years ago

Screenshot_2022-12-01-12-38-02-49_680d03679600f7af0b4c700c6b270fe7

the reponse of Ford Italy assistance

balthisar commented 2 years ago

Interesting that Home Assistant isn't on that list. The engineer I talked to definitely knew what Home Assistant was.

I wonder what the track is to get this integration out of HACS and into Home Assistant proper? Doing so might lend it a bit more credibility in corporate's eyes.

stroodle96 commented 2 years ago

Does setting up a second fordpass account for this integration bypass the issue? Or do they lock your account based on vehicle vin#?

If I create a separate 2nd account and use those credentials for HA integration, then will I still be able to use the Fordpass app with my original account even if the 2nd one gets blocked? Or do they block any fordpass account linked to that vin?

Can anybody with an actively blocked account try signing up for a new Ford account and link their vehicle vin to see if that works?

jrprinty commented 2 years ago

@stroodle96 You cannot set up another account while being locked out without completely abandoning your original Ford account.

With that said, I have read a couple of people's posts on other forums stating only the account they were using was locked out, and their wife's account was still working without issue.

As soon as my account is unlocked, I will set up a burner account for HA, and I'll see what happens. Hopefully, the reports I have read are correct, and if they disable my burner account, who cares.

serlinGi commented 2 years ago

I'll tell you my experience: I shared my wife's car with my account; as a result on my app and HA I was seeing and managing the 2 cars.
with locked account I couldn't view either of the 2. My wife's account (who has only her car on the app) remained active, I reinstalled on HA in her name and the car was manageable (although her VIN I had already used it, it didn't create blocking problems). this afternoon they unlocked my account; i will wait a couple of days before putting it back on HA. in the meantime I think I'll share my car with my wife, so I hope that at least one of the 2 accounts will always be active: I'm not going to give up on our custom (again and always thanks @itchannel )

stroodle96 commented 2 years ago

Thank you both for sharing. I thankfully have not been locked out yet. I created a separate shared account to use with this integration and changed the refresh rate to 30 minutes to be extra safe. Hopefully this works to prevent a total lockout of the FordPass app on my my main account.

Madbeefer commented 2 years ago

I just called them again since they still haven't unlocked my account from like 2 months ago.. The person I spoke to said she has about 5 people call her a day about their accounts getting locked out..

GeneralLouis commented 2 years ago

Got a definite answer on why my account was locked: Cyber security team checked the logs and my account swapped IPs 1500 times in a 24 hour period so they disabled my account.

The FordPass help desk can enable peoples accounts again.

What I'm confused about is how that happened? Would the Nabu Casa cloud cause such a thing with outbound traffic?

gpf2ez commented 2 years ago

They were not able to unlock my account. My ticket was escalated on Wed and they have till the 6th to get back to me according to the rep I spoke to just now.

GeneralLouis commented 2 years ago

Sorry should have clarified, the FordPass help desk is the way to get your ticket submitted to the internal team to enable the account.

jrprinty commented 1 year ago

@GeneralLouis I think the rep misspoke or was blowing smoke. I can't see any possible way your IP would change that many times in 24 hours. But, of course, that is just my opinion.

I finally got my account unlocked and set up a burner account to use in Home Assistant. I set up two vehicles and set the refresh rate to 30 minutes. We will see what happens.

The lady I was talking to wasn't very helpful at all. I explained multiple times we were using this integration within Home Assistant, and it follows all of their API guidelines for FordConnect API, but she just couldn't grasp that concept. I also informed her that I have a developer account with them and asked if they had changed their guidelines or have new recommendations to please send them, but she said she hadn't heard anything about this. She ultimately told me that the API had to have made hundreds of API requests, and their system found them fraudulent, which we all know it didn't.

She also informed me that they are dealing with a lot of calls about accounts being locked out, and their cyber security team is looking at changing the way it works, and hopefully, it will be less restrictive in the future. I'm not holding my breath, though.

Like I said not very helpful, but I thought I would share my experience with you guys.

blaine07 commented 1 year ago

I still don’t have my account unlocked. Facepalm

jrprinty commented 1 year ago

@blaine07 If you have social media, start annoying them on there.

They kept telling me I had to wait and that they would get back to me, so I started bugging them on Twitter(@Ford), and once someone else commented on my tweet, they were very quick to get my account unlocked. From the time they told me to call in to getting my account unlocked was about three hours, but I started the process 7 days ago.

I don't know if the Twitter thing helps or if it was a coincidence, but it can't hurt. I know you have been waiting a lot longer than I did.

gpf2ez commented 1 year ago

@GeneralLouis I think the rep misspoke or was blowing smoke. I can't see any possible way your IP would change that many times in 24 hours. But, of course, that is just my opinion.

I finally got my account unlocked and set up a burner account to use in Home Assistant. I set up two vehicles and set the refresh rate to 30 minutes. We will see what happens.

The lady I was talking to wasn't very helpful at all. I explained multiple times we were using this integration within Home Assistant, and it follows all of their API guidelines for FordConnect API, but she just couldn't grasp that concept. I also informed her that I have a developer account with them and asked if they had changed their guidelines or have new recommendations to please send them, but she said she hadn't heard anything about this. She ultimately told me that the API had to have made hundreds of API requests, and their system found them fraudulent, which we all know it didn't.

She also informed me that they are dealing with a lot of calls about accounts being locked out, and their cyber security team is looking at changing the way it works, and hopefully, it will be less restrictive in the future. I'm not holding my breath, though.

Like I said not very helpful, but I thought I would share my experience with you guys.

From your description it really sounds like they implemented a new security software or system and it started flagging because they didn't have their rules set up to match their API. And now they're dealing with the fallout from that. Did they call you directly to talk to you and get your account unlocked? I was told mine would be unlocked by today but I haven't received any correspondence from them about it at all.

GeneralLouis commented 1 year ago

@jrprinty , my personal theory is, after reading the code here, that every single HA user has the same client ID, so if cyber security was simply tracking Client ID they would see TONS of calls coming from all over the world (every HA instance with Fordpass integration) since this implementation has a static client ID coded in. (I have gone in and changed mine and it appears to be working fine on my new burner account after getting unlocked) EDIT: Don't do this, see itchannel's comment below.

I personally reached out to team members on the cyber security team and they looked at the logs and got back to me, though I have not heard back when I asked for details, this was not a phone/support agent.

@gpf2ez , I called last week to the Ford support number AND emailed an internal contact, so I had 2 tickets going basically. Yesterday I called the Fordpass support line instead, they told me the previous call was in a different workstream and they would get a new ticket in since I had not heard back, within 3 hours I got the generic IBM email that said "Account : Enabled", so I am not entirely sure which "ticket" caused it to go through, I have had no follow up from any source.

When any of you with disabled accounts call and they ask about Third-party use, make sure you tell them, you are using the APIs yourself. Since this code is open source and you are running it locally, your credentials are not being "given" to anyone else, you are simply using code written to help you use their API's, something you could do yourself if you read their API documentation. (and had TONS of time on your hand to figure it out like the author(s) of this code have done)

itchannel commented 1 year ago

@GeneralLouis I would be very careful changing Client ID's especially ones that are not approved. A Client ID is per application not per user. The current Client ID is the one used for Fordpass as they haven't currently issued specific ones for 3rd party applications that I am aware of.

GeneralLouis commented 1 year ago

@itchannel , thanks I will revert my changes, as I did not fully understand client ID and it's use.

JVTEAM commented 1 year ago

I need to call them back today as it's been a week. I wrote down my case number, but it's among other numbers that I didn't label. Does it start with cas-?

GeneralLouis commented 1 year ago

@JVTEAM yes, they start with cas-. I was told to call 800-336-0486 to get to the correct queue if I had to call back. Double-check their hours, they may still be closed for 30 minutes.

blaine07 commented 1 year ago

Yeah I need to follow up on mine getting unlocked but no idea how to get a hold of someone without being on hold for stupid days.

gpf2ez commented 1 year ago

I called right at 9am. Spoke to a guy there who looked at my case and told me there were no updates yet. I explained that the only people getting their accounts unlocked were those who complained on Twitter or made a huge fuss and I did not want to do that. He offered to get a supervisor and came back saying that there was not one free but would call me by noon. 10 mins after I hung up I got an email showing my account was unlocked and I can now get in.

jrprinty commented 1 year ago

@gpf2ez I agree completely with them not having the rules set up properly when they implemented this new security software. I know things happen, and they are ultimately trying to protect our accounts which I appreciate. Hopefully, in the future, they will have this all squared away, and we, as the Ford community, will not have to deal with account lock-outs and still be able to use the API.

@blaine07 I called ‭+1 (833) 385-0512‬ , and I think it was option 2 to talk to a FordPass Guide. It only took a few minutes before someone picked up.

@gpf2ez I'm glad to hear you got your account unlocked. I wish this process was more streamlined, but at the same time, it shouldn't have happened in the first place.

blaine07 commented 1 year ago

@gpf2ez I agree completely with them not having the rules set up properly when they implemented this new security software. I know things happen, and they are ultimately trying to protect our accounts which I appreciate. Hopefully, in the future, they will have this all squared away, and we, as the Ford community, will not have to deal with account lock-outs and still be able to use the API.

@blaine07 I called ‭+1 (833) 385-0512‬ , and I think it was option 2 to talk to a FordPass Guide. It only took a few minutes before someone picked up.

@gpf2ez I'm glad to hear you got your account unlocked. I wish this process was more streamlined, but at the same time, it shouldn't have happened in the first place.

Called again, guy was very helpful. Said someone should be calling me today :cross fingers:

cjramseyer commented 1 year ago

We both have Ford vehicles, and had both setup in HA with this addon. For my vehicle this had been working great for over a year.

As of a week ago our accounts were disabled, and I have been told more than once that it is violating the Terms & Conditions. So I challenged that notion and was directed to sections 13 & 14, neither of which say anything about NOT being able to use 3rd party tools.

I really like having this addon and I am able to build several automations including calendar entries to start our vehicles on cold mornings while getting ready for work. I am not sure what is able to be done, however, I would appreciate any updates that can be done to help prevent this issue.

What they told me today, was if they "see the traffic continue they will disable our account permanently" I was not told about any call limits or volume of traffic, etc. However, the only way they would really be able to tell is something like the user agent string. I suggest adding a default user agent string and an option to change that in the configuration.

Looking forward to updates for this

itchannel commented 1 year ago

@cjramseyer User-Agent is set on line 19 in "fordpass_new.py" you are welcome to change it to another value however by default I have it set to the Fordpass application User-Agent.

blaine07 commented 1 year ago

@gpf2ez I agree completely with them not having the rules set up properly when they implemented this new security software. I know things happen, and they are ultimately trying to protect our accounts which I appreciate. Hopefully, in the future, they will have this all squared away, and we, as the Ford community, will not have to deal with account lock-outs and still be able to use the API. @blaine07 I called ‭+1 (833) 385-0512‬ , and I think it was option 2 to talk to a FordPass Guide. It only took a few minutes before someone picked up. @gpf2ez I'm glad to hear you got your account unlocked. I wish this process was more streamlined, but at the same time, it shouldn't have happened in the first place.

Called again, guy was very helpful. Said someone should be calling me today :cross fingers:

About a hour later feller got me sorted out. Something about 3000 tickets in last 30 days or something. Very polite but seemed exhausted lol

dmkjr commented 1 year ago

I created my ticket last night. I had the rep add to the case that I was looking at API call information in regards to at what point their tools trip as "fraud". Not hopeful for a response, but we will see.

JVTEAM commented 1 year ago

Well...It's been 2 weeks. guess I should call them back. What a nightmare.

boctok commented 1 year ago

I would really like to have this in my HA, but I also don't want to get locked out of my app. Is there any surefire way to use this, limiting the API calls so that I won't bring down the wrath of the Fordpass gods?