JLR blocked API access again (Unable to connect to the JLRIncontrol servers)

[JonSilver]

JLR app is fine, the HA integration isn't, since midnight today.

[msp1974]

Adding @ardevd to this issue. I see the same with a 407 Proxy Auth Required error.

[alainsch]

me too. Same in wattcat

[BenjiBoisemont]

The same for me, connection lost with jlr last night. MyPace app also broken on iPhone

[ardevd]

I'm on it.

[tdipilot01]

My Android Jaguar Remote app works, but HA integration's been down all day.

[ardevd]

It's going to take some time to figure out the new update but I think it's doable.

[krenlund]

Phew, this is a relief! Thank you @ardevd in advance!

[ardevd]

Phew, this is a relief! Thank you @ardevd in advance!

No promises! :) But I'll try!

[Jazmodo]

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

[callumeveratt]

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

It's the latter - they've taken a stance against third party apps recently.

[pmharris77]

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

Same question asked here on jlrpy repo (the API library this integration uses to talk to JLR servers) https://github.com/ardevd/jlrpy/issues/131#issuecomment-2118657099

They are doing it on purpose in a game of cat and mouse with third party apps.

Looks like they've added an extra layer of authentication this time according to @ardevd. It's annoying as I find the jlrpy library and this integration incredibly useful, but I guess I can see that JLR don't want potentially malicious actors hacking JLR cars using their API, especially with the recent Range Rover security scandal...

[pmharris77]

FYI, for those interested in the underlying issue within the jlrpy Python library, see here: https://github.com/ardevd/jlrpy/issues/131

[ardevd]

I'd argue that

a) you can't really lock out third party API access, you can just make it tedious for people to access it.

b) locking out API access is the wrong solution to a problem that doesn't exist.

JLR could easily put restrictions on sensitive remote operations. If they wanted to prevent malicious remote unlocks they could make the double lock disable that feature for example.

I doubt any Land Rovers have been stolen through "malicious" third party API access.

Furthermore, the way JLR have consistently undermined years of community effort by arbitrarily changing their APIs without notice or explanation is incredibly hostile.

[pmharris77]

I'd argue that

a) you can't really lock out third party API access, you can just make it tedious for people to access it.

b) locking out API access is the wrong solution to a problem that doesn't exist.

JLR could easily put restrictions on sensitive remote operations. If they wanted to prevent malicious remote unlocks they could make the double lock disable that feature for example.

I doubt any Land Rovers have been stolen through "malicious" third party API access.

Furthermore, the way JLR have consistently undermined years of community effort by arbitrarily changing their APIs without notice or explanation is incredibly hostile.

@ardevd I agree. You can't drive a JLR away without the key being present. At most you can unlock the car and could steal stuff inside, but you'd have to know the car owner's app credentials, which would only be possible if they had a data breach leaking car reg or VIN mapped to user account details...

I wonder what other manufacturer's approach to third party API access has been?

[ismarslomic]

I agree that this approach from JLR does not improve the overall security, in regards to third party usage.

But from my understanding, the main issue in this context is that the JLR API we are using is not designed nor handled as third party API. Its basically private API designed for JLR app(s), which they have full control of and can handle any breaking changes in the API. The API is not documented, nor supported for third parties by JLR. We basically did reverse engineering to understand how it works and how to make use of it. So the main problem is that JLR does not provide any official public API with close collaboration with the open source community.

In that meaning, we should not be frustrated over breaking changes in the private API, but rather that JLR ignores third party apps and innovative usage.

[ardevd]

JLR can do what they want with their private APIs. My argument is simply that it's incredibly dumb to agitate your most enthusiastic customers by cutting off community tools that have been developed and maintained over several years.

[ismarslomic]

Yeah, agreed to that!

[Jazmodo]

Thanks for the replies @ardevd 🙏 && I completely agree.

As you say, cutting off access to some of their more enthusiastic owners seems counter-productive. I love the ability to integrate Car charging/location features into Home Assistant, fantastic you've managed to get it working at all.

Not sure I can help much, but I can be a small but persistent voice, if nothing else.

FYI, for those interested in the underlying issue within the jlrpy Python library, see here: ardevd/jlrpy#131

I'll check this out! Cheers 😃

[msp1974]

To add to both @ismarslomic and @ardevd, most other manufacruters also have 'private' and undocumented apis but do not forceably try and stop 3rd party apps like this. If you just look at the list in HA for different makes, there are many.

I feel their approach is both narrow minded and old fashioned. We can also note that they have blocked Octopus Energy from using it too with their energy product designed to allow cheaper charging of EV and PHEV cars, thus meaning JLR customers will pay more!

I think we need to find some route into JLR to have a grown up discussion as to how we can utilise this api to the benefit of their paying customers, which we all are.

As a note, just had to pay for my Incontrol service for the first time @ £280 for the year!

[ismarslomic]

Agree with you @msp1974! We should initiate dialog with JLR. They already know that their API is used by many third parties, so it should not be an surprise that this approach does not work for anyone. Im using Tibber as energy provider in Norway, and they also use the same API to connect to the vehicle in order to do smart charging when the electricity is cheap. This service is currently broken too, and I need to manually start charging when the price is cheap.

Does anyone know a name of the product owner or developer of the JLR API? I think we need to reach out directly and not through the customer support.

[pmharris77]

Considering they are about to exclusively sell electric cars, blocking third party API access, whilst not even providing any official integrations themselves, is just madness and is just going to make current and future customers go to their competitors. Way to go bust JLR! 😅

[ardevd]

@ismarslomic all my previous JLR contacts with positions related to connected car tech have left the company. I haven't heard from JLR reps for years.

[adec]

Until recently I used to work for JLR but now work for one of their suppliers. I'll see if I can find a way to raise this internally with them. I can't promise anything but I will do what I can. I'd quite like to use the integration myself but would be more comfortable if the public API allowed some of the more risky options to be disabled (such as opening the car doors).

[ardevd]

Until recently I used to work for JLR but now work for one of their suppliers. I'll see if I can find a way to raise this internally with them. I can't promise anything but I will do what I can. I'd quite like to use the integration myself but would be more comfortable if the public API allowed some of the more risky options to be disabled (such as opening the car doors).

That would be awesome!

Personally, I don't think remote car lock/unlock should be a thing. But a way to opt out would be nice.

[pmharris77]

Been doing a lot of reading about the whole Octopus smart charging saga with API access being v suddenly removed for energy companies back in March. There has been a very public backlash on X, and someone mentioned that JLR stated that they were working on improving the API security. Given the amount of complaints that both Octopus and JLR have had about removing support, it would be difficult to imagine they aren't talking about how to resolve the issue.

It got me thinking, maybe the extra app secret layer added to the API authentication is being introduced as a way to allow JLR-approved third party apps, such as Octopus, to access to the API in the future with a JLR-issued app secret? If so, this might actually be a positive sign, if they are willing to engage with third-party app developers. I can live in hope 🤞

[pmharris77]

Some people have gotten the following response from JLR recently, which does state they are "working to develop authorised solutions to smart home and energy services as quickly as possible"

Full response:

“We are responsible for the performance and safety of our vehicles and provide a warranty for this. We are not able to test, approve and warrant the safety and security of unauthorised third-party access to our vehicles and its data. This is a permanent change and only approved, authorised and tested apps will have access to our vehicles and their data. We are working to develop authorised solutions to smart home and energy services as quickly as possible. We apologise for any inconvenience this may have caused, but hope you understand why this was necessary.”

[ardevd]

Been doing a lot of reading about the whole Octopus smart charging saga with API access being v suddenly removed for energy companies back in March. There has been a very public backlash on X, and someone mentioned that JLR stated that they were working on improving the API security. Given the amount of complaints that both Octopus and JLR have had about removing support, it would be difficult to imagine they aren't talking about how to resolve the issue.

It got me thinking, maybe the extra app secret layer added to the API authentication is being introduced as a way to allow JLR-approved third party apps, such as Octopus, to access to the API in the future with a JLR-issued app secret? If so, this might actually be a positive sign, if they are willing to engage with third-party app developers. I can live in hope 🤞

It's possible but so far JLR has shown exceptionally little interest in dialog with third party developers. Maybe Octopus would be an exception, but their equivalent here in Norway, Tibber, have had no contact. I've reached out to JLR repeatedly and they have never responded.

[CorceiroPT]

I would say this time will be more difficult. It could be that they (JLR) used a proxy with different credentials (and maybe temporary rotating passwords for it) to access and validate the API keys. This would be a way to block third-party access to it and I supposed they did it...

[ardevd]

From my findings so far it seems like the only change here is that the app secret now rotates (at least) every time you start the app. It seems to be happening offline from what I can tell.

[ismarslomic]

Is it possible to replicate the code for code generation in jlrpy, if the generation happens on client side (app)?

[ardevd]

Is it possible to replicate the code for code generation in jlrpy, if the generation happens on client side (app)?

Should be, but first we need to figure out how it all works. The app is heavily obfuscated, so it's a bit of work.

[siobhanellis]

To add to both @ismarslomic and @ardevd, most other manufacruters also have 'private' and undocumented apis but do not forceably try and stop 3rd party apps like this. If you just look at the list in HA for different makes, there are many.

I feel their approach is both narrow minded and old fashioned. We can also note that they have blocked Octopus Energy from using it too with their energy product designed to allow cheaper charging of EV and PHEV cars, thus meaning JLR customers will pay more!

I think we need to find some route into JLR to have a grown up discussion as to how we can utilise this api to the benefit of their paying customers, which we all are.

As a note, just had to pay for my Incontrol service for the first time @ £280 for the year!

Complain to the Customer Care Centre.

[MZorzy]

this may help ? https://github.com/evcc-io/evcc/pull/13960/commits/808fc8bef3f2bc01c4352a90fcdc0c27aef47489

[ardevd]

Unfortunately it doesn't work.

[CorceiroPT]

Don't give up! I'm sorry I can't help much, but I try to send good vibes! You can do it!

[pmharris77]

Had a reply from Jag on X, saying: "We are developing authorised solutions for Intelligent/Smart charging and some smart home services as quickly as possible, that will be tested and warranted for quality and security.

Quite how jlrpy or indeed HomeAssistant become authorised solutions if they aren't even talking to anybody about it, I have no idea.

[ardevd]

Reading that I doubt JLR will be engaging or vetting any open source solution such as jlrpy. They are probably working with commercial energy and charger provider companies and that's it.

[MightyWomble2020]

Sadly I think this has now gone the same way as the Life360 integration and there won't be a workable solution anytime soon if at all. I think the time has come to disable the service which is a real shame as I found it useful as it displayed more information than the official app did.

Thanks for all your hard work everyone who contributed to the integration.

[piotrpot]

Have you guys tried reverse engineering the current version of the official remote app? I mean setting up a proxy with a custom Fiddler SSL certificate, rebuilding the APK with a modified manifest to allow user certificates, and then monitoring the decrypted HTTPS traffic to see what they are doing under the hood? I assume you have, but I'm just curious if it is still worth trying or if it has already been analyzed.

[msp1974]

Been done. The app secret is now dynamic (previously it was fixed and found using above). However the code is DexGuarded and the challenge is working out the code that generates it.

[web-dc]

Unfortunately I'm not able to fetch the requests from ios app with Proxyman. Did you check e.g. with ChatGPT some intersections between the app keys? Looks like uuid, maybe it's just uuid7 with a "fresh" timestamp?

[ardevd]

Easiest way (on Android) to get the API requests is by instrumenting the (obfuscated) methods that invoke http requests. However, I don't see any exchange of app keys. Hence my theory is that the app creates a UUID encoded hash based on the device-id, timestamp, etc. I just haven't had the time to dissect the mechanism fully.

[garrettcook]

I am very inexperienced in these matters and don't know what current or previous approaches you have done, so this comment may be of limited value. But I do notice that the 2.18.5 apk seems to be compiled and obfuscated in a way that is different from two recent others (2.18.0 and 2.18.3). That actually did help me recognize what certain parameters and strings originally meant, as there are some elements in 2.18.0 and 2.18.3 that are obfuscated but not in 2.18.5, or vice-versa. There are aspects of the earlier versions that lead me to think that an encoding mechanism for the api-key was being developed by the authors but not fully propagated to all the java functions. I stopped poking around in 2.18.5 when I got to a point where what appeared to be a dynamic api-key value became a massive multi-function calculation. I wish I had better tools to discern what is going on.

[web-dc]

@garrettcook Do we have to understand the calculation of the api key? As long as we do find the functions call we may could abstract it and use it as it is without understanding it?

[garrettcook]

@web-dc I'm not really sure, as so far it is beyond my expertise. I do believe the app is generating the key dynamically using time or a random number generator as an input, then at some point notifying the server that the key is no longer valid and generates a new key. So many functions use multiple layers of other functions that it is difficult to parse.

[markfordham2]

Is there any update on this? I still seem to be unable to connect sadly. This was a great integration when it was working so hoping a solution was found

[ismarslomic]

We will share any updates as soon as we have those. Please do not spam this thread with status questions and other comments not contributing to the solution.

[metalmarco]

I think I found a way to access our Jaguar / Land Rover vehicles APIs using this service: https://smartcar.com/ The SmartCar.com service seems legitimate, as it's been used by insurance companies and car sharing services like Uber to monitor their fleet.

I've created an account, and followed the instructions to create my first app and retrieve the Client ID and Secret.

Then, I downloaded their Postman APIs from here: https://www.postman.com/smartcar/workspace/smartcar-api/collection/13172949-de596180-4770-4b4c-b128-31679d28e170

I've setup the Authorization, adding my Client ID, Secret and a redirect URL (I used http://localhost:3000, it's not important but I think you need to enabled it on the SmartCard dashboard).

After that, I was able to get an access Token and retrieve my Evoque data from their APIs. I tried to retrieve several info, and the results are all correct.

So, as a proof of concept. this integration works.

The service looks very developer-oriented, providing SDKs, documentation and sample codes in several different languages including Python and Node.js.

I work in Software development but I'm not a developer. I'm willing to assist anyone capable of trying to use this APIs to create a new Home Assistant integration for our cars.

The most difficult thing would be how to find a way to let HA users to self configure the integration, because it requires the user to have both a SmartCar and an InControl account. Next we need to figure out how to maintain the access token updated without having to perform the full oAuth everytime.

There is a limitation: the platform provides a free tier with 300 monthly requests per vehicle. So it looks like if we restrict the info that we request, we may be able to refresh the vehicle data for free at least once/day, which is not bad considering that currently we have no integration.

Another important factor to consider, is that this SmartCar company somehow has legitimate access to Jaguar / Land Rover APIs! So it looks like JLR somehow allows access to their APIs to some companies...

What do you guys think?

---

Update: After a little bit of research, it looks like somebody already developed a SmartCar integration for Home Assistant, but it's 3 years old. Maybe it could be a starting point? https://github.com/Cquential/SmartCar-Integration

[ismarslomic]

Thanks @metalmarco! This is really interesting information! I have not spent a lot of time thinking about all the (good and relevant) questions you are raising, but I want to share my thoughts right away.

The smartcar.com API is conceptually replacing the jlrpy python api maintained by @ardevd , at least seen from the jlrincontrol point of view. But the smartcar API is a proxy API towards Jaguar and multiple other brands, and is probably maintained by larger team of developers. That is a good thing. For instance, I like their uptime monitoring, and its very interesting that they did also experience downtime from JLR API on may 17 2024. But the integration went healthy again may 23 2024, so they obviously have legit access to the JLR API, which we don't.

I did a quick search and found two related feature request on HA forum, for getting an integration for smartcar.com (no integrations exists as far as I can tell):

• https://community.home-assistant.io/t/smartcar-com-integration/292538
• https://community.home-assistant.io/t/connected-car-lots-of-similar-request-new-idea-smart-car/609321

So, to comment on your concerns/questions:

The most difficult thing would be how to find a way to let HA users to self configure the integration, because it requires the user to have both a SmartCar and an InControl account. Next we need to figure out how to maintain the access token updated without having to perform the full oAuth everytime.

HA support OAuth authentication out of box, with automatic refreshing of token. So I don't see any issue with this at all. But yes, you are right. Users will need SmartCar account + InControl account in order to use new version of our integration.

There is a limitation: the platform provides a free tier with 300 monthly requests per vehicle. So it looks like if we restrict the info that we request, we may be able to refresh the vehicle data for free at least once/day, which is not bad considering that currently we have no integration.

If data freshness once/day is sufficient or not depends highly on the usage of the data. Automations will suffer big time with this refresh interval, but at least you can get key info about your car for historical usage? Anyway, users wanting to have more fresh data can choose to buy a plan that increase the number of request per day. But this changes the whole spirit of HA, it's your data and you can do what ever you want whenever you want, for free. But in reality, JLR does not share this open source, free data approach.

Another important factor to consider, is that this SmartCar company somehow has legit access to Jaguar / Land Rover APIs! So it looks like JLR somehow allows access to their APIs to some companies...

I do agree that it looks like they have legit access to JLR API. They do also list Jaguar as compatible vehicle on their list.

So I see pros and cons with smartcar.com:

Pros:

• we can get our jlrincontrol integration back on track and see our vehicle data in HA 🍾
• we have more developer oriented API to work with, that is backed up with an development team monitoring the service health continuously

Cons:

• we should take time to do a larger refactoring of the existing code, if we choose to switch to the new API. For instance this integration should be updated to follow the latest integration standards from HA
• we are getting dependency towards an (commercial) third party (with no open source) we do not have any relation to and who we must trust will be there for the next years.
• to really get use of the vehicle data in HA, for instance for automation, users of our integration will need to pay for it

It's interesting to hear thoughts from @msp1974 and @ardevd. From my point of view I would rather wait little bit longer, to see if JLR provides any alternative solution after closing the access to their APIs.

[metalmarco]

I agreed with what you wrote. I think SmartCar.com should be a separate HA integration, because it could serve any car manufacturer supported by them, and not specifically Land Rover.

I also agree that the once/day updates will make any automation impossibile. But at least it will bring our car back in our dashboards.

Paying SmartCar subscription could increase the refresh rate, but also enable the webhooks functionality which I haven't studied yet and may be useful.