JLR blocked API access again

[JonSilver]

JLR app is fine, the HA integration isn't, since midnight today.

[msp1974]

Adding @ardevd to this issue. I see the same with a 407 Proxy Auth Required error.

[alainsch]

me too. Same in wattcat

[BenjiBoisemont]

The same for me, connection lost with jlr last night. MyPace app also broken on iPhone

[ardevd]

I'm on it.

[tdipilot01]

My Android Jaguar Remote app works, but HA integration's been down all day.

[ardevd]

It's going to take some time to figure out the new update but I think it's doable.

[krenlund]

Phew, this is a relief! Thank you @ardevd in advance!

[ardevd]

Phew, this is a relief! Thank you @ardevd in advance!

No promises! :) But I'll try!

[Jazmodo]

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

[callumeveratt]

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

It's the latter - they've taken a stance against third party apps recently.

[pmharris77]

Apologies if this has already been answered, and many many thanks for getting this working at all!

I just wondered if the changes by JLR were simply updates that are breaking the integration? Or are JLR purposefully breaking it?

If the latter, the I’d be happy to start harassing JLR to stop doing that. I’m sure many JLR drivers find this (and other) 3rd party apps useful, and they need to realise they’re at risk of losing future business if they keep a walled-garden approach.

Same question asked here on jlrpy repo (the API library this integration uses to talk to JLR servers) https://github.com/ardevd/jlrpy/issues/131#issuecomment-2118657099

They are doing it on purpose in a game of cat and mouse with third party apps.

Looks like they've added an extra layer of authentication this time according to @ardevd. It's annoying as I find the jlrpy library and this integration incredibly useful, but I guess I can see that JLR don't want potentially malicious actors hacking JLR cars using their API, especially with the recent Range Rover security scandal...

[pmharris77]

FYI, for those interested in the underlying issue within the jlrpy Python library, see here: https://github.com/ardevd/jlrpy/issues/131

[ardevd]

I'd argue that

a) you can't really lock out third party API access, you can just make it tedious for people to access it.

b) locking out API access is the wrong solution to a problem that doesn't exist.

JLR could easily put restrictions on sensitive remote operations. If they wanted to prevent malicious remote unlocks they could make the double lock disable that feature for example.

I doubt any Land Rovers have been stolen through "malicious" third party API access.

Furthermore, the way JLR have consistently undermined years of community effort by arbitrarily changing their APIs without notice or explanation is incredibly hostile.

[pmharris77]

I'd argue that

a) you can't really lock out third party API access, you can just make it tedious for people to access it.

b) locking out API access is the wrong solution to a problem that doesn't exist.

JLR could easily put restrictions on sensitive remote operations. If they wanted to prevent malicious remote unlocks they could make the double lock disable that feature for example.

I doubt any Land Rovers have been stolen through "malicious" third party API access.

Furthermore, the way JLR have consistently undermined years of community effort by arbitrarily changing their APIs without notice or explanation is incredibly hostile.

@ardevd I agree. You can't drive a JLR away without the key being present. At most you can unlock the car and could steal stuff inside, but you'd have to know the car owner's app credentials, which would only be possible if they had a data breach leaking car reg or VIN mapped to user account details...

I wonder what other manufacturer's approach to third party API access has been?

[ismarslomic]

I agree that this approach from JLR does not improve the overall security, in regards to third party usage.

But from my understanding, the main issue in this context is that the JLR API we are using is not designed nor handled as third party API. Its basically private API designed for JLR app(s), which they have full control of and can handle any breaking changes in the API. The API is not documented, nor supported for third parties by JLR. We basically did reverse engineering to understand how it works and how to make use of it. So the main problem is that JLR does not provide any official public API with close collaboration with the open source community.

In that meaning, we should not be frustrated over breaking changes in the private API, but rather that JLR ignores third party apps and innovative usage.

[ardevd]

JLR can do what they want with their private APIs. My argument is simply that it's incredibly dumb to agitate your most enthusiastic customers by cutting off community tools that have been developed and maintained over several years.

[ismarslomic]

Yeah, agreed to that!

[Jazmodo]

Thanks for the replies @ardevd 🙏 && I completely agree.

As you say, cutting off access to some of their more enthusiastic owners seems counter-productive. I love the ability to integrate Car charging/location features into Home Assistant, fantastic you've managed to get it working at all.

Not sure I can help much, but I can be a small but persistent voice, if nothing else.

FYI, for those interested in the underlying issue within the jlrpy Python library, see here: ardevd/jlrpy#131

I'll check this out! Cheers 😃

[msp1974]

To add to both @ismarslomic and @ardevd, most other manufacruters also have 'private' and undocumented apis but do not forceably try and stop 3rd party apps like this. If you just look at the list in HA for different makes, there are many.

I feel their approach is both narrow minded and old fashioned. We can also note that they have blocked Octopus Energy from using it too with their energy product designed to allow cheaper charging of EV and PHEV cars, thus meaning JLR customers will pay more!

I think we need to find some route into JLR to have a grown up discussion as to how we can utilise this api to the benefit of their paying customers, which we all are.

As a note, just had to pay for my Incontrol service for the first time @ £280 for the year!

[ismarslomic]

Agree with you @msp1974! We should initiate dialog with JLR. They already know that their API is used by many third parties, so it should not be an surprise that this approach does not work for anyone. Im using Tibber as energy provider in Norway, and they also use the same API to connect to the vehicle in order to do smart charging when the electricity is cheap. This service is currently broken too, and I need to manually start charging when the price is cheap.

Does anyone know a name of the product owner or developer of the JLR API? I think we need to reach out directly and not through the customer support.

[pmharris77]

Considering they are about to exclusively sell electric cars, blocking third party API access, whilst not even providing any official integrations themselves, is just madness and is just going to make current and future customers go to their competitors. Way to go bust JLR! 😅

[ardevd]

@ismarslomic all my previous JLR contacts with positions related to connected car tech have left the company. I haven't heard from JLR reps for years.

[adec]

Until recently I used to work for JLR but now work for one of their suppliers. I'll see if I can find a way to raise this internally with them. I can't promise anything but I will do what I can. I'd quite like to use the integration myself but would be more comfortable if the public API allowed some of the more risky options to be disabled (such as opening the car doors).

[ardevd]

Until recently I used to work for JLR but now work for one of their suppliers. I'll see if I can find a way to raise this internally with them. I can't promise anything but I will do what I can. I'd quite like to use the integration myself but would be more comfortable if the public API allowed some of the more risky options to be disabled (such as opening the car doors).

That would be awesome!

Personally, I don't think remote car lock/unlock should be a thing. But a way to opt out would be nice.

[pmharris77]

Been doing a lot of reading about the whole Octopus smart charging saga with API access being v suddenly removed for energy companies back in March. There has been a very public backlash on X, and someone mentioned that JLR stated that they were working on improving the API security. Given the amount of complaints that both Octopus and JLR have had about removing support, it would be difficult to imagine they aren't talking about how to resolve the issue.

It got me thinking, maybe the extra app secret layer added to the API authentication is being introduced as a way to allow JLR-approved third party apps, such as Octopus, to access to the API in the future with a JLR-issued app secret? If so, this might actually be a positive sign, if they are willing to engage with third-party app developers. I can live in hope 🤞

[pmharris77]

Some people have gotten the following response from JLR recently, which does state they are "working to develop authorised solutions to smart home and energy services as quickly as possible"

Full response:

“We are responsible for the performance and safety of our vehicles and provide a warranty for this. We are not able to test, approve and warrant the safety and security of unauthorised third-party access to our vehicles and its data. This is a permanent change and only approved, authorised and tested apps will have access to our vehicles and their data. We are working to develop authorised solutions to smart home and energy services as quickly as possible. We apologise for any inconvenience this may have caused, but hope you understand why this was necessary.”

[ardevd]

Been doing a lot of reading about the whole Octopus smart charging saga with API access being v suddenly removed for energy companies back in March. There has been a very public backlash on X, and someone mentioned that JLR stated that they were working on improving the API security. Given the amount of complaints that both Octopus and JLR have had about removing support, it would be difficult to imagine they aren't talking about how to resolve the issue.

It got me thinking, maybe the extra app secret layer added to the API authentication is being introduced as a way to allow JLR-approved third party apps, such as Octopus, to access to the API in the future with a JLR-issued app secret? If so, this might actually be a positive sign, if they are willing to engage with third-party app developers. I can live in hope 🤞

It's possible but so far JLR has shown exceptionally little interest in dialog with third party developers. Maybe Octopus would be an exception, but their equivalent here in Norway, Tibber, have had no contact. I've reached out to JLR repeatedly and they have never responded.

[CorceiroPT]

I would say this time will be more difficult. It could be that they (JLR) used a proxy with different credentials (and maybe temporary rotating passwords for it) to access and validate the API keys. This would be a way to block third-party access to it and I supposed they did it...

[ardevd]

From my findings so far it seems like the only change here is that the app secret now rotates (at least) every time you start the app. It seems to be happening offline from what I can tell.

[ismarslomic]

Is it possible to replicate the code for code generation in jlrpy, if the generation happens on client side (app)?

[ardevd]

Is it possible to replicate the code for code generation in jlrpy, if the generation happens on client side (app)?

Should be, but first we need to figure out how it all works. The app is heavily obfuscated, so it's a bit of work.

[siobhanellis]

To add to both @ismarslomic and @ardevd, most other manufacruters also have 'private' and undocumented apis but do not forceably try and stop 3rd party apps like this. If you just look at the list in HA for different makes, there are many.

I feel their approach is both narrow minded and old fashioned. We can also note that they have blocked Octopus Energy from using it too with their energy product designed to allow cheaper charging of EV and PHEV cars, thus meaning JLR customers will pay more!

I think we need to find some route into JLR to have a grown up discussion as to how we can utilise this api to the benefit of their paying customers, which we all are.

As a note, just had to pay for my Incontrol service for the first time @ £280 for the year!

Complain to the Customer Care Centre.

[MZorzy]

this may help ? https://github.com/evcc-io/evcc/pull/13960/commits/808fc8bef3f2bc01c4352a90fcdc0c27aef47489

[ardevd]

Unfortunately it doesn't work.

[CorceiroPT]

Don't give up! I'm sorry I can't help much, but I try to send good vibes! You can do it!

[pmharris77]

Had a reply from Jag on X, saying: "We are developing authorised solutions for Intelligent/Smart charging and some smart home services as quickly as possible, that will be tested and warranted for quality and security.

Quite how jlrpy or indeed HomeAssistant become authorised solutions if they aren't even talking to anybody about it, I have no idea.

[ardevd]

Reading that I doubt JLR will be engaging or vetting any open source solution such as jlrpy. They are probably working with commercial energy and charger provider companies and that's it.

[MightyWomble2020]

Sadly I think this has now gone the same way as the Life360 integration and there won't be a workable solution anytime soon if at all. I think the time has come to disable the service which is a real shame as I found it useful as it displayed more information than the official app did.

Thanks for all your hard work everyone who contributed to the integration.

[piotrpot]

Have you guys tried reverse engineering the current version of the official remote app? I mean setting up a proxy with a custom Fiddler SSL certificate, rebuilding the APK with a modified manifest to allow user certificates, and then monitoring the decrypted HTTPS traffic to see what they are doing under the hood? I assume you have, but I'm just curious if it is still worth trying or if it has already been analyzed.

[msp1974]

Been done. The app secret is now dynamic (previously it was fixed and found using above). However the code is DexGuarded and the challenge is working out the code that generates it.

[web-dc]

Unfortunately I'm not able to fetch the requests from ios app with Proxyman. Did you check e.g. with ChatGPT some intersections between the app keys? Looks like uuid, maybe it's just uuid7 with a "fresh" timestamp?

[ardevd]

Easiest way (on Android) to get the API requests is by instrumenting the (obfuscated) methods that invoke http requests. However, I don't see any exchange of app keys. Hence my theory is that the app creates a UUID encoded hash based on the device-id, timestamp, etc. I just haven't had the time to dissect the mechanism fully.

[garrettcook]

I am very inexperienced in these matters and don't know what current or previous approaches you have done, so this comment may be of limited value. But I do notice that the 2.18.5 apk seems to be compiled and obfuscated in a way that is different from two recent others (2.18.0 and 2.18.3). That actually did help me recognize what certain parameters and strings originally meant, as there are some elements in 2.18.0 and 2.18.3 that are obfuscated but not in 2.18.5, or vice-versa. There are aspects of the earlier versions that lead me to think that an encoding mechanism for the api-key was being developed by the authors but not fully propagated to all the java functions. I stopped poking around in 2.18.5 when I got to a point where what appeared to be a dynamic api-key value became a massive multi-function calculation. I wish I had better tools to discern what is going on.

[web-dc]

@garrettcook Do we have to understand the calculation of the api key? As long as we do find the functions call we may could abstract it and use it as it is without understanding it?

[garrettcook]

@web-dc I'm not really sure, as so far it is beyond my expertise. I do believe the app is generating the key dynamically using time or a random number generator as an input, then at some point notifying the server that the key is no longer valid and generates a new key. So many functions use multiple layers of other functions that it is difficult to parse.