search

reneboer / python-carnet-client

python script that emulated the VW WE Connect (formerly CarNet) web site
GNU General Public License v3.0
21 stars 9 forks source link

does not work with "we connect" update #5

Closed Mirarkitty closed 5 years ago

Mirarkitty commented 5 years ago

Seems like VW decided to update the login part of the services and this no longer works?

reneboer commented 5 years ago

Just found that out my self as well. It looks like quite a change and I do not have time the next four weeks to look at this.

Mirarkitty commented 5 years ago

I'm not a python person, I did an attempt but all I end up with is a 403 Forbidden page. :/

    base = "https://www.portal.volkswagen-we.com/"
        auth_base = "https://identity.vwgroup.io"
...
        hidden__csrf_re = re.compile('<input type="hidden" name="_csrf" value="([^"]*)"/>')
    hidden_relaystatetoken_re = re.compile('<input type="hidden" name="relayStateToken" value="([^"]*)"/>')

    def extract_hidden_relaystatetoken(r):
        return hidden_relaystatetoken_re.search(r.text).group(1)

    def extract_hidden__csrf(r):
        return hidden__csrf_re.search(r.text).group(1)

    # now get actual login page and get session id and ViewState
    r = s.get(ref_url, headers=AUTHHEADERS)
        # print "",r.text
    if r.status_code != 200:
        return ""

        hidden__csrf=extract_hidden__csrf(r);
        hidden_relaystatetoken=extract_hidden_relaystatetoken(r);
        login_url=extract_login_url(r);

        print "csrf",hidden__csrf
        print "rst",hidden_relaystatetoken

    AUTHHEADERS["Referer"] = ref_url
#   AUTHHEADERS["X-CSRF-Token"] = ''
#   AUTHHEADERS["Faces-Request"] = "partial/ajax"

    post_data = {
                'email': email,
                'password': password,
                'relayStateToken': hidden_relaystatetoken,
                '_csrf': hidden__csrf,
                'login': 'true',
    }

        print "session 5 - login at ",auth_base+login_url

        AUTHHEADERS["Content-Type"]='application/json'

    r = s.post(auth_base + login_url, data=post_data, headers=AUTHHEADERS)
Mirarkitty commented 5 years ago

Looks like the requests after logging in are very similar. No update in that part of the system.

birgersp commented 5 years ago

I'm hoping someone can find out if and how to log in to the service. I'd like to contribute in any way I can, but I'm fumbling in the dark a little. What's the best way to determine what traffic goes between the client and VW servers? Do you have to use Wireshark?

Mirarkitty commented 5 years ago

I use firefox, ctrl-shift+k brings up debug opportunities, network tab gives all the headers and talk that's going on.

jaybora commented 5 years ago

Looks like the requests after logging in are very similar. No update in that part of the system.

Mirakitty are you actively working on fixing the login part? More if I should give it a go. I'm not a python man but have done a couple of screen scraping apps before in java and golang. And I need it to work for my e-golf integration into my domoticz home automation setup.

birgersp commented 5 years ago

All hands on deck! Anyone capable and willing should give it a go, imo ;)

jaybora commented 5 years ago

I think I am halfway through the login procedure. This site has almost 10 pages to go through in order to just login.

FritzJo commented 5 years ago

I'am trying to get it working as well, but no real progress on my side. Its a shame that VW doesn't offer a public API...

jaybora commented 5 years ago

7 Is ready for you service :-) It is working for me now. I can start and stop the charging.

Mirarkitty commented 5 years ago

No, sorry, been busy.

7 didn't work for me.

"'ve entered too many wrong email/password combinations. Please wait a moment before you try again."

I get that in this post:


    r = s.post(login_action_url, data=post_data, headers=AUTHHEADERS, allow_redirects=False)

    if r.status_code != 302:
        return ""

status code is 200 and that's a new login form with that text. I can login fine in the browser.

oakedcorn commented 5 years ago

7 Is ready for you service :-) It is working for me now. I can start and stop the charging.

Great job, thanks a lot! It works perfectly for me.

I was using vw_carnet_app.py before, which provided 1% battery SoC accuracy (web api rounds to nearest 10% multiple) and outside temperature data - very useful. Unfortunately this script also got broken with the new login. It would be great if anyone is able to fix that one too.

bausi2k commented 5 years ago

Hi,

having an typecast issue? any ideas:

Traceback (most recent call last):
  File "vw_carnet_rb1.py", line 206, in <module>
    url = CarNetLogin(s,CARNET_USERNAME,CARNET_PASSWORD)
  File "vw_carnet_rb1.py", line 65, in CarNetLogin
    responseData = json.loads(r.content)
  File "/usr/lib/python3.5/json/__init__.py", line 312, in loads
    s.__class__.__name__))
TypeError: the JSON object must be str, not 'bytes'

br.kg

birgersp commented 5 years ago

@Mirarkitty #7 works like a charm for me. Are you absolutely sure there's no error on your end?

Mirarkitty commented 5 years ago

I can't be 100% sure, but it doesn't let me log in anyway. I only changed uid and password. I'm not sure why the script gives the error but not the website.

Mirarkitty commented 5 years ago

And yes, running with python 3 gives that type error.

jaybora commented 5 years ago

@bausi2k It is related to you python version. You can use python 3.7 or wait until I fix next week.

Mirarkitty commented 5 years ago

Or do this change:

login_url = json.loads(r.content).get("loginURL").get("path")

login_url = r.json().get("loginURL").get("path")
bausi2k commented 5 years ago

Or do this change:

login_url = json.loads(r.content).get("loginURL").get("path")

login_url = r.json().get("loginURL").get("path")

worked! thx

Mirarkitty commented 5 years ago

Is it just me or did the login change again? I get a two step (email, password) now.

youpixel commented 5 years ago

Today I did some changes to the code to match it with the new login screen. Sorry, it is a bit messy... But you get an idea how it could work.

Pull: https://github.com/reneboer/python-carnet-client/pull/8/commits/65a5381e9081cc6c78a4cf4ab843c5f25aa4c6fb

Raw file: https://github.com/reneboer/python-carnet-client/blob/65a5381e9081cc6c78a4cf4ab843c5f25aa4c6fb/vw_carnet_rb1.py

Cheers

jdrescher2006 commented 5 years ago

Today I did some changes to the code to match it with the new login screen. Sorry, it is a bit messy... But you get an idea how it could work.

Thank you very much, it works for me.

ottopaulsen commented 5 years ago

Thank you for the great work done here! The latest version (PR#8) works fine for me.

I am trying to make a javascript version of this procedure (node.js), but I ran into a problem. When posting the login_action_url with email and other data in the body, I get 403 Forbidden, and no further explanation. First I got a message that cookies and javascript must be enabled, but after enabling cookies, I got one step further, now just getting Forbidden.

Have you any ideas for how to get around this?

Otto

iemenuyttenhove commented 5 years ago

I made a working C# version for an Azure Function, if anyone is interested I can clean it up and share it 😉

JimKnopf1503 commented 5 years ago

Latest version dont seem to work for me. Login looks good, but i got no data from the requests. ============= RESTART: D:\Download\CarNetPython\CarNetPython.py ============= /-/msgc/get-new-messages b'{"messageList":[],"errorCode":"0"}' /-/vsr/request-vsr b'' /-/vsr/get-vsr b'' /-/cf/get-location b'' /-/vehicle-info/get-vehicle-details b'{"vehicleDetails":{"lastConnectionTimeStamp":["--","--"],"distanceCovered":"--","range":"","serviceInspectionData":"--","oilInspectionData":"--","showOil":true,"showService":true,"flightMode":false},"errorCode":"0"}' /-/emanager/get-emanager b''

I've tryed a look at the app protocol using burp but I gave up, it looks verry complexe.

Thanks for help, Burkhard

ottopaulsen commented 5 years ago

I made a working C# version for an Azure Function, if anyone is interested I can clean it up and share it 😉

Yes, @iemenuyttenhove, I would like to take a look, as I am struggling with my Javascript/node.js version. I have noticed that the CarNet app uses X-XSRF-TOKEN, but it seems to not be used in the Python version, and I cannot figure out where it is coming from. I am getting Forbidden (see above).

Otto

0rsa commented 5 years ago

I have this error with youpixel modification: Traceback (most recent call last): File "/home/flo/carnet/vw_carnet_app.py", line 271, in url = CarNetLogin(s,CARNET_USERNAME,CARNET_PASSWORD) File "/home/flo/carnet/vw_carnet_app.py", line 170, in CarNetLogin portlet_code = extract_code(r.url) File "/home/flo/carnet/vw_carnet_app.py", line 80, in extract_code return authcode_re.search(r).group(1) AttributeError: 'NoneType' object has no attribute 'group'

FIXED: you need to login at least once to the new desktop portal (to set some preferences at first login) and then it works! Thanks!

birgersp commented 5 years ago

@ottopaulsen

Thank you for the great work done here! The latest version (PR#8) works fine for me.

I am trying to make a javascript version of this procedure (node.js), but I ran into a problem. When posting the login_action_url with email and other data in the body, I get 403 Forbidden, and no further explanation. First I got a message that cookies and javascript must be enabled, but after enabling cookies, I got one step further, now just getting Forbidden.

Have you any ideas for how to get around this?

Otto

Yes, @iemenuyttenhove, I would like to take a look, as I am struggling with my Javascript/node.js version. I have noticed that the CarNet app uses X-XSRF-TOKEN, but it seems to not be used in the Python version, and I cannot figure out where it is coming from. I am getting Forbidden (see above).

Otto

I'm having the same issue as you, 403 Forbidden on the POST request (but in a Kotlin app). I'm going to clean up the python script, to try to pinpoint exactly what it does different compared to my app. Please let us know if you find out anything.

reneboer commented 5 years ago

Hi,

Thanks for all your help on this. I will give it a test run and update the master.

Cheers Rene

birgersp commented 5 years ago

@ottopaulsen in an attempt to figure out how to succesfully port the code and get the changes working, I've created my own repo to clean up this code as much as possible. If you're interested: https://github.com/birgersp/carnet-client

Mirarkitty commented 5 years ago

I can't login using firefox right now, so I'm not sure what's up.

birgersp commented 5 years ago

I can login without problems. Using Chrome, and using the norwegian VW website (if it matters). Script is also working fine.

youpixel commented 5 years ago

@ottopaulsen in an attempt to figure out how to succesfully port the code and get the changes working, I've created my own repo to clean up this code as much as possible. If you're interested: https://github.com/birgersp/carnet-client

Great! Looks solid and works like a charm

FritzJo commented 5 years ago

@ottopaulsen in an attempt to figure out how to succesfully port the code and get the changes working, I've created my own repo to clean up this code as much as possible. If you're interested: https://github.com/birgersp/carnet-client

works perfectly for my usecase! Thanks. Will you create a pull request?

reneboer commented 5 years ago

Hi,

I am working on a new version taking a lot of the changes done by both @ottopaulsen and @birgersp and updating the commands supported as there are some changes.

I will merge all that next week I think.

Cheers.

birgersp commented 5 years ago

works perfectly for my usecase! Thanks. Will you create a pull request?

I would, if I thought it had a chance of being merged ;-) Also, I'm planning to add my Kotlin code to the same repository, once I get the time for it.

@reneboer Please note you have 3 PRs. And if you're looking for contributions and collaboration, I suggest you consider merging (or rejecting) them. If you change the codebase now without merging, you're just messing up our PRs.

Mirarkitty commented 5 years ago

Seems like my ban is over and I can use this (#8) again.

reneboer commented 5 years ago

All PRs are now merged. The last one (#8) gave some conflicts so I have too see if it all is working as it should now. New version is coming later.

reneboer commented 5 years ago

Hi, I created a development branch with a new script called we_connect_client.py. It does not seem reliable though, but the portal gives me problems right now as well. On the portal I get this after the login instead of the overview page

System - MBB-AuthorizationServer Status - 503 - Service Unavailable

Not sure if it is because of too many logins in a short period or what else. I get the same with the other carnet_client.py script from @birgersp btw.

Please give it a try and let me know your experiance. @Mirarkitty @birgersp, @ottopaulsen @youpixel

birgersp commented 5 years ago

Just gave the script a go, and I didn't have any issues with it

reneboer commented 5 years ago

Must have been the portal as the script is working fine for me now too. I made a few more updates and I think it is ready to go out, and close this issue.

birgersp commented 5 years ago

I agree, this issue is long gone now ;-)

Mirarkitty commented 5 years ago

The portal is up and down a lot. I keep logs... This script is working a lot better than their backend XD

reneboer commented 5 years ago

Hi, I think the script is stable for the WE Connect portal so I am closing this issue. All of you thanks for your help.

Mirarkitty commented 5 years ago

Yes, seems stable now.

JimKnopf1503 commented 5 years ago

Hi!

I have still problems. Login is working. Get all cars ist working. Car info is working. But: get location = errorcode 2 get vehicle details = errorcode 0, but no data {"vehicleDetails":{"lastConnectionTimeStamp":["--","--"],"distanceCovered":"--","range":"","serviceInspectionData":"--","oilInspectionData":"--","showOil":true,"showService":true,"flightMode":false},"errorCode":"0"}

get emanager: {"errorCode":"0","EManager":{"rbc":null,"rpc":null,"rdt":null,"actionPending":false,"rdtAvailable":false}}

In my browser all infos are displayed.

This is an eGolf, as the second car, I changed the line "vin= response .....[1]" Burkhard

Mirarkitty commented 5 years ago

That's odd. Location works for me. Make a new ticket though?

reneboer commented 5 years ago

Hi, Yes please make a new issue. So we can track under that. I noticed in the latest responses from the site that the VIN of the car is now part of the URL to use and it as I have only one car I only see that. I am assuming if you just replace that with the VIN of the second car it will work. This will require some changes though. I can make an extra parameter for example.

Mirarkitty commented 5 years ago

Did they break it again?

birgersp commented 5 years ago

No. What makes you think that they did?