Request: Remote 3d view

[ronmichel]

Is it possible to get the image(s) of the remote 3d view? This is not possible with the connecteddrive website but only available in the app.

[ronmichel]

Any update on this? Can someone help me how to dump the requests from the app? Can't get around certificate pinning... (dump see post below)

[gerard33]

Can you elaborate what you mean with this? And do you have some screenshots from the app?

[lawtancool]

I think this refers to the ability to view the 360 camera remotely through the BMW Connected app, available on some newer (Gxx series) models. (picture from internet, my car doesn't support this)

As @ronmichel pointed out, this is only available through the app and not from the web portal, so I'm not sure supporting this will be possible unless we can figure out how the app gets the 360 images.

[ronmichel]

Hi @gerard33, in my app there is the Item: Remote 3D View, see here:

The next screen takes you to a rendered 3D image of your car:

Again the next screen gives you the possiblility to select and download the single images (front sides and back)

They look like this (fisheye camera)

It would be very handy to get a hold of these images (use them in homeassistant as camera etc...)

I cannot however get the requests to the bmw server due to certificate pinning of the app which makes mim difficult. (see below)

[ronmichel]

@gerard33 @ChristianKuehnel

Here is the dump of the connection (XXXXXXXXX for privacy):

POST
myc-profile.bmwgroup.com/api/gateway/brs/webapi/v1/user/vehicles/WBAXXXXXXXXX/executeService 

Headers:

Authorization: Bearer XXXXXXXXX
User-Agent: Android(9); bmw; 6.2.1.5860; release
Accept-Language: de-DE
X-BTC-CorrelationID: XXXXXXXXX
ClientRegion: DE
Content-Type: application/x-www-form-urlencoded
Content-Length: 1110
Connection: Keep-Alive
Accept-Encoding: gzip
Cookie: TiPMix=XXXXXXXXX; x-ms-routing-name=secondary
ADRUM_1: isMobile:true
ADRUM: isAjax:true

Body:

serviceType=REMOTE360&data={"remote360":{"publicKey":"XXXXXXXXX","encryptedPackageKey":"XXXXXXXXX","imageCompression":0,"pushToken":"xxxx"}}

Response:

{
  "executionStatus": {
    "serviceType": "REMOTE360",
    "status": "INITIATED",
    "eventId": "XXXXXXXXX@bmw.de"
  }
}

Request status of action:

GET
myc-profile.bmwgroup.com/api/gateway/brs/webapi/v1/user/vehicles/WBAXXXXXXXXX/remote360executionStatus?eventId=XXXXXXXXX%40bmw.de

HTTP/1.1

Headers:

Authorization: Bearer XXXXXXXXX
User-Agent: Android(9); bmw; 6.2.1.5860; release
Accept-Language: de-DE
X-BTC-CorrelationID: XXXXXXXXX
ClientRegion: DE
Connection: Keep-Alive
Accept-Encoding: gzip
Cookie: TiPMix=XXXXXXXXX; x-ms-routing-name=secondary
ADRUM_1: isMobile:true
ADRUM: isAjax:true

Response while in progress:

{
  "remote360Status": {
    "eventId": "XXXXXXXXX@bmw.de",
    "executionStatus": "IN_PROGRESS",
    "statusHistory": [
      {
        "statusId": 0,
        "statusCode": 3004,
        "timestamp": "1566478851420"
      },
      {
        "statusId": 1,
        "statusCode": 3006,
        "timestamp": "1566478851493"
      },
      {
        "statusId": 2,
        "statusCode": 3008,
        "timestamp": "1566478856219"
      }
    ],
    "availablePackages": []
  }

Response when finished:

{
  "remote360Status": {
    "eventId": "XXXXXXXXX@bmw.de",
    "executionStatus": "OK",
    "statusHistory": [
      {
        "statusId": 0,
        "statusCode": 3004,
        "timestamp": "1566478851420"
      },
      {
        "statusId": 1,
        "statusCode": 3006,
        "timestamp": "1566478851493"
      },
      {
        "statusId": 2,
        "statusCode": 3008,
        "timestamp": "1566478856219"
      },
      {
        "statusId": 3,
        "statusCode": 3009,
        "timestamp": "1566478862622"
      },
      {
        "statusId": 4,
        "statusCode": 3010,
        "timestamp": "1566478880594"
      },
      {
        "statusId": 5,
        "statusCode": 1006,
        "timestamp": "1566478881104"
      },
      {
        "statusId": 6,
        "statusCode": 3009,
        "timestamp": "1566478884199"
      },
      {
        "statusId": 7,
        "statusCode": 3012,
        "timestamp": "1566478888243"
      },
      {
        "statusId": 8,
        "statusCode": 2004,
        "timestamp": "1566478889585"
      },
      {
        "statusId": 9,
        "statusCode": 3012,
        "timestamp": "1566478903792"
      },
      {
        "statusId": 10,
        "statusCode": 3012,
        "timestamp": "1566478915107"
      },
      {
        "statusId": 11,
        "statusCode": 3012,
        "timestamp": "1566478924772"
      },
      {
        "statusId": 12,
        "statusCode": 3012,
        "timestamp": "1566478946858"
      }
    ],
    "availablePackages": [
      {
        "dataPackageId": 6443285,
        "filenamePackage": "le.mp4"
      },
      {
        "dataPackageId": 6443278,
        "filenamePackage": "re.mp4"
      },
      {
        "dataPackageId": 6443277,
        "filenamePackage": "ri.mp4"
      },
      {
        "dataPackageId": 6443276,
        "filenamePackage": "fr.mp4"
      },
      {
        "dataPackageId": 6443274,
        "filenamePackage": "Mei-Calibration.xml"
      }
    ],
    "mobileNetworkRssi": 13,
    "mobileNetworkTechnology": "LTE",
    "position": {
      "lat": 48.139305,
      "lon": 11.548841,
      "heading": 342,
      "status": "OK"
    }
  }
}

Download picture:

GET
myc-profile.bmwgroup.com/api/gateway/brs/webapi/v1/user/vehicles/WBAXXXXXXXXX/remote360downloadPackage?eventId=XXXXXXXXX%40bmw.de&dataPackageId=6443285 

HTTP/1.1

Headers:

Authorization: Bearer XXXXXXXXX
User-Agent: Android(9); bmw; 6.2.1.5860; release
Accept-Language: de-DE
X-BTC-CorrelationID: XXXXXXXXX
ClientRegion: DE
Connection: Keep-Alive
Accept-Encoding: gzip
Cookie: TiPMix=XXXXXXXXX; x-ms-routing-name=secondary
ADRUM_1: isMobile:true
ADRUM: isAjax:true

[gerard33]

This is apparantly using an url which is a bit different -> myc-profile.bmwgroup.com/api/gateway/brs/webapi/v1/user/vehicles/WBAXXXXXXXXX/executeService

As this is not used in the current implementation, this is not something I can add without some major changes I guess. But it can be interesting to do some more test with this new url to see if we can get the same data and to check if it's more realible than the current API.

Can you check if you can get the status of the car when using this url?

myc-profile.bmwgroup.com/api/gateway/brs/webapi/v1/user/vehicles/WBAXXXXXXXXX/status

[robthebold]

Aha. This shows why BMW provides functionality to download rendered views of your car from multiple angles and with transparent backgrounds: So the app can superimpose one from the right POV on top of the remapped and stitched-together car cam images to show a facsimile of the surroundings with your own car in the middle. Pretty cool.

[ronmichel]

get the status

@gerard33

Thanks for your interest. Yes, a GET requests to

https://myc-profile.bmwgroup.com/api/gateway/brs/webapi/v1/user/vehicles/WBAXXX/status?dlat=0.0&dlon=0.0&deviceTime=2019-09-04T14%3A56%3A08

results in the full status report

[gerard33]

Cool! Can you share that status report with me?

And also how you got the bearer token and other xxxxxx stuff in the header? Then I can have a look if this url is more reliable than the currently used url.

[ronmichel]

status report

{"vehicleStatus":{"vin":"WBAXXX","mileage":8739,"updateReason":"CHARGING_DONE","updateTime":"2019-08-22T12:33:13+00:00","doorDriverFront":"CLOSED","doorDriverRear":"CLOSED","doorPassengerFront":"CLOSED","doorPassengerRear":"CLOSED","windowDriverFront":"CLOSED","windowDriverRear":"CLOSED","windowPassengerFront":"CLOSED","windowPassengerRear":"CLOSED","trunk":"CLOSED","rearWindow":"INVALID","hood":"CLOSED","doorLockState":"SECURED","parkingLight":"OFF","positionLight":"OFF","remainingFuel":34,"remainingRangeElectric":41,"remainingRangeElectricMls":25,"remainingRangeFuel":558,"remainingRangeFuelMls":346,"maxRangeElectric":43,"maxRangeElectricMls":26,"connectionStatus":"CONNECTED","chargingStatus":"FINISHED_FULLY_CHARGED","chargingLevelHv":100,"lastChargingEndReason":"CHARGING_GOAL_REACHED","lastChargingEndResult":"SUCCESS","position":{"lat":48.139305,"lon":11.548841,"heading":342,"status":"OK"},"publicKey":"XXX","internalDataTimeUTC":"2019-08-22T12:33:13","singleImmediateCharging":false,"chargingConnectionType":"CONDUCTIVE","chargingInductivePositioning":"NOT_POSITIONED","steering":"LH","vehicleCountry":"DE","checkControlMessages":[],"cbsData":[{"cbsType":"OIL","cbsState":"OK","cbsRemainingMileage":23000,"cbsDueDate":"2020-12","cbsDescription":"Nächster Service nach der angegebenen Fahrstrecke oder zum angegebenen Termin."},{"cbsType":"VEHICLE_CHECK","cbsState":"OK","cbsRemainingMileage":60000,"cbsDueDate":"2022-12","cbsDescription":"Nächste Sichtprüfung nach der angegebenen Fahrstrecke oder zum angegebenen Termin."},{"cbsType":"BRAKE_FLUID","cbsState":"OK","cbsDueDate":"2021-12","cbsDescription":"Nächster Wechsel spätestens zum angegebenen Termin."}],"DCS_CCH_Activation":"ACTIVE","DCS_CCH_Ongoing":false,"climateFunction":"SUPPORTED"}}

[gerard33]

Quick check on the phone shows me the data format is the same. I can make a new branch of the library which uses this new url to see if we can get that working. Will probably need some more info from you how you got all data used in the headers.

[ronmichel]

I just used (copied) the token from the app which seems to be valid one hour. The app used an oauth process:

1. POST to

   https://customer.bmwgroup.com/gcdm/oauth/token

with http Basic auth and in the data

grant_type=password&username=XXXX&password=XXX

which gives an access and refresh token.

2. GET (with some ocp-apim-subscription-key in the Header)

https://myc-profile.bmwgroup.com/api/Account/ExternalLogin?provider=GCDM&response_type=token&redirect_uri=https%3A%2F%2Fmyc-profile.bmwgroup.com%2F&client_id=self

which gives a state token

3. GET (also with the Same ocp-apim-subscription-key)

https://myc-profile.bmwgroup.com/oauth_callback.html?scope=authenticate_user&refresh=XXX&state=XXX&token=XXX

which sets the cookie with the final bearer token.

[gerard33]

Thanks! Hopefully I have some time next week to test this.

[gerard33]

This is still on my to do list, but not sure when I have time to dive into it. So if anyone wants to give it a try :+1:

[ronmichel]

I am sorry but I don't have the skills... For the oauth login to myc-profile.bmwgroup.com maybe https://github.com/requests/requests-oauthlib could help?

[rikroe]

@ronmichel cool stuff, however I need your help!

I've tried to rebuild the authentication flow you described by hand, both using Postman and in Python . I can follow your steps and retrieve a really long token in a cookie from oauth_callback.html, but cannot get any vehicle data - the error message just reads Authorization has been denied for this request..

Could you please try if you are able to get some results (either in Python or Postman)? Or alternatively, how did you manage to get around the certificate pinning of your Android app? I tried on Android 9 using mitmproxy but failed...

[ronmichel]

how did you manage to get around the certificate pinning of your Android app?

@rikroe I use https://appcloner.app to create a copy of the original connected app with the setting: "network->trust all certificates" enabled and in "clone-mode: manifest" (Also used this to allow screenshots within the app)

[ronmichel]

@rikroe there is one more step I did not see before:

with the header "Cookie:.AspNet.Correlation.GCDM=; .AspNet.ExternalCookie=xxxxx" (this should be the really long string you already mentioned) you have to do another (second) GET to:

https://myc-profile.bmwgroup.com/api/Account/ExternalLogin?provider=GCDM&response_type=token&redirect_uri=https%3A%2F%2Fmyc-profile.bmwgroup.com%2F&client_id=self

for the access token. This token then is used with https://myc-profile.bmwgroup.com/api/gateway/brs/webapi/v1/user/vehicles/WBA...

[rikroe]

Alright, I am able to authenticate myself to myc-profile.bmwgroup.com using python 🎉

However for my car, the result is rather small (probably too old...). Will try compare the results of the old api vs. the new api sometime next week.

Also thanks for the tip regarding appcloner! Will try that out sometime!

[ronmichel]

Cool! @rikroe : Can you share your python code here or in some gist?

[rikroe]

Sure, I've updated the python gist. Works standalone as python script and only needs requests installed.

@robthebold could you do a quick check as well if the general setup allows also for US-based ConnectDrive accounts?

If it works for you, I will take on the task to update the current URLs and account.py to use the new APIs (including token storage and renewal).

[robthebold]

@robthebold could you do a quick check as well if the general setup allows also for US-based ConnectDrive accounts?

If it works for you, I will take on the task to update the current URLs and account.py to use the new APIs (including token storage and renewal).

I'll test it today.

[robthebold]

I put my credentials in the python module and gave it a try.

From the US, running bimmer_connected-myc.py standalone in Python3 I get:

Traceback (most recent call last):                                                                                                                                                                                           
  File "bimmer_connected-myc.py", line 57, in <module>                                                                                                                                                                       
    "refresh": gcdm_tokens["refresh_token"],                                                                                                                                                                                 
KeyError: 'refresh_token'                                                                                                                                                                                                    

[rikroe]

Did you try the US url mentioned in the code?

[robthebold]

No, 'cause I totally blew through that comment :( Oops.

Anyway . . .

when I set the url="https://customer.bmwgroup.com/gcdm/usa/oauth/token" I get tokens valid for the next 58(?) minutes.

I didn't do anything to https://myc-profile.bmwgroup.com

So that looks ok. Great!

[rikroe]

Well of you just change the gcdm URL, leave the rest and you get a valid vehicle status report that's perfect :tada:

[gerard33]

Great work! I have run the script and get valid vehicleStatus data for my car (The Netherlands).

[gerard33]

Comments for the work on the new url -> #130 so we can keep this issue only for it's original goal being the remote 3d view.

[rikroe]

@ronmichel are you able to see the the 360° degree image in the ConnectedDrive web application as well?

The discussions in #139 got me thinking if all of what we're doing is also available via the web API (which would be much easier to reverse-engineer) so we could bring this to #140 as well...

[ronmichel]

Hi @rikroe Remote 3D view is only available in the android and iOS app. sadly. I know it would be much easier with the web API... Any progress on #140 for US and china? Thanks!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[davecarlson]

I'm commenting on this rather than opening a new issue - wondering how far we are with this with the new myBMW api ?