Play Console Authentication

[devgianlu]

I want to create an API to be able to fetch information from the Play Console exactly how this app does. However, since it's impossible to run our copy of the app (then no step by step debug is possible) I'd like someone, if possible, to explain how the app authenticate and how it is allowed to access the web page (the only thing I've not figured out yet).

I'll be creating an open-source repo for the API in Java.

Thank you in advance.

[hasanjamshaid]

I think, this app is not using public android developer API to fetch data. It is using API used by google chrome.

Android developer API supports very limited features.

[devgianlu]

Yes, I now that. I only need to understand how the authorization bearer is generated since it can't be created with Google SignIn API because a scope for the developer console doesn't exist.

[nelenkov]

Do ready the Wiki and the code:

https://github.com/AndlyticsProject/andlytics/wiki/Developer-Console-v2---Login-Process https://github.com/AndlyticsProject/andlytics/tree/master/src/com/github/andlyticsproject/console/v2

You should be able to run a copy of the app using the password-based authenticator: https://github.com/AndlyticsProject/andlytics/blob/master/src/com/github/andlyticsproject/console/v2/PasswordAuthenticator.java

[devgianlu]

Unlucky the login process is not similar to that one anymore. The AD cookie doesn't even appear on any of the requests. Also the URLs aren't the same, 'https://play.google.com/apps/publish/v2/' never appears.

What I found is the mobile api used in the Play Store app. The API base url is 'https://play.google.com/apps/publish/mobileapi/1' and I've successfully obteined some data (as protobuf) using the bearer token generated from the app (capturing packets).

These are the endpoints:

    @POST("/experiment/apply")
    fty applyExperimentVariant(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("experimentId") String str2, @Query("variantIndex") int i, @Body fsm fsm);

    @POST("/experiment/end")
    fty endExperiment(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("experimentId") String str2, @Body fsm fsm);

    @GET("/applications/ids")
    fty fetchAppIds(@Header("Authorization") String str, @Query("account") long j);

    @GET("/applications/releases")
    fty fetchAppReleases(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("minTimestampMs") long j3);

    @GET("/applications/state")
    fty fetchAppState(@Header("Authorization") String str, @Query("account") long j, @Query("app") long... jArr);

    @GET("/applications/details")
    fty fetchApps(@Header("Authorization") String str, @Query("account") long j, @Query("app") long... jArr);

    @GET("/accounts")
    fty fetchDeveloperAccounts(@Header("Authorization") String str);

    @GET("/errorclusterstats")
    fty fetchErrorClusterStats(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("errorType") int i, @Query("errorId") long j3);

    @GET("/errorclusters")
    fty fetchErrorClusters(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("errorClusterType") int i, @Query("minTimestampMs") long j3, @Query("excludeMuted") boolean z, @Query("maxClusterCount") Integer num, @Query("latestAppVersionOnly") boolean z2, @Query("dailyVolumeNumDays") int i2);

    @GET("/ratings")
    fty fetchRatings(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2);

    @GET("/applications/releasetracks")
    fty fetchReleaseTracksSummary(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2);

    @GET("/infomessages")
    fty getInfoMessages(@Header("Authorization") String str, @Query("account") long j);

    @GET("/reviews")
    fty getReview(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query(encodeValue = false, value = "review") String str2);

    @GET("/reviews")
    fty getReviews(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("languageCode") String str2, @Query("versionCode") Integer num, @Query("minTimestampMillis") Long l, @Query("maxTimestampMillis") Long l2, @Query("rating") Iterable<Integer> iterable, @Query("token") String str3, @Query("count") int i);

    @GET("/statistics/series")
    fty getStatisticsSeries(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("statsType") int i, @Query("fromDaysAgo") int i2, @Query("timeZone") String str2, @Query("convertToDaily") boolean z);

    @GET("/statistics/series/batch")
    fty getStatisticsSeriesBatch(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("statsType") Iterable<Integer> iterable, @Query("fromDaysAgo") int i, @Query("timeZone") String str2, @Query("convertToDaily") boolean z);

    @GET("/statistics/table")
    fty getStatisticsTable(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("statsType") int i, @Query("fromDaysAgo") int i2);

    @GET("/statistics/table/batch")
    fty getStatisticsTableBatch(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("statsType") Iterable<Integer> iterable, @Query("fromDaysAgo") Iterable<Integer> iterable2);

    @POST("/canary/halt")
    fty haltCanary(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Body fsm fsm);

    @GET("/listingexperiments")
    fty listingExperiments(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2);

    @POST("/reviews/reply")
    fty replyToReview(@Header("Authorization") String str, @Query("account") long j, @Body fup fup);

    @POST("/canary/resume")
    fty resumeCanary(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Body fsm fsm);

    @POST("/canary/fraction")
    fty setCanaryFraction(@Header("Authorization") String str, @Query("account") long j, @Query("app") long j2, @Query("fraction") double d, @Body fsm fsm);

    @POST("/dismissables")
    fty updateDismissablesState(@Header("Authorization") String str, @Query("account") long j, @Body fve fve);

    @POST("/apps/pinned")
    fty updatePinnedApps(@Header("Authorization") String str, @Query("account") long j, @Body fvg fvg);

    @POST("/preferences")
    fty updatePreferences(@Header("Authorization") String str, @Query("account") long j, @Query("freshnessMillis") long j2, @Body fvi fvi);

Anyway it doesn't make much sense for me to keep going on this way as the data provided is pretty restricted.

I'll keep investigating on the web version.

[nelenkov]

Oh, this is interesting. What data are you missing?

In any case, the exact cookies don't matter very much, if you set your HTTP client to follow redirects, they will be set for you automatically. The hard part is getting the original master token.

If you are working on a server, you can just use the Chrome or GMS package signature to get the token. See here for some sample code (tested recently, so should work):

https://github.com/nelenkov/gdrive-appdata/blob/master/get-gdrive-appdata.py

The only difference from Android is that Android saves the master token for you in the accounts.db, and thus you don't need to send the password each time.

[devgianlu]

Oh, this is interesting. What data are you missing?

Well, the web version has much more details.

Ok. So If I've got the master token (suppose I'm on Android) I can retrieve the auth token with AccountManager#getAuthToken, but I'm having troubles with this as I don't understand what should be specified as authTokenType (2nd parameter). Also, when I have the auth token what should I do? Is there a specific API to authenticate web pages with an auth token?

Sorry but I'm into this thing from 2 weeks and I really want to understand. Thank you for your explanations.

[nelenkov]

Did you look at the code?

https://github.com/AndlyticsProject/andlytics/blob/master/src/com/github/andlyticsproject/console/v2/OauthAccountManagerAuthenticator.java

It's fairly straightforward:

You get the token, put it in an Authorization header, then you issue a GET, the returned redirect sets a bunch of cookies to your HttpClient. Within the same session, you open the console top page. Then scrape the data.

[devgianlu]

Unlucliky, after copying the code to a new project, I got, as many other users, the INVALID_SCOPE exception. Then I found this: https://github.com/tdryer/hangups/issues/260#issuecomment-246578670, I was able to retrieve an oauth_token which I tried using as oauthLoginToken in the code but I got "AuthenticationException: Cannot get uber token. Got: Error=badauth."

The workaround URL also set a SIDCC cookie, not sure if it's needed for the uber token request.

[devgianlu]

If I tryed to replace the obteined oauth_token directly with the uber token, I got a 302 and I'm redirected to the console, but if I try to request https://play.google.com/apps/publish/?dev_acc=00000000000000000000 (with the same HttpContext), the page wants to redirect me to: https://accounts.google.com/ServiceLogin?service=androiddeveloper&passive=1209600&continue=https://play.google.com/apps/publish/?dev_acc%3D00000000000000000000&followup=https://play.google.com/apps/publish/?dev_acc%3D00000000000000000000

(all the zeros are a dummy dev_acc obviously)

[nelenkov]

Interesting.

Did you replace the client_id parameter with your own client ID (registered in API console)?

I tried a few other things, but it seems even the browser-based login flows are being replaced with challenge/response mechanisms, that seem to use temporary encryption keys. So even if this works, I guess it will eventually be blocked and/or phased out.

If you get it working, I'll merge the workaround in Andlytics, if people still want it.

However, going forward, using the official dev console app protocol seems to be a much better idea. You just need to get a token for the oauth2:https://www.googleapis.com/auth/playdeveloperapp scope from AccountManager, and you can call the `/apps/publish/mobileapi/' APIs directly. You'd have to extract/reverse engineer the protobuff classes though.

[nelenkov]

It seems the protobuf data is essentially the same as JSON (same key/value mappings), so migrating to protobuf should be fairly straightforward. Also /mobilestats API seems to have all the data Andlytics shows, including comments, ratings, revenue, current version info, etc.

For example, a comment for Andlytics looks like this (parsed with Protobuf editor Burp extension):

 1 {
    1: "gp:AOqpTOE2hzJUHsZsA783rEZ3o1-VtLebMPgjgnIhCDAihTTo18YVILSq7a-NVDPn6gdK2rKUvlFREPTQUFaPBQ"
    2 {
      8: 0x6e616d6d
      14: 0x4d206c65
      12: 0x6e696863
    }
    3: 1477004792538
    4: 5
    5 {
      1: "fr_FR"
      2: ""
      3: "Pratique"
    }
    6: 262
    7: "2.7.4"
    9: 1
    10 {
      1: "en"
      2: ""
      3: "Convenient"
      4: "fr"
      5: "French"
    }
...

[nelenkov]

Reference:

• https://github.com/tdryer/hangups/issues/260#issuecomment-246578670
• https://developers.googleblog.com/2016/08/modernizing-oauth-interactions-in-native-apps.html
• https://github.com/tdryer/hangups/blob/82786f40567acf811cc83340162e7bfc2e1dfa65/hangups/auth.py
• https://gist.github.com/cdellin/6b9bae1310f2ee6b47ba3663bdcb1ded
• https://www.essakaram.com/how-google-instagram-and-linkedin-authenticate-users-in-their-native-mobile-apps/
• https://github.com/tdryer/hangups/commit/1f31a927fca550a7424439f52b3e6f70a5a228ba

[devgianlu]

You just need to get a token for the oauth2:https://www.googleapis.com/auth/playdeveloperapp scope from AccountManager

I'm not able to do that. No matter if I use GoogleAuthUtil helper class or AccountManager#getAuthToken I always get INVALID_SCOPE.

You'd have to extract/reverse engineer the protobuff classes though.

I decompiled the Play Console APK so I should find the structs somewhere in the 316064 lines of source code.

[devgianlu]

It seems I'm doing it right: https://developers.google.com/api-client-library/java/google-api-java-client/oauth2#android.

[nelenkov]

Getting INVALID_SCOPE also, it seems it's locked down to the the Google app package name and signing certificate. That's unfortunate.

Assuming you somehow got the token, you wouldn't need to reverse engineer the whole app, just parse the received Protobuf recursively, and then match the properties you need to the proper keys, like we did with Andlytics.

[devgianlu]

I also captured these requests. They seem pretty complicated.

The Auth value is used to authenticate all the other requests.

[devgianlu]

It seems it's locked down to the the Google app package name and signing certificate.

That's bad, we must get to work the web auth then.

[devgianlu]

This might be useful: https://bitbucket.org/EionRobb/purple-hangouts/src/default/hangouts_auth.c?fileviewer=file-view-default

[devgianlu]

I did it. I was able to retrieve data from https://play.google.com/apps/publish/androidapps. Unluckly a WebView is needed and the user may need to insert his credentails (into a Google page). Not sure how often the user is asked for login.

I'll clean up the code and I'll open a repo in the near feature.

[nelenkov]

Great. How do you get the token from the WebView? Custom scheme redirect?

[devgianlu]

I did it the dirty way: I request the Play Console page in the WebView so that Google does everything for me, then when the Play Console page loads I gather everything I need such as the XSRF token and some headers.

It looks like the CookieManager keeps track of all the cookies. So when I restart the app Google doesn't ask for login again and redirect me directly to the Play Console page.

EDIT: just to point out: I don't deal with any token with this method.

[nelenkov]

Yes, WebView is persisted per app, so you won't be required to re-login as long as the cookie is valid. Login could be a bit of a hassle if you are using 2FA, though.

[devgianlu]

I have 2 factor auth and it's not a problem until the user deselects "Don't ask again on this computer".

[devgianlu]

Here is the repo: https://github.com/devgianlu/Play-Console-Android-API. If you have any question, open an issue.

[nelenkov]

Great, thanks! It works :)