Protobuf structure for MapCells has changed

[JSchwerberg]

An update today caused the Protobuf structure for Mapcells to change -- looks like it is now in a tuple (?)

Proto files need update to reflect.

PLEASE KEEP THIS ISSUE THREAD ON TOPIC ie donations/thankyous/unrelated questions/flaming etc.

Thankyou

[trisk]

I tried just modifying one of the cellids (two bytes changed instead of one byte by accident) in GetMapObjects only without touching any other requests and I get the same empty map_cells response, so it's presumably the whole request being checksummed.

[alex89200]

Seems that they also turned on the SSL certificate validation (but not sure). Charles can't sniff pogo calls anymore :(

@trisk which tool do you use to sniff and replace packets?

[0xAcid]

@alex89200 please refer to google. It has been a while since they integrated certificate pinning and there are already ways to bypass it. Thanks to ask for this somewhere else.

[iGio90]

@alex89200 im sniffing good with charles

[trisk]

@alex89200 use the Xposed module or Cydia package to disable cert pinning in the app, see https://github.com/rastapasta/pokemon-go-mitm-node

I am using a wrapper for node-http-mitm-proxy (basically ripped out the protobuf parsing code from pokemon-go-mitm-node).

[broach]

@d-pollard Player stats works as well as an individual call without filling in unknown6.

Since unknown6 is on the RequestEnvelope it's still possible that it's signing, but it may only be using fields from certain request_type in the requests array (e.g. GET_MAP_OBJECTS) when doing so (and of course possibly only certain fields in those types :/ )

[trisk]

@alex89200 https://github.com/trisk/pokemon-go-mitm-node/tree/dump is the current script (run dump.coffee which uses lib/mitm-dump.coffee). It expects a dumps directory to exist when you run it to write out the raw files. lib/mitm-dump.coffee currently has the bit that alters GetMapObjects which you will want to remove if you want non-empty cells.

[solderzzc]

Where can I download the native part which would handle the encryption?

[socrates2510]

@broach It can also be that the unknown6 is ignored for not intensive calls like inventory, but enforced for the big ones like map objects. Client time making unkown6 is not important but server time validating it may more than they want to waste.

[srg2k5]

just so I'm tracking properly, unknown6 is actually generated by the client and from what we can tell, it's never handed from the server in a previous request?

[broach]

@srg2k5 Correct. The protobufs haven't changed. What's being sent in the protobufs from the game client, specifically the contents of unknown6 on the RequestEnvelope, has. Prior to now requests via the various APIs weren't sending anything in unknown6 and it was working (obviously).

(It's possible the real game client was sending unknown6 all along and the server side just wasn't enforcing it; I can't find any old dumps people had done to check that. )

[sanktanglia]

I can confirm the client was sending unknown6 previously, when I was initially working on a bot, I initially tried to send a hardcoded value I got from a log of a valid request and it wouldnt return any results

[Nostrademous]

anyone trying using AES256 encryption of packet contents using unknown6.unknown1 value as block_size or passphrase (with standard block_size)?

[srg2k5]

You need the key though for the signature, which would most likely be stored as a constant in the app, but as someone said previously with all the compiling and transpiling it would probably be hard to hunt down. I don't have a clue how to look at memory management on mobile, but on a PC i could easily rip out of memory.

[DrBlackross]

hmm trashcraft said "030a3476-668a-47fb-95ed-2bcfc5c15637" is a GUID

translate the guid

Standard 030A3476-668A-47FB-95ED-2BCFC5C15637 Bracketed {030A3476-668A-47FB-95ED-2BCFC5C15637} Oracle RAW(16) format 76340A038A66FB4795ED2BCFC5C15637 <---? Oracle hextoraw HEXTORAW('76340A038A66FB4795ED2BCFC5C15637') SQLServer using Oracle byte order 0x76340A038A66FB4795ED2BCFC5C15637

hmmm seed after slash? 1: "030a3476-668a-47fb-95ed-2bcfc5c15637/1467338129695000" just throwing it out there

[chenditc]

Maybe we can use things like blue stack to run the app on PC, then use inspect the thread it has.

There was a heartbeat message that generated every 5 seconds which contains get_map_object call. By looking at which thread is wake up every 5 seconds, we can probably identify the encryption method behind the screen.

I never did this before. Just some thoughts.

[dmadisetti]

Anyone want to grab us a memory dump? https://github.com/504ensicslabs/lime

[mitman7]

Heap dump: com.nianticlabs.pokemongo.hprof.zip

[trisk]

Updated example dump from a normal login, with more detail of the envelope fields in the log file: https://github.com/trisk/pkre-dumps/tree/master/dumps-example-0.31.0

[andibuch]

Has anyone tried grabbing a request, modifying it (and having it fail), and then resending it in its original form and seeing whether that last one returns empty? It would help us to find out as to whether, for instance, the signature depends only on the contents of the request by telling us if a request will remain valid at any time under any conditions so long as the contents did not change

[dmadisetti]

@andibuch According to Reddit https://www.reddit.com/r/pokemongodev/comments/4w1cvr/pokemongo_current_api_status/d635ppi

[andibuch]

Also does the server send anything pre-request that could possibly be the key for the signature?

Also-also, has anyone tried taking a request sent from one client in one login on one device, and resending it on a different client / different login / different device?

Useful methodology to diagnose where the signature is coming from / what it's dependant on

Also-also-also, I'd the unknown6 field the only field we don't understand yet? If not, why are we sure that it's the signature?

[jdleo]

@andibuch good point , trying that now. going to try in two different bluestacks instances (different UIDs)

[iGio90]

Summary of what we know... - Unknown6 has always existed before the current version(v0.31.0) - Unknown6 validation is highly likely switched ‘on’ server-side along with new client version release - sub_AF804 is just CRC checksum (zlib checksum) - libpgpplugin.so has nothing to do with PGP, it means Pokemon Go Plus, please don’t bother with it - Diff-ing different client versions does NOT produce meaningful results. Nothing to compare. - Modified request with same Unknown6 does not produce a valid response - Replay requests work within a limited timeframe (< 1 minute) - Unknown6 has variable length up to 256 bytes - Only CRC32_m_tab is found as a crypto constant in libNianticLabsPlugin.so, and believed to be related to trivial functions (inflate) only - We are not sure whether Unknown6 is a hash or a signature or a struct.

@jdleo it wont work

[mitman7]

In case anyone is interested, someone in discord chat is keeping a document: https://docs.google.com/document/d/1gVySwQySdwpT96GzFT9Tq0icDiLuyW1WcOcEjVfsUu4/preview

[TheSZ11]

@Kirex74 The location can def be changed there, just click the options funnel on the top left and type it in/toggle follow.

[Trolldemorted]

@iGio90 did you have any success finding out where the requestenvelope is constructed?

[TwinFuture]

Or drag the map around and zoom in.. Its pretty fast actually!

[Mantriur]

If this is about signing the data, it's likely some sort of hash. Does anyone know how big unknown6 is? Does it fit a common hash size? Would the C library potentially use easy to identify calls to OS/crypto libraries when generating the hash? Maybe it's something to look for in a decompile.

[iGio90]

yep! the string is built in sub_AA72C. actually we are trying to live debug with breakpoints

[lepschay]

@Mantriur Unknown6 has variable length up to 256 bytes (NOT always 256 bytes!!!)

from: https://docs.google.com/document/d/1gVySwQySdwpT96GzFT9Tq0icDiLuyW1WcOcEjVfsUu4/preview

[ghost]

@Mantriur Max size is 256 bytes if I remember correctly, but not always 256, they've seen some smaller ones as well

[IamCarbonMan]

I think it might be useful to look into ClientBlob, a similar value in Ingress which to my knowledge has not yet been cracked.

[keyphact]

Not sure if this helps, but for those that are wanting to debug with us (/r/pokemongodev), we have a moderated channel for developers/engineers that are actively debugging only, on Discord: https://discord.gg/kcx5f You'll have to get one of the admins to ask for access to the channel upon joining but there is reduced clutter and noise.

Sorry we've had to close it down to devs only, there is still general discussions going on here: https://discord.gg/dDBAp

[Mila432]

welp.. http://imgur.com/FuuRZzG i will give you a hint, it is the infamous sixth field too bad i am not allowed to play with pokemon anymore :(

[Trolldemorted]

@iGio90 thanks! after work i can try to acquire a copy of IDA and have a look myself, but i have never worked with arm asm so far, i do not know whether i will be a great help. but hey, it can't be harder than x86 :)

[Mila432]

all info you need is inside the bin "pokemongo" on ios / android i think it was the biggest bin inside the lib folder

[joshie]

FWIW, setting my system time right before checking man in the middle, the first 32 bytes of unknown6 seem fully dependent on the time and nothing else. Slightly offsetting the clock and restarting the app lets me move identical headers around between requests

[brantje]

@Mila432 Don't know where you live, but hosting a own Gitlab isn't a solution?

[BadLamb]

I just started reading the CFAA and @Mila432 isn't violating it because it applies only in the to "protected " computers and we are not violating those server's security, also the ToS aren't legally binding. https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act

[IamCarbonMan]

@CricketGUGC,the C&D tells him not to even access their servers, which he's clearly doing in that screenshot.

@ioio101, telling Mila432 he can't access the servers is obviously a scare tactic (how would they know it was him?) But the C&D may still be valid.

[rubenvereecken]

More PRs less talk :smile:

[TheFalseKing]

Scare tactics 101. I'd release the data anonymously. They have no proof you released it and not to mention anyone with knowledge could come up with that fix.

[IamCarbonMan]

@KingArsalan14, this is a Github discussion which is intended to make progress in solving the above issue. If you're hear to praise us and so on, we all appreciate it, but this isn't the place.

[Viper896]

Guys, based on the an earlier item i saw about about it changing w/ time. and the fact his script is called dirty.py .... I think Unknown6 is this: http://docs.unity3d.com/Manual/UNetStateSync.html

[rastapasta]

@trisk it's more that its not a MITM of a client - its the MITM of his bot afaik

[youtubesteak]

Any way it could be an api key or something similar provided by a 3rd service, such as google? Seems Niantic rolled out an update for iOS the API stopped working. So, if that's at all connected, it's possible they were required to have elevated permissions in iOS to access this API, so didn't validate at all? Could be way off here, but timing was definitely in-line What I'm referring to: Update: Ari Rubinstein from Slack has dived deeper and performed an analysis of what can be done with tokens obtained using this authentication flow. It appears additional steps must be performed to obtain a full access token. These steps are possible for Pokemon GO, as it is granted access to the necessary APIs, though the app does not appear to be using them. This issue may be attributed to programming mistakes and a permissive API on the part of Google. Both Google and Niantic are looking into the issue and attempting to provide a fix as soon as possible.

Update 2: Niantic has released an official statement regarding the Pokemon Go iOS permissions problem. And also, the app has been updated in the App Store with the correct permission scopes.

[IamCarbonMan]

@Viper896, that actually would make a lot of sense. Can anybody find use of that in a decompiled apk, from any point?

[CodeCompiler]

@Mila432 contact me ill put attorneys together for you.

[IamCarbonMan]

[removed by moderator - don't force someone to do something he has a C&D on]