protobufs versus WebGL array buffers

[wolfgangvonludwigsburg]

in step2/index.html: change props: lat is lon, and lon is lat (vice versa).

Instead of trying to decode the protobufs (or from where to get the .proto files ?) one could track the WebGL array buffers (arrays of [x,y,z]) before these get processed by the vertex shaders, e.g. to construct the geometries ...

These could be accessed by injecting approbiate proxys (catching XHRs ...), and could be stored thru dual-context techs (NW.js, electron, ...)

I tried to talk to your eMail at https://www.wafflesystems.com/, but my eMail was flashed back ... ?

I prepare to do a better mesh for my little hometown, can we bundle our knowledge ... ?

BTW, you did a fantastic work, so far ...

[kaylathedev]

Thank you for leaving a comment! I fixed the latitude/longitude error.

I saw that there were files in the form of protobufs that were requested by the script, so I attempted to decode them but I haven't gotten anywhere with that. Most of it was a bunch of binary data. A few of the protobufs had images embedded in them that looked like map tiles. Example: http://imgur.com/a/BWVzI

Sure, we can combine our knowledge! And, I'm not familiar with WebGL. Do you have any good tutorials to get me started?

[wolfgangvonludwigsburg]

Kayla, I'm very glad you immediately noticed this issue.

(1) Concerning WebGL: Best Online: http://webglfundamentals.org/webgl/lessons/webgl-fundamentals.html Best Book : Webgl Programming Guide Interactive 3D Graphics Programming with Webgl, Addison Wesley, 2013 Hypothesis : GMAPS uses 'Compressed Textures': e.g. http://blog.tojicode.com/2011/12/compressed-textures-in-webgl.html

(2) Where are the shaders?

The "WebGL-module" is one of the four "t=zcms"-Webworker instances.

Take a "space view" scenario look (e.g. https://www.google.de/maps/@0,0,22994720m/data=!3m1!1e3?hl=de) Chrome Debugger > Network > find entry 'PlanetoidMetadata' > Click on Initiator:

code at initiatoring line 35.493 should be: a.send() //=> emitting a XHR request)

In this module:

search for 'main' you should get about 60 matches

"de-stringifying" each of these matches, you get about the 'vertex' resp. 'fragment shader' codes

Hypothesis: GMAPS uses protobufs that implement >dynamically< compressed structures (no fixed ".proto" schemes), and if I a were a Google developer, I would use shader coding for the dynamic decompressing algorithm ... (... but that's only my guess so far)

(3) What could be a next step ?

Come to an (Offline-) Application, that includes all Webworker instances, especially that "WebGL-module" ...

How are your opinions ... ?

[kaylathedev]

I am working towards making this into a standalone application. The only requests that would be made to the servers would be for the textures and the 3d models.

[wolfgangvonludwigsburg]

Maybe before you should concern about the right starting point ...

If you exec. with node.js

var request = require('request')
var fs      = require('fs')

var url     = 'https://www.google.de/maps/@0,0,22994720m/data=!3m1!1e3?hl=de'
var file    = '2016-07-24_gmaps.html'

request(url, function (error, response, body) {
    if (!error && response.statusCode == 200) {
        console.log("REQ: " + url)
        fs.writeFile(file, body, function(err) {
            if(err) {
                return console.log(err);
            }

            console.log("FILE: " + file + " was saved!");
        });
    }
})

This downloaded "startup-HTML" 2016-07-24_gmaps.html differs from the "index.html", that you see by regularly inspecting the GMAPS source code.

Then loading this "startup-HTML", chrome debugger > Event Listener Breakpoints > Script > Script First Statement ... you'll get the whole (about 20) js-module parts, that bootstrap towards the whole GMAPS application.

One enhanced target could be to end up in a totally Offline-App., within a >predefined< scenario (i.e. a certain location), for sure. (... and the end target is, to load all into Blender3D ... ;-))

[wolfgangvonludwigsburg]

The GMAPS application resources are retrieved at least from 4 servers (i.e. 4 different server addresses) Server (1) => "script" server => provides document("index.html"), scripts while bootstrapping, and XHR scripts at runtime Server (2), Server (3) => some other resources (roadmap?), static assets Server (4) => "data(base)" server => provides the protobuf encoded vector tiles/textures e.g. "pb=!1m2!1s03!2u561!2e1!3u695!4b0"

This protobuf addressing scheme (separated by '!'s) denotes self-typed parameters, where parameter 2 seems to be the variant (e.g. above "1s03" => 03) A tiling schema (for the "tile pyramid") usually denotes as {zoom}/{x}/{y}.png (=> easy mapping to filesystem) I work on recognize, how GMAPS does this sort of mapping ...

Unfortunately I am no PHP expert. Would there be an easy way to get your regex/transpiling coding into JavaScript ? (... maybe based on NW.js, electron)

[kaylathedev]

I am currently working on a Node.js script that is similar to the PHP regex mess that I currently have. It's at step2-a/compile-scripts.js and it uses the esprima module so it can read the Google Maps scripts.

[wolfgangvonludwigsburg]

I am very glad, I do not have to install PHP on my machine ;-) (... since in the meantime, GMAPS modules have changed a bit)

If we are/were on JavaScript/Node.js platform, we could even use further tools, among JavaScript editing, even AST-Analysis (Haverbeke), ...

Links, I had previously found: https://github.com/jquery/esprima https://github.com/beautify-web/js-beautify

Marijn Haverbeke https://github.com/codemirror/CodeMirror https://github.com/ternjs

I found another cool online resource, that might be helpfull in studying WebGL and shaders (see chapter 6, 7): http://math.hws.edu/graphicsbook/source/

[wolfgangvonludwigsburg]

(1) According to https://developers.google.com/closure/library/?csw=1 > Who uses Closure Library? GMaps uses Closure as a base lib (https://github.com/google/closure-library)

However, at first overlook, I could'nt find much of 'goog.' uses in GMaps modules ...

• except module 'closure_ui.js' as part of the "bootstrap modules".

Surely, GMaps developers use further Closure Tools (e.g. closure-compiler, closure-templates, ...), (see reps on https://github.com/google/)

(2) We should come to an identifying/naming concept:

• please always include a download-time and module-link-address comment in a header comment
• how do we name these different XHR-worker-scripts (e.g. 'scriptloader', 'webgl-shader', ... ?)
• maybe we can build up a (separate?) dictionary/name/reference table of all used functions, ... (mhmm, should'nt that already be available as part of esprima/ternjs ?)

(3) When it comes to understanding the GMaps-WebGL-calls, we should substitute numerical constants to their according programatic mnemonics (see closure-library > webgl/webgl.js, or Khronos WebGL API specifications)

(5) Further we should define some test scenarios/locations, so we can talk about the same issues.

Kayla, what are your opinions ... ?

[kaylathedev]

I agree with coming up with an identification concept. So, I'll have to re-download the page, as I don't have the last download time.

I agree with 3. If you're correct about GMaps using open source libraries, then it might be possible to find them inside of the obfuscated JS files. If we can find them, then we can isolate the library from everything else, and also give it a name.

[wolfgangvonludwigsburg]

I ordered a book about Google Closure Library, since I think Closure offers (for me) very helpful services (Google I/O 2011: JavaScript Programming in the Large with Closure Tools, https://www.youtube.com/watch?v=M3uWx-fhjUc).

Before I dive into my own coding, I look what's on the market ...

• Concerning 'DEOBFUSCATION', http://www.jsnice.org/ looks quite promising.

It depends, what the primary target should be. Do we really want to understand the code, or do we just want to get the 3D models, geometry and textures ...

• to help Google, to improve their datasets - for sure!