dart2js: improve deobfuscation of minified stack traces

[sigmundch]

Upcoming tasks to improve deobfuscation:

• [x] add a dart2js source-map format extension x_org_dartlang_dart2js
• [x] add minified global names and minified instance names to the source-map format
• [x] add inline frame data to the source-map format
• [ ] change sourceName information on local variables and fields to point to record the unminified name (and not current method name)
• [x] ensure function tokens do hold the enclosing function name.
• [x] add unit tests for:
  • [x] deobfuscating all known exceptions produced by the SDK that contain symbol names
  • [x] deobfuscating errors mentioning any possible entity we can think of (class, method, fields, constructor-bodies, async-bodies, etc.)
  • [x] deobfuscating inlined frames: across files (ensure uri is decoded correctly), and with complex control flow (ensure recording of inlining data is not sensitive to the code structure).
• [ ] expose an open-source dart package for deobfuscating stacktraces
  • [x] add package with deobfuscation logic [done: see pkg/dart2js_tools]
  • [x] add templates for deobfuscating all known exceptions produced by the SDK that contain symbol names.
  • [ ] ensure those template regexps are tested in the SDK repo, so we detect regressions as early as possible
  • [ ] create coverage tests: ensure that no piece of JS is missing deobfuscation data, even synthetic code generated from the compiler
  • [x] add special tests for common synthetic code: interceptors, one-shot interceptors, stubs for default parameters and instantiations, async-body and constructor-body
  • [ ] consider using the "sourcesContent" section of the source-map format to include deobfuscation details for synthetic code
• [x] create integration tests: tests that run a program in several browsers, send exceptions to a server, and validate that servers can recover the original stack trace.

[vsmenon]

Dropping to p3 to reflect no active work in the next few months. We should revisit later this year.

[sigmundch]

Actually most of this work is now complete. The main missing item is to improve documentation for customers interested in the new features. Internally this is integrated with our build systems. External users need to use some of the deobfuscation tools we provide to take advantage of the inlining frame data.

[sigmundch]

@ekweible - I heard that you have interest in using some of the deobfuscation tools on your project. Can you also CC here others on your team who I should be in contact with?

@vsmenon - ditto, can you CC others you know that are interested in this?

[sigmundch]

There is very little documentation at the moment, but if there is interest we can put something together and clean up some of the tools to make them available more easily.

We have a set of tools under https://github.com/dart-lang/sdk/tree/master/pkg/dart2js_tools/ that help with debugging production stack traces. Given the source-map file we generate in dart2js, you can take an exception collected at runtime, and run it to the deobfuscate.dart tool. The tool should be able to convert minified names to the original names in error messages and should be able to produce a Dart stack trace that unwinds any inilning decision the compiler made.

[evanweible-wf]

Thanks @sigmundch! @robbecker-wf is also a good point of contact for my team. We're not sure who is going to dig into this yet, but we'll follow up sometime next week.

I imagine we'll take a stab at what you've linked us and see how far we get, but we're definitely interested in solidifying our story around this so any additional documentation in this area would be greatly appreciated!

[patkujawa-wf]

Maybe @yuanmwang-wf is interested as well. I believe he worked on a sourcemapper internal tool for us.