Rewrite isolate.py / isolateserver.py in a performant language (C++ or Go)

[GoogleCodeExporter]

No wonder, isolateserver.py is slow. Rewriting it in golang or C++ would be a 
relatively low hanging fruit.

Advantages:
C++
- Execution is faster
- Smaller binaries
- Lots of chromium tools are written in C++ (ninja, GN).
- Larger acceptance.

Go
- Quicker to prototype a MVP
- Faster coding general and safer language, easier to refactor.
- Easier to test and mock interfaces.
- Faster to build.
- Cross compiling is simpler. Cross compiling to Windows from linux is already 
supported (unlike C++).

Issues:
- Two implementations means trickier to update the API in an agile way; need to 
create a v1 stable set of APIs. See issue 178 for a formal API definition. The 
API didn't change significantly in multiple months and is under heavy use. The 
only likely new API to be improved is /pre-upload on issue 58 to help with 
deterministic build: when a .isolated is a cache hit, all the included objects 
should be tagged by the server implicitly, saving a lot of unneeded HTTP 
request to offload work server side in an asynchronous way.

Original issue reported on code.google.com by maruel@chromium.org on 27 Nov 2014 at 3:06

• Blocking: #58
• Blocked on: #178

[GoogleCodeExporter]

See issue 181 for run_isolated.py.

Original comment by maruel@chromium.org on 27 Nov 2014 at 3:14

• Added labels: Type-Feature
• Removed labels: Type-Bug

[GoogleCodeExporter]

+1 for go (if my say matters). Chrome infra is ramping up on go quickly, and it 
has a lot less pitfalls than C++ (sometimes frustratingly so, but I digress). I 
don't think the execution differences between C++ and go will be noticeable. 
We'll mostly be doing async-io stuff and string computation.

Original comment by iannu...@google.com on 18 Dec 2014 at 9:40

[GoogleCodeExporter]

s/has a lot less pitfalls/is much simpler/

Original comment by iannu...@google.com on 18 Dec 2014 at 9:40

[GoogleCodeExporter]

Andrii accepted to look at it.

Original comment by maruel@chromium.org on 8 Jan 2015 at 4:04

• Changed state: Accepted

[GoogleCodeExporter]

Original comment by maruel@chromium.org on 8 Jan 2015 at 4:05

• Now blocking: #58

[GoogleCodeExporter]

So, I started with Go. I've verified that cross compilation really works, but 
testing it on Mac, Win64 and 32, including XP. 

However, some Go standard libraries use cGo to interface with OS. By default, 
cross compilation doesn't make use of these libs, but it does cause problems to 
other projects. This project https://github.com/inconshreveable/gonative fixes 
that by downloading official binary distro and linking against it, thus making 
use of cGo standard libraries for destination platform. 
https://github.com/mitchellh/gox is a great companion:
  $ gox build # produces binaries for all platforms/archs it can in parallel

For speeding some go code, C using cgo http://golang.org/cmd/cgo/ is 
possibility.

However, compiling it on windows is not trivial: I still couldn't do it on 
32-bit XP bot, but it worked on Win7 64. The problem is due to missing/wrong 
env for Go and MinGW gcc. Once the bot is set up correctly, building is 
trivial. I requested dedicated WinXP bot for this purpose 
http://crbug.com/452028 .

Original comment by tandrii@chromium.org on 26 Jan 2015 at 5:46

• Changed state: Started

[GoogleCodeExporter]

Hey tandrii, I carved out some time to look at this, is it possible to share 
any WIP code? I'm primarily interested in the client side. Else I can just 
start on my own and see how far I get.

Original comment by andrew.w...@cloudera.com on 10 Feb 2015 at 2:48