Create web version of the tool

[dendibakh]

I really like your tool, Travis! And it would be even more cool to have web version like https://gcc.godbolt.org/ or http://quick-bench.com/ The use case that I'm looking for is that a user writes a piece of assembly and selects perf counters he wants to measure.

[travisdowns]

It's a good idea, one that has crossed my mind before. I'll certainly keep it in mind (and help is welcome!).

[lemire]

@dendibakh The user would input assembly?

One thing that has me worried is the idea that random users can execute random code on a machine. That seems a tad dangerous. How do you guard against hacking?

(People will hack you to install a porn server on your machine. Happened to me.)

[travisdowns]

I think the user would probably usually input C or C++ code, although assembly would be easy to support as well. If I had to do this, I'd probably just fork godbolt and add the benchmarking stuff on top.

@lemire - well there are a lot of "run arbitrary code" services out there and I assume they use at least one of these two possible approaches:

• Sandbox the compilation and runtime using OS services like cgroups or some higher level solution that does the same thing, or possibly even a virtual machine (although that last option probably isn't desirable for benchmarking).
• Just throw away the server instance as soon as the job is done, or at least after a small time limit. Then even if the hackers hack their way out of the sandbox, the instance is shut down, so you are safe. This process is orchestrated from the outside so it can't be tampered with.

Personally I would do both. The first one dramatically raises the bar to do something naughty in the first place, since you'd have no network access, minimal file system access, etc. You'd be looking for a privilege escalation or cgroup hole or guest-to-hypervisor exploit, all of which are very serious and usually rapidly patched when discovered. Still, it's possible that some either gets out or finds something naughty to do even within the rules (e.g., mining some crypto-currency).

That's where the second approach comes in: limiting the damage of anything that 1 doesn't stop.

This all assumes you are running on some cloud service that makes all of this easy. The bad part about that for benchmarks is that there are either no performance counters available, or only a few (some Amazon EC2 instances allow some performance counters, for example).

So I don't think the security is an unsolvable problem at all (indeed many other people have solved it) - and if I'm wrong, I guess I get some free porn out of it, right?

The bigger problem is my lack of front-end skills.