[www] Add support for user-provided benchmark source.

[ChrisCummins]

This adds a benchmark_source attribute to the step API that enables users to provide their own code to use as a benchmark. To use your own benchmark source, you inline the contents of your source file into the benchmark_source attribute and set the benchmark attribute to the name of the local file:

{
  "benchmark": "/tmp/foo.c",
  "benchmark_source": "int A() {\n  return 1;\n}",
   ...
}

The file name is important because we use the file extension to determine how to process it (e.g. .c files are compiled, .ll files are interpreted as bytecode, etc).

Here are two example files for testing.

The first is example1.c:

int A() {
  return 0;
}

The second is example2.ll:

; ModuleID = 'example1.c'
source_filename = "example1.c"
target datalayout = "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128"
target triple = "x86_64-unknown-linux-gnu"

; Function Attrs: noinline nounwind optnone uwtable
define dso_local i32 @A() #0 {
  ret i32 0
}

attributes #0 = { noinline nounwind optnone uwtable "correctly-rounded-divide-sqrt-fp-math"="false" "disable-tail-calls"="false" "frame-pointer"="all" "less-precise-fpmad"="false" "min-legal-vector-width"="0" "no-infs-fp-math"="false" "no-jump-tables"="false" "no-nans-fp-math"="false" "no-signed-zeros-fp-math"="false" "no-trapping-math"="false" "stack-protector-buffer-size"="8" "target-cpu"="x86-64" "target-features"="+cx8,+fxsr,+mmx,+sse,+sse2,+x87" "unsafe-fp-math"="false" "use-soft-float"="false" }

!llvm.module.flags = !{!0}
!llvm.ident = !{!1}

!0 = !{i32 1, !"wchar_size", i32 4}
!1 = !{!"clang version 10.0.0 "}

[sahirgomez1]

@ChrisCummins, based on the comment, we rely on the user to pass the content of the source file inlined, correct?

[ChrisCummins]

@ChrisCummins, based on the comment, we rely on the user to pass the content of the source file inlined, correct?

I'm imagining the UI for this would be an "Upload" button that would pull up a native local file selector. Selecting a file would then read the contents so that it can set the benchmark_source and benchmark attributes for future requests. Is that do-able?

Cheers, Chris

[sahirgomez1]

@ChrisCummins, based on the comment, we rely on the user to pass the content of the source file inlined, correct?

I'm imagining the UI for this would be an "Upload" button that would pull up a native local file selector. Selecting a file would then read the contents so that it can set the benchmark_source and benchmark attributes for future requests. Is that do-able?

Cheers, Chris

We'll definitely need a list of whitelisted files extensions to cross check users upload a readable file from the UI. I've used the FileReader api *see in the past for this. But never with .c or .ll. But I imagine this can be do-able, we can test.

[ChrisCummins]

We'll definitely need a list of whitelisted files extensions to cross check users upload a readable file from the UI. I've used the FileReader api *see in the past for this. But never with .c or .ll. But I imagine this can be do-able, we can test.

File reader api looks good! As long as you can read the file contents into a string, the backend can figure out how to handle the string by the file extension. I don't know if an allow-list in the frontend would help

Cheers, Chris