niklas-heer / speed-comparison

A repo which compares the speed of different programming languages.
https://niklas-heer.github.io/speed-comparison
MIT License
475 stars 76 forks source link

Refactor to use earthly and more isolated environments #18

Closed niklas-heer closed 1 year ago

niklas-heer commented 1 year ago

Having one big Docker image for testing all languages is not ideal. Thus, we can use earthly to refactor running the speed tests for each language. Allowing for using the optimal Docker image for a language.

This is made possible by a new custom written Go application scbench which gets added to each test run for every language which outputs a JSON file with the results. Here is an example:

{
    "Started": "01-10-2022 03:09:55",
    "Ended": "01-10-2022 03:09:56",
    "Language": "Crystal",
    "Version": "1.4.1",
    "Command": "./leibniz",
    "Accuracy": 0.6666666666666666,
    "CalculatedPi": "3.1415916535917745",
    "Iterations": 10,
    "Average": "45.557034ms"
}

This also adds other improvements to the structure and README.md.

niklas-heer commented 1 year ago

earthly makes this pattern possible with the way you can save artefacts locally and also reuse them in other containers. Here is an example:

build:
  FROM golang:1.19.1-alpine
  WORKDIR /go-build
  COPY --dir scbench ./
  WORKDIR /go-build/scbench
  RUN go build -v -o build/scbench ./cmd/scbench
  SAVE ARTIFACT build/scbench /scbench

alpine:
  FROM alpine:3.16

  COPY ./src/rounds.txt ./
  COPY +build/scbench ./

crystal:
  FROM +alpine
  RUN apk add --no-cache crystal

  COPY ./src/leibniz.cr ./
  RUN --no-cache crystal build leibniz.cr
  RUN --no-cache ./scbench "./leibniz" -i $iterations -l "crystal --version" --export json --lang "Crystal"
  SAVE ARTIFACT ./scbench-summary.json AS LOCAL ./results/crystal.json