Rewrite in rust under the AGPL

[euank]

This rewrites the whole thing in rust instead.

The primary motivations are:

1. Improve user freedom by ensuring a large percent of the code is available under the AGPL only, not the AGPL or Public Domain (at your choice, as currently applies to the javascript and css files).
2. Use fewer resources at runtime. The php version took significantly more memory, and I'm always happy to save me some compute power.
3. I just plain don't like php

Still TODO on this:

• [x] Integ tests
• [x] Travis CI
• [x] Dockerfile / image to deploy

[euank]

Benchmark results:

Both were running locally in their own docker containers. Stats were collected via checking their respective cgroup's statistics.

Load was generated using wrk, with relevant commands shown.

1000 conn/10s test:

$ wrk -c 1000 -d 10s http://localhost:8083 # old
  2 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.37s     2.55s    9.20s    91.45%
    Req/Sec     3.67k     1.48k    9.33k    69.43%
  68845 requests in 10.00s, 76.01MB read
  Socket errors: connect 0, read 0, write 0, timeout 2504
Requests/sec:   6882.87
Transfer/sec:      7.60MB

Memory usage peak: 440M
CPU peak: 260%

$ wrk -c 1000 -d 10s http://localhost:8084 # new
Running 10s test @ http://localhost:8084
  2 threads and 1000 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     6.96ms   10.05ms  23.96ms   70.32%
    Req/Sec    33.81k    12.54k   71.78k    64.72%
  620817 requests in 10.00s, 691.87MB read
  Socket errors: connect 0, read 0, write 0, timeout 2972
Requests/sec:  62074.90
Transfer/sec:     69.18MB

Memory usage peak: 12M
CPU peak: 170%

High bandwidth, image downloads:

$ wrk -d 10s http://localhost:8083/img/img_4330.jpg # old
Running 10s test @ http://localhost:8083/img/img_4330.jpg
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.83ms    5.56ms  46.97ms   79.53%
    Req/Sec   496.46     92.58   814.00     71.01%
  9746 requests in 10.00s, 15.55GB read
Requests/sec:    974.20
Transfer/sec:      1.55GB

Memory usage peak: 41M
CPU peak: 120%

$ wrk -d 10s http://localhost:8084/img/img_4330.jpg 
Running 10s test @ http://localhost:8084/img/img_4330.jpg
  2 threads and 10 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     9.33ms    7.47ms  54.02ms   92.13%
    Req/Sec   502.23    106.44   807.00     71.99%
  9872 requests in 10.00s, 15.75GB read
Requests/sec:    986.90
Transfer/sec:      1.57GB

Memory usage peak: 14M
CPU peak: 150%

As we can see, the rust version manages to handle roughly 10 times as many requests for the index, while using a quarter of the memory and roughly 1 core less of cpu.

For static files (the image test), both perform quite similarly.

I think this is enough data to claim that the rust version is a clear improvement and to switch over to it.

[Mechazawa]

Just for maintainability. It’s easier to keep track of what’s happening that way.

Disk IO should not be an an issue since there is a disk cache in memory. As long as the site has been served recently it shouldn’t make a round trip to the disk. https://linuxatemyram.com

On 31 Aug 2018, at 19:08, Euan Kemp notifications@github.com wrote:

@euank commented on this pull request.

In src/main.rs:

+https://github.com/euank/rms.sexy + +It is also available at "/code.tar.gz" on this server.

• -->
• {}
• 
• 
• +"#, There's not really a need to since it's so short.

Even if I stored it as a file, I'd want to include! it into the built binary to be able to just serve it from memory without any disk IO. I don't think it matters much either way. Is there a reason you think it's better stored as a file?

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[euank]

Don't worry, I know what the page cache and shared memory are... Though since I run it on my Kubernetes cluster, there are cases when other pods might end up dominating the page cache.

Storing it as a constant blob vs as a static file is still quicker either way though since there'll be fewer total syscalls each time it's served.

Storing it as a file in the repo, but compiling it into the binary with the include_str! macro seems like the best of both worlds, so I'll do that in a followup.

[Mechazawa]

Sounds like a great idea!

On 1 Sep 2018, at 00:02, Euan Kemp notifications@github.com wrote:

Don't worry, I know what the page cache and shared memory are... Though since I run it on my Kubernetes cluster, there are cases when other pods might end up dominating the page cache.

Storing it as a constant blob vs as a static file is still quicker either way though since there'll be fewer total syscalls each time it's served.

Storing it as a file in the repo, but compiling it into the binary with the include! macro seems like the best of both worlds, so I'll do that in a followup.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.