the-benchmarker / web-frameworks

Which is the fastest web framework?
MIT License
6.91k stars 641 forks source link

refactor(dart): Runtime using AOT #7147

Closed medz closed 4 months ago

medz commented 4 months ago

I find the previous way of building Dockerfiles unfair compared to other frameworks.

[!TIP] The final executable program of Dart server should be the AOT compiled binary file.

The previous Dockerfile used the Dart VM to run the web server, which was usually only used during development in Dart development. Formal deployment is the result of using AOT compilation.

Since VM will affect some performance, the test results of Dart-related frameworks are inaccurate.

medz commented 4 months ago

@waghanza Hi, I have refactored it to ensure that the Dart server-side app compiles through AOT and then runs the server.

The previous Dart Dockerfile used Dart VM. Normally, Dart developers only use Dart VM during the development stage, and formal deployment uses AOT-compiled binaries.

medz commented 4 months ago

Is it useful then to compare AOT vs VM ?

It is useful to compare AOT and VM performance, but I personally think it is unnecessary. In the case of formal deployment in Dart, few people will use VM to run Dart apps. VMs cause a lot of unnecessary performance penalties.

In fact, when I submitted this PR, I wanted to let Dart have both AOT data and VM data, but I didn't know how to implement the dockerfile. Because there is no document found in the config.yaml file to configure the dockerfile for the engine.

medz commented 4 months ago

I personally believe that performance testing, in a fair way, should show the best way to build the language in which the framework is built.

Dart is one of the few cross-platform languages. AOT compilation is used here just like no one would use cargo run to run a Rust app, but it is usually recommended to use VM during development.

medz commented 4 months ago

Off topic: If the Dart server-side application uses dart run to run, then I think the Rust app should also use cargo run to run instead of cargo build to build the AOT binary. It is only fair that other language frameworks with both JIT and AOT runtimes should implement this. (Although Dart AOT helps improve Dart app performance, there are still many gaps compared with other high-performance language frameworks.)

medz commented 4 months ago

@waghanza

https://github.com/the-benchmarker/web-frameworks/blob/fc88364f0818b279ac2806fde97cdb772f51cdef/config.yaml#L38

I found it but I don't understand it, I don't know why this command requires the container to have shell software.

medz commented 4 months ago

If you simply run docker run -p 3000:3000 -td <name>, there will be no problem. It seems that the > symbol is used to output the content to the file, thus causing the /bin/sh command in the image to be called. But the minimal image constructed through AOT compilation usually does not provide it.

waghanza commented 4 months ago

@waghanza

https://github.com/the-benchmarker/web-frameworks/blob/fc88364f0818b279ac2806fde97cdb772f51cdef/config.yaml#L38

I found it but I don't understand it, I don't know why this command requires the container to have shell software.

This command only run the container, using its default CMD or ENTRYPOINT

medz commented 4 months ago

Now we have two options:

waghanza commented 4 months ago

If you simply run docker run -p 3000:3000 -td <name>, there will be no problem. It seems that the > symbol is used to output the content to the file, thus causing the /bin/sh command in the image to be called. But the minimal image constructed through AOT compilation usually does not provide it.

No it is just to capture output of docker command. Docker run output the container id, which is used to know the IP address

medz commented 4 months ago

OK, I commit and temporarily log out of the minimal container build.

I haven’t deeply understood the principle of docker run, and I don’t know why using > to output to a file requires image to provide sh

waghanza commented 4 months ago

I haven’t deeply understood the principle of docker run, and I don’t know why using > to output to a file requires image to provide sh

medz commented 4 months ago

Well, I gave up on building the minimal image in this test because I couldn't solve the problem. Of course, it goes beyond PR.

I noticed that other AOT apps do not use the minimum image

medz commented 4 months ago

Of course you asked me to go ahead and implement the minimal image, but I can't solve the above problem because I don't know why I get this error. And you can try to build minimal image and run it, it is successful. But in CI, he requested /bin/sh, which caused the failure.

waghanza commented 4 months ago

Well, I gave up on building the minimal image in this test because I couldn't solve the problem. Of course, it goes beyond PR.

yes, it is out of the original idea of this PR, but I'm sure it is for the best (minimal image could be a valuable argument when facing production usage)

medz commented 4 months ago
image

I'm very puzzled. It still succeeds without a shell, but I don't know why it fails in CI. I tried docker run > file locally and it still failed. Maybe it's a problem with docker itself?

waghanza commented 4 months ago

I look into this

medz commented 4 months ago

An issue should be launched for the minimum image construction plan. Because I think it is an excellent choice for other aot apps to implement the minimum image as well.