jklincn commented 9 months ago

138

Use multi-stage building, significantly reducing the image size. (3.56GB -> 2.06GB)
Use mirrors.tuna.tsinghua.edu.cn to accelerate the fetching of software packages.
Simplify the installation process of Rust.
Fixed an issue where the experimental environment could not be correctly pre-set due to differences between the 'nightly' and 'nightly-2023-10-09' versions. Now it can dynamically adapt to the current Rust version based on rust-toolchain.toml.
Add a test stage for image building.
Clean up some invalid or irrelevant documentation links.

The image built by the new Dockerfile has been tested by

cd os
make run

···

**************/
Rust user shell
>> usertests

···

34 of sueecssed apps, 9 of failed apps run correctly. 
Usertests passed!

and the results remain the same as before.

wyfcyx commented 9 months ago

That's great! Could you please:

Fix the conflicts and install the gdb as the latest Dockerfile does
Setup the rustup/cargo mirror to tsinghua/ustc by default. This is because, now it still takes a long period of time to install rust.

Thank you!

jklincn commented 9 months ago

It's done.

Tsinghua mirror for rustup may exist some problems. So I use ustc mirror.

Details

According to [website instructions](https://mirrors.tuna.tsinghua.edu.cn/help/rustup/), I set the environment variable. ``` RUSTUP_DIST_SERVER=https://mirrors.tuna.tsinghua.edu.cn/rustup RUSTUP_UPDATE_ROOT=https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup ``` But when I was installing Rust, some errors occurred. ``` 0.729 curl: (22) The requested URL returned error: 404 0.730 rustup: command failed: downloader https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup/dist/x86_64-unknown-linux-gnu/rustup-init /tmp/tmp.bg0230SesP/rustup-init x86_64-unknown-linux-gnu ``` So I set ustc mirror, and there is nothing wrong. ``` RUSTUP_DIST_SERVER=https://mirrors.ustc.edu.cn/rust-static RUSTUP_UPDATE_ROOT=https://mirrors.ustc.edu.cn/rust-static/rustup ```

About the gdb, I think gdb-multiarch can meet the demand, which can install directly through apt. To maintain compatibility with the existing makefile, I set up a symbolic link.

ln -s /usr/bin/gdb-multiarch /usr/bin/riscv64-unknown-elf-gdb

I conducted the following tests:

First Windows PowerShell (create the container and run QEMU in gdb mode)

PS C:\Users\jklincn\rCore-Tutorial-v3> docker run --rm -it -v C:\Users\jklincn\rCore-Tutorial-v3:/mnt -w /mnt --name rcore-tutorial-v3 rcore-tutorial-v3:latest bash
root@67bd3a91d578:/mnt# cd os 
root@67bd3a91d578:/mnt/os# make gdbserver

Second Windows PowerShell (create the second terminal and run gdb)

PS C:\Users\jklincn\rCore-Tutorial-v3> docker exec -it rcore-tutorial-v3 bash      
root@67bd3a91d578:/mnt# cd os 
root@67bd3a91d578:/mnt/os# make gdbclient

GNU gdb (Ubuntu 9.2-0ubuntu1~20.04.1) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
  <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word".
Reading symbols from target/riscv64gc-unknown-none-elf/release/os...
The target architecture is assumed to be riscv:rv64
Remote debugging using localhost:1234
0x0000000000001000 in ?? ()
(gdb) b src/main.rs:57
Breakpoint 1 at 0x80219380: file src/main.rs, line 58.
(gdb) c
Continuing.

Breakpoint 1, os::rust_main () at src/main.rs:58
58          mm::init();
(gdb)

This shows that gdb-multiarch works well.

By the way, why I use raw command instead of rules in makefile? I think most Docker users are in a Windows environment. Firstly, Windows does not have 'make', and secondly, syntax like '$pwd' is not usable in Windows.

However, the latest commit further simplifies the use of Docker, which can avoid these problems with the makefile. The dev container is great! #134

wyfcyx commented 9 months ago

This error occurred when I tried to make run:

error: failed to run custom build command for `os v0.1.0 (/workspaces/rCore-Tutorial-v3/os)`

Caused by:
  process didn't exit successfully: `/workspaces/rCore-Tutorial-v3/os/target/release/build/os-7e6bed51b1b92f69/build-script-build` (exit status: 1)
  --- stderr
  /workspaces/rCore-Tutorial-v3/os/target/release/build/os-7e6bed51b1b92f69/build-script-build: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by /workspaces/rCore-Tutorial-v3/os/target/release/build/os-7e6bed51b1b92f69/build-script-build)
  /workspaces/rCore-Tutorial-v3/os/target/release/build/os-7e6bed51b1b92f69/build-script-build: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by /workspaces/rCore-Tutorial-v3/os/target/release/build/os-7e6bed51b1b92f69/build-script-build)
  /workspaces/rCore-Tutorial-v3/os/target/release/build/os-7e6bed51b1b92f69/build-script-build: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by /workspaces/rCore-Tutorial-v3/os/target/release/build/os-7e6bed51b1b92f69/build-script-build)
warning: build failed, waiting for other jobs to finish...
make: *** [Makefile:60: kernel] Error 101
root@b357e6421778:/workspaces/rCore-Tutorial-v3/os# ldd --version
ldd (Ubuntu GLIBC 2.31-0ubuntu9.14) 2.31
Copyright (C) 2020 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

It seems that the latest Rust toolchain requires glibc version 2.32. Unfortunately, the version of glibc is 2.31 in our docker environment by default. Maybe we need to upgrade the glibc?

By the way, we still need some network proxy tricks even if we have set the rustup/cargo mirror...

remove proxy in ~/.gitconfig
set network proxy in docker desktop

jklincn commented 9 months ago

Are you using GitHub's workspace? I haven't encountered such errors in my local Docker container.
When building without a network proxy, I found that it gets stuck during the download of the QEMU source code package. In such cases, if one insists on using the official download link, there might not be a good solution. Typically, I have two solutions for this: one is to download the package to a server where it can be directly accessed from within China, like Alibaba Cloud Object Storage. The other is to directly provide users with a pre-built image, which can be hosted on Docker Hub (though Docker Hub might need a change of mirror source, but this is very simple).

wyfcyx commented 9 months ago

~It seems that you are not using the devcontainer extension of the vscode to build and run the container. When I tried to do that, the error of glibc version encountered. When I manually build and run the container, the same error encountered. That is weird...~

It works! I forgot to clean the objects compiled in my ubuntu22.04 wsl, making the difference of the glibc version. Thanks to this post.

jklincn commented 9 months ago

Congratulation!

rcore-os / rCore-Tutorial-v3

Rewrite Dockerfile #139

138