rust-lang / rust

Empowering everyone to build reliable and efficient software.
https://www.rust-lang.org
Other
93.58k stars 12.05k forks source link

[EXPERIMENT] Ownership analysis [DO NOT MERGE] #124756

Closed xFrednet closed 3 days ago

xFrednet commented 1 week ago

TL;DR: I want to collect data about ownership in Rust's ecosystem. I modified Clippy to extract things from MIR. This PR is intended for a crater run, but should definitely not be merged.


Roses are red, Violets are blue, Can I borrow, A pattern from you?

So, what is this project all about? Well, last year I was looking for a topic for my master's thesis. I had the incredible luck that the programming languages group at my university is not only a collection of incredibly cool people but also has some interest in Rust. It just so happens that @amandasystems is part of the group.

We then spent some time looking for a thesis project in the domain of Rust's memory ownership model, and finally settled on the idea to analyze how the memory ownership model is used in practice.

For Polonius, it could be interesting to know how many bodies contain borrows and if the changes from the NLL RFC are actually used, etc.

Implementation

Disclaimer, this PR contains some of the most convoluted and spaghettified code I've ever written. Do not use this as an example for anything! I'm pretty sure that the insufficiencies of this PR single-handedly increases the compile time by 100%. It's honestly a miracle that the whole thing doesn't ICE on every function.

So at the beginning, I decided to use MIR as that is the IR that the borrow check uses, and it contains all the borrows and moves which might not be explicit in the HIR. While all nice and good it turns out that MIR also gets writ of information which I would have wanted for my analysis.

To be clear, MIR is an excellent tool for what it was designed to do, but its loss of information has been a struggle for this project specifically.

I would like to collect way more metadata, more complex patterns, and data how named references are used, but the deadline for my thesis is coming up. We decided to limit the scope to named owned variables and see how the analysis goes. If I find some interesting things there is potential to extend this work.

Run

I decided to implement the analysis on Clippy for the shorter compile times and familiar codebase. This branch disables all other lints by default to safe on some runtime. The crater run with Clippy from master might also expose some ICEs which could be interesting for us.

The results of this will be aggregated and published as part of my thesis. The time imposed limitations of this implementation probably mean that the results have to be taken with a metric ton of salt, but they should be sufficient to pass my thesis and know if more work in this area would be interesting.

I don't know if there is much more too to say. Let's hope:


r? @ghost

rust-log-analyzer commented 1 week ago

The job x86_64-gnu-llvm-17 failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot) ```plain #16 exporting to docker image format #16 sending tarball 28.6s done #16 DONE 40.6s ##[endgroup] Setting extra environment values for docker: --env ENABLE_GCC_CODEGEN=1 --env GCC_EXEC_PREFIX=/usr/lib/gcc/ [CI_JOB_NAME=x86_64-gnu-llvm-17] --- sccache: Starting the server... ##[group]Configure the build configure: processing command line configure: configure: build.configure-args := ['--build=x86_64-unknown-linux-gnu', '--llvm-root=/usr/lib/llvm-17', '--enable-llvm-link-shared', '--set', 'rust.thin-lto-import-instr-limit=10', '--set', 'change-id=99999999', '--enable-verbose-configure', '--enable-sccache', '--disable-manage-submodules', '--enable-locked-deps', '--enable-cargo-native-static', '--set', 'rust.codegen-units-std=1', '--set', 'dist.compression-profile=balanced', '--dist-compression-formats=xz', '--disable-dist-src', '--release-channel=nightly', '--enable-debug-assertions', '--enable-overflow-checks', '--enable-llvm-assertions', '--set', 'rust.verify-llvm-ir', '--set', 'rust.codegen-backends=llvm,cranelift,gcc', '--set', 'llvm.static-libstdcpp', '--enable-new-symbol-mangling'] configure: target.x86_64-unknown-linux-gnu.llvm-config := /usr/lib/llvm-17/bin/llvm-config configure: llvm.link-shared := True configure: rust.thin-lto-import-instr-limit := 10 configure: change-id := 99999999 ```
xFrednet commented 1 week ago

It's green :D

Now let's see bors destroy all my hopes and dreams. /s JK love you bors

@bors try

bors commented 1 week ago

:hourglass: Trying commit 813c97a401764cabf8291f2b28da2af33b47bd3b with merge 541a4da3b58309ad64ca0d6b33aabfba94b445bb...

bors commented 1 week ago

:sunny: Try build successful - checks-actions Build commit: 541a4da3b58309ad64ca0d6b33aabfba94b445bb (541a4da3b58309ad64ca0d6b33aabfba94b445bb)

Mark-Simulacrum commented 1 week ago

@craterbot run mode=clippy

craterbot commented 1 week ago

:ok_hand: Experiment pr-124756 created and queued. :robot: Automatically detected try build 541a4da3b58309ad64ca0d6b33aabfba94b445bb :mag: You can check out the queue and this experiment's details.

:information_source: Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Mark-Simulacrum commented 1 week ago

@craterbot abort

Need to add clippy to try CI.

craterbot commented 1 week ago

:wastebasket: Experiment pr-124756 deleted!

:information_source: Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

xFrednet commented 1 week ago

@bors try

bors commented 1 week ago

:hourglass: Trying commit 4d7e11d65d6559f1d9c2a424ddbc1c03b099aa81 with merge a353969eafa00eade807d5bd90455d3bc3e4a489...

bors commented 1 week ago

:sunny: Try build successful - checks-actions Build commit: a353969eafa00eade807d5bd90455d3bc3e4a489 (a353969eafa00eade807d5bd90455d3bc3e4a489)

Mark-Simulacrum commented 1 week ago

@craterbot run mode=clippy

craterbot commented 1 week ago

:ok_hand: Experiment pr-124756 created and queued. :robot: Automatically detected try build a353969eafa00eade807d5bd90455d3bc3e4a489 :mag: You can check out the queue and this experiment's details.

:information_source: Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Mark-Simulacrum commented 6 days ago

@craterbot p=1

craterbot commented 6 days ago

:memo: Configuration of the pr-124756 experiment changed.

:information_source: Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

xFrednet commented 6 days ago

Thank you @Mark-Simulacrum, now it will run before the beta-1.79-2 test, should be increase the prio of that one to make sure that it runs first or is it not time critical?

craterbot commented 6 days ago

:construction: Experiment pr-124756 is now running

:information_source: Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

craterbot commented 4 days ago

:tada: Experiment pr-124756 is completed! :bar_chart: 30230 regressed and 17032 fixed (446250 total) :newspaper: Open the full report.

:warning: If you notice any spurious failure please add them to the blacklist! :information_source: Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

xFrednet commented 3 days ago

I'm closing this since the experiment is done running. Thank you for kicking off crater!

Now I'll need to take some time to look at the data. :)