Closed EFanZh closed 2 months ago
Is this PR trying to take advantages of https://github.com/rust-lang/rust/pull/109999/ ?
Is this PR trying to take advantages of rust-lang/rust#109999 ?
Not exactly. This PR intends to minimize code generated by macro expansion. This is done by moving as much code logic as possible into functions that can be shared by each macro call.
Is this PR trying to take advantages of rust-lang/rust#109999 ?
Not exactly. This PR intends to minimize code generated by macro expansion. This is done by moving as much code logic as possible into functions that can be shared by each macro call.
Thanks for explanation.
Note that we use to have a special case for literals: #366, but it was removed in 2d1a477.
Yes, I’m aware of that, I am trying to bring that optimization back.
It seems that using "opt-level=z", the compiler won’t inline Arguments::as_str
if there are multiple calls to Arguments::as_str
: https://godbolt.org/z/rzsG3fYEK. So both None
and Some
branches are kept in the final binary, which will cause binary size to grow.
Maybe marking Arguments::as_str
as #[inline(always)]
could help, but for now, I think I’ll have to give up specializing literal arguments.
Maybe marking Arguments::as_str as #[inline(always)] could help, but for now, I think I’ll have to give up specializing literal arguments.
You could just pass &fmt::Argument<'_>
instead of pass it by-value, it should remove the generated code for copying it.
@NobodyXu: I have tested the effect of passing &Arguments
, and the result is not much different than passing Arguments
directly. Maybe it’s because Rust already passes large object as references: https://godbolt.org/z/Wz35xYrsr. You can see in the example, passing Arguments
and passing &Arguments
generate the same assembly code.
@NobodyXu: I have tested the effect of passing
&Arguments
, and the result is not much different than passingArguments
directly. Maybe it’s because Rust already passes large object as references: https://godbolt.org/z/Wz35xYrsr. You can see in the example, passingArguments
and passing&Arguments
generate the same assembly code.
Your linked code does not actually use the Arguments, so the compiler might omits the copy requires to happen.
In your PR, you passes the Arguments to log2, log1, log0 by value, that would require copying Arguments between stacks unless inlined.
@NobodyXu: I have tested the effect of passing
&Arguments
, and the result is not much different than passingArguments
directly. Maybe it’s because Rust already passes large object as references: https://godbolt.org/z/Wz35xYrsr. You can see in the example, passingArguments
and passing&Arguments
generate the same assembly code.Your linked code does not actually use the Arguments, so the compiler might omits the copy requires to happen.
In your PR, you passes the Arguments to log2, log1, log0 by value, that would require copying Arguments between stacks unless inlined.
log0
, log1
, log
can only have a fixed number of instances, which means they can only contribute to at most a constant size to the final binary, while the majority binary size comes from codes generated by macro calls, since there could be a lot of log calls for a large project. So optimizing function implementations should not matter too much. But still, I will compare different ways to pass arguments, and choose the best one.
OK, I think this is ready for the next step. This PR is just a proof of concept of some ideas I have, and should not be merged as is. I have tested this PR on my personal project and gained about 2.4% binary size reduction, which is significant to me. Now I want to implement these ideas in the log
crate. Since this PR contains several optimization ideas, and I am not certain that all of them have positive effect on the binary size, so the best way maybe to discuss each idea, and decide whether it is reasonable. Finally, we may implement and merge each acceptable idea individually.
&str
instead of Arguments
if the message is a string literal.
'{'
or '}'
character, see the is_literal
function.&'static (&'static str, &'static str)
, so that logs from a single file can share the same argument.unused
function in this PR), so that downstream crates can utilize them by enabling share-generics
.Also, is there a known open source project that uses log
extensively, so I can test the effect of my optimization on it? I only have a private project to test. I think my optimization will need a reproducible result.
Also, is there a known open source project that uses
log
extensively, so I can test the effect of my optimization on it? I only have a private project to test. I think my optimization will need a reproducible result.
In cargo-binstall
, log
is a dependency that is heavily used:
In
cargo-binstall
,log
is a dependency that is heavily used
It seems that cargo-binstall
mostly uses logging macros from tracing
, not log
. Even though it depends on the log
crate, not using its macros means it may not benefit from these optimizations.
In
cargo-binstall
,log
is a dependency that is heavily usedIt seems that
cargo-binstall
mostly uses logging macros fromtracing
, notlog
. Even though it depends on thelog
crate, not using its macros means it may not benefit from these optimizations.
It uses many dependencies which depends on log
, like gix
, async-zip
, etc.
If you check the "details" which contained the dependency tree, you would find it actually heavily depends on log
.
I have tests this PR on cargo-binstall, and here is the binary size comparison:
opt-level | lto | codegen-units | original size | optimized size | diff |
---|---|---|---|---|---|
3 | off | 1 | 14211752 | 14203560 | -8192 |
3 | off | 16 | 16739048 | 16726760 | -12288 |
3 | thin | 1 | 15170120 | 15170120 | 0 |
3 | thin | 16 | 17566376 | 17574568 | 8192 |
3 | fat | 1 | 13859336 | 13859336 | 0 |
3 | fat | 16 | 15231496 | 15227400 | -4096 |
"z" | off | 1 | 11451048 | 11430568 | -20480 |
"z" | off | 16 | 13531848 | 13503176 | -28672 |
"z" | thin | 1 | 12307048 | 12286568 | -20480 |
"z" | thin | 16 | 13699720 | 13736584 | 36864 |
"z" | fat | 1 | 9951752 | 9931272 | -20480 |
"z" | fat | 16 | 10476040 | 10447368 | -28672 |
You can reproduce the result using this program.
Thanks for investigating this @EFanZh! These were a useful point-in-time reference that helped make some improvements, but I think we should close now that things have moved along. Please feel free to open any fresh PRs that optimize binary size based on the current state of the code though.
Experiment for some ideas I have on binary size optimization.