apache / datafusion

Apache DataFusion SQL Query Engine
https://datafusion.apache.org/
Apache License 2.0
5.87k stars 1.11k forks source link

Building time for `cargo bench` takes quite a long time #11569

Open jayzhan211 opened 1 month ago

jayzhan211 commented 1 month ago

Is your feature request related to a problem or challenge?

Running cargo bench --bench math_query_sql takes almost 4min for building, @goldmedal reported that it takes about 10min for him

Most of the time stuck at datafusion/core crate.

Describe the solution you'd like

Find the root cause and reduce the building time if possible

Describe alternatives you've considered

No response

Additional context

No response

goldmedal commented 1 month ago

By the way, I only modified a small part of the codebase, but it takes about 10 minutes to compile (I have builded it once). I guess it rebuilds everything every time.

Omega359 commented 1 month ago

5:45 here in a docker .devcontainer to run that bench command. I suspect a good portion of it is because cargo bench uses the bench profile which inherits from release. Adding --profile=dev to the command reduces the build down to 56s here.

jayzhan211 commented 1 month ago

5:45 here in a docker .devcontainer to run that bench command. I suspect a good portion of it is because cargo bench uses the bench profile which inherits from release. Adding --profile=dev to the command reduces the build down to 56s here.

release mode is prefer for benchmarking, therefore it doesn't seem like a good idea to add --profile=dev

btw, I thought dev mode usually has longer build time, why is it not 🤔