search

zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.44k stars 6.4k forks source link

Improve build times with userspace #8559

Open SebastianBoe opened 6 years ago

SebastianBoe commented 6 years ago

Enabling user mode causes the build to run significantly slower.

With warm caches ninja increases it's build time from 1 to 6 seconds.

The script with this performance bug is

sebo@mach:~/zephyr/tests/shell/b$ time /usr/bin/python3 /home/sebo/zephyr/scripts/gen_priv_stacks.py --kernel /home/sebo/zephyr/tests/shell/b/zephyr/priv_stacks_prebuilt.elf --output priv_stacks_hash.gperf

real 0m2.493s

This should be optimized such that incremental builds, which are a part of a developers workflow and therefore user experience, is not harmed.

galak commented 6 years ago

Also seeing gen_kobject_list.py taking a few seconds to run. Adding comments here, and will close PR #9196

galak commented 5 years ago

The build times have vastly increased how long it takes to do full CI. We really need to address this for 1.14.

nashif commented 5 years ago

@AdithyaBaglody can you please take a look?

nashif commented 5 years ago

@dcpleung if you can help with this, that would be great.

dcpleung commented 5 years ago

The bottleneck in gen_kobject_list.py seems to be the inside ElfHelper.find_kobject() (step 1). It is using the pyelftools to process the ELF file. So we are at the mercy of the speed of pyelftools.

dcpleung commented 5 years ago

I played with it a bit more. However, it was not possible to parallelize pyelftools via concurrent.futures. The pyelftools uses one file stream object so using threads does not work. This is because of all the file seeking/reading and it cannot be done with one file stream object. Using processes does not work either, as Python cannot pickle pyelftools data fields correctly.

nashif commented 5 years ago

actually, lets not treat this as a bug in Zephyr, cause it is not, moving to an enhancment.

andrewboie commented 5 years ago

I think it's probably OK to default to userspace off, and just leave the hw stack protection stuff on for all tests. Then for all tests that actually create user mode threads, enable userspace there. Just turning on userspace without using it doesn't do much.

andrewboie commented 5 years ago

I considered this some more on how we could optimize it.

gen_kobject_list.py and gen_priv_stacks.py work on the same information, scanning DWARF debug information looking for kernel objects. I think as a first step, we could implement a scanning tool which examines DWARF data only once (instead of twice), and then emits a JSON file containing the relevant data. those scripts would then consume that data.

then the scanning tool could be re-written as a multithreaded application in C, which would be tons faster, although it would introduce a tool in the build system which would need to be compiled, something we have been moving away from in favor of having all our build tools be scripts. is this for Windows build compatibility?

andrewboie commented 5 years ago

C++ libraries https://github.com/aclements/libelfin https://github.com/kittel/libdwarfparser

Actively maintained, but in Rust: https://github.com/gimli-rs/gimli

SebastianBoe commented 5 years ago

is this for Windows build compatibility?

Yes.

andrewboie commented 4 years ago

So at this point we only enable user mode for tests that use it, and not too long ago I merged the privilege stack and kernel object gperf tables, improving build times for ARM.

I think at this point the only further optimization is a faster DWARF scanner. I'm going to lower the priority of this ticket back to 'low'.