SebastianBoe
commented
5 years ago PR submitted for CONFIG_LTO: https://github.com/zephyrproject-rtos/zephyr/pull/12163
Closed ulfalizer closed 5 years ago
SebastianBoe
commented
5 years ago PR submitted for CONFIG_LTO: https://github.com/zephyrproject-rtos/zephyr/pull/12163
galak
commented
5 years ago Can you provide a script that generates this output so we can update this PR from time to time with the latest results.
galak
commented
5 years ago Can you provide a script that generates this output so we can update this PR from time to time with the latest results.
@ulfalizer are you able to provide a script
nashif
commented
5 years ago CONFIG_ALTERA_AVALON_EPCS ext/hal/altera/CMakeLists.txt:84
CONFIG_ALTERA_AVALON_SGDMA ext/hal/altera/CMakeLists.txt:52
CONFIG_ALTERA_AVALON_SPI ext/hal/altera/CMakeLists.txt:76
CONFIG_ALTERA_AVALON_UART ext/hal/altera/CMakeLists.txt:27
CONFIG_ARM_SAU arch/arm/core/cortex_m/tz/arm_core_tz.c:138, arch/arm/core/cortex_m/tz/arm_core_tz.c:166, arch/arm/include/cortex_m/tz.h:234, arch/arm/include/cortex_m/tz.h:270addressed in https://github.com/zephyrproject-rtos/zephyr/pull/12875
ulfalizer
commented
5 years ago @galak Here's the script I used, run from the Zephyr root. I'll clean it up a bit later once I'm done moving.
Will need to have scripts/kconfig in $PYTHONPATH so that kconfiglib can be found. Could also install it with sudo pip3 kconfiglib or just copy/symlink it into the Zephyr root.
import collections
import os
import subprocess
import sys
import kconfiglib
os.environ["SOC_DIR"] = "soc/"
os.environ["BOARD_DIR"] = "boards/*/*"
os.environ["ARCH"] = "*"
defined_syms = set(sym.name for sym in kconfiglib.Kconfig().unique_defined_syms)
# gitglossary "magic signature: /"
grep_cmd = ("git", "grep", "--only-matching", "--line-number",
"--extended-regexp", "--word-regexp", "-I", "--null",
"CONFIG_[A-Z0-9_]+")
grep_process = subprocess.Popen(grep_cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, stderr = grep_process.communicate()
if grep_process.returncode:
# stderr will be 'bytes' here, which prints as b'oops'
sys.exit("error: '{}' failed with exit code {} and wrote {} to stderr"
.format(" ".join(grep_cmd), grep_process.returncode, stderr))
stdout = stdout.decode("utf-8")
undef_to_locs = collections.defaultdict(list)
for line in stdout.splitlines():
path, lineno, sym_name = line.split("\0")
if sym_name[7:] not in defined_syms and not sym_name.endswith("_"):
undef_to_locs[sym_name].append("{}:{}".format(path, lineno))
for name in sorted(undef_to_locs):
print("{:35} {}".format(name, ", ".join(undef_to_locs[name])))CONFIG_UART_QMSI_0_BAUDRATE boards/x86/arduino_101/arduino_101_mcuboot_defconfig:10
CONFIG_UART_QMSI_1_BAUDRATE boards/x86/arduino_101/doc/index.rst:446
CONFIG_COMPILER_OPT_AS_LIST CMakeLists.txt:286, CMakeLists.txt:287
CONFIG_CMU soc/arm/silabs_exx32/common/soc.c:124
CONFIG_COUNTER_RTC_STM32_CLOCK_SRC drivers/counter/counter_ll_stm32_rtc.c:286
CONFIG_LIS2DW12_INT_PIN drivers/sensor/lis2dw12/lis2dw12.c:358
CONFIG_DCD_DSR samples/net/wpan_serial/src/main.c:600
CONFIG_DIR doc/CMakeLists.txt:175, doc/CMakeLists.txt:203, doc/CMakeLists.txt:232
CONFIG_FIRQ_NO_LPCC arch/arc/core/fast_irq.S:78, arch/arc/core/fast_irq.S:135
CONFIG_ETH_DW_0_IRQ_SHARED_NAME drivers/ethernet/eth_dw.c:370, soc/x86/intel_quark/quark_x1000/dts_fixup.h:48
CONFIG_GPIO_DW_0_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:505, soc/x86/intel_quark/quark_x1000/dts_fixup.h:44
CONFIG_GPIO_DW_1_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:574
CONFIG_GPIO_DW_2_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:641
CONFIG_GPIO_DW_3_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:708
CONFIG_I2C_DW_0_IRQ_SHARED_NAME drivers/i2c/i2c_dw.c:707, soc/x86/intel_quark/quark_x1000/dts_fixup.h:40
CONFIG_ISA_ARM include/toolchain/gcc.h:170
CONFIG_ISA_THUMB include/toolchain/common.h:67, include/toolchain/gcc.h:153, include/toolchain/gcc.h:296
CONFIG_MPXXDTYY_LOG_LEVEL drivers/audio/mpxxdtyy-i2s.c:10, drivers/audio/mpxxdtyy.c:9
CONFIG_NET_DHCPV6 samples/net/nats/src/main.c:135, samples/net/nats/src/main.c:174, samples/net/nats/src/main.c:269
CONFIG_NET_GPTP_STACK_SIZE subsys/net/l2/ethernet/gptp/gptp.c:22, subsys/net/l2/ethernet/gptp/gptp.c:23, subsys/net/l2/ethernet/gptp/gptp.c:35, subsys/net/l2/ethernet/gptp/gptp.c:36
CONFIG_NET_ICMPV4 subsys/net/ip/net_shell.c:731
CONFIG_NET_ICMPV6 subsys/net/ip/net_shell.c:731
CONFIG_REG1 drivers/gpio/gpio_dw_registers.h:59
CONFIG_REG2 drivers/gpio/gpio_dw_registers.h:58
CONFIG_SRAM2 samples/application_development/code_relocation/linker_arm_sram2.ld:24, samples/application_development/code_relocation/linker_arm_sram2.ld:28, samples/application_development/code_relocation/linker_arm_sram2.ld:37
CONFIG_NS16550_REG_SHIFT drivers/serial/uart_ns16550.c:193, drivers/serial/uart_ns16550.c:194, soc/arm/microchip_mec/mec1701/dts_fixup.h:9
CONFIG_UART_NS16550_PORT_2_BAUD_RATE soc/arc/snps_arc_iot/dts_fixup.h:36
CONFIG_UART_NS16550_PORT_2_IRQ_FLAGS drivers/serial/uart_ns16550.c:911, soc/arc/snps_emsk/soc.h:53
CONFIG_UART_NS16550_PORT_2_IRQ_PRI soc/arc/snps_arc_iot/dts_fixup.h:38
CONFIG_UART_NS16550_PORT_2_NAME soc/arc/snps_arc_iot/dts_fixup.h:37
CONFIG_UART_NS16550_PORT_3_BASE_ADDR drivers/serial/uart_ns16550.c:943, soc/x86/apollo_lake/soc.c:52
CONFIG_UART_NS16550_PORT_3_BAUD_RATE drivers/serial/uart_ns16550.c:944, soc/arc/snps_arc_iot/dts_fixup.h:44
CONFIG_UART_NS16550_PORT_3_CLK_FREQ drivers/serial/uart_ns16550.c:925
CONFIG_UART_NS16550_PORT_3_IRQ drivers/serial/uart_ns16550.c:962, drivers/serial/uart_ns16550.c:966
CONFIG_UART_NS16550_PORT_3_IRQ_FLAGS drivers/serial/uart_ns16550.c:965
CONFIG_UART_NS16550_PORT_3_IRQ_PRI drivers/serial/uart_ns16550.c:963, soc/arc/snps_arc_iot/dts_fixup.h:46
CONFIG_UART_NS16550_PORT_3_NAME drivers/serial/uart_ns16550.c:952, soc/arc/snps_arc_iot/dts_fixup.h:45
CONFIG_XUK_64_BIT_ABI arch/x86_64/core/xuk-config.h:22, arch/x86_64/core/xuk.c:111Deal with CONFIG_MCUX_IGPIO* in PR #12883
carlescufi
commented
5 years ago @ulfalizer @galak please update the script to whitelist CONFIG_BT_6LOWPAN. That one is there because it's a Linux Kconfig option.
ulfalizer
commented
5 years ago @carlescufi At some point (probably soon), I will add a CI check and just whitelist everything that remains.
ulfalizer
commented
5 years ago Here's an updated list. Some of them aren't meant to come from Kconfig, and should probably be renamed.
CONFIG_DCD_DSR samples/net/wpan_serial/src/main.c:600
CONFIG_ETH_DW_0_IRQ_SHARED_NAME drivers/ethernet/eth_dw.c:370, soc/x86/intel_quark/quark_x1000/dts_fixup.h:48
CONFIG_FIRQ_NO_LPCC arch/arc/core/fast_irq.S:78, arch/arc/core/fast_irq.S:135
CONFIG_GPIO_DW_0_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:505, soc/x86/intel_quark/quark_x1000/dts_fixup.h:44
CONFIG_GPIO_DW_1_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:574
CONFIG_GPIO_DW_2_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:641
CONFIG_GPIO_DW_3_IRQ_SHARED_NAME drivers/gpio/gpio_dw.c:708
CONFIG_I2C_DW_0_IRQ_SHARED_NAME drivers/i2c/i2c_dw.c:707, soc/x86/intel_quark/quark_x1000/dts_fixup.h:40
CONFIG_ISA_ARM include/toolchain/gcc.h:170
CONFIG_ISA_THUMB include/toolchain/common.h:67, include/toolchain/gcc.h:153, include/toolchain/gcc.h:296
CONFIG_MPXXDTYY_LOG_LEVEL drivers/audio/mpxxdtyy-i2s.c:10, drivers/audio/mpxxdtyy.c:9
CONFIG_NET_DHCPV6 samples/net/nats/src/main.c:135, samples/net/nats/src/main.c:174, samples/net/nats/src/main.c:269
CONFIG_NET_GPTP_STACK_SIZE subsys/net/l2/ethernet/gptp/gptp.c:22, subsys/net/l2/ethernet/gptp/gptp.c:23, subsys/net/l2/ethernet/gptp/gptp.c:35, subsys/net/l2/ethernet/gptp/gptp.c:36
CONFIG_NET_ICMPV4 subsys/net/ip/net_shell.c:731
CONFIG_NET_ICMPV6 subsys/net/ip/net_shell.c:731
CONFIG_NS16550_REG_SHIFT drivers/serial/uart_ns16550.c:193, drivers/serial/uart_ns16550.c:194, soc/arm/microchip_mec/mec1701/dts_fixup.h:9
CONFIG_REG1 drivers/gpio/gpio_dw_registers.h:59
CONFIG_REG2 drivers/gpio/gpio_dw_registers.h:58
CONFIG_SEL boards/nios2/altera_max10/doc/index.rst:29
CONFIG_SRAM doc/guides/dts/index.rst:299
CONFIG_SRAM2 samples/application_development/code_relocation/linker_arm_sram2.ld:24, samples/application_development/code_relocation/linker_arm_sram2.ld:28, samples/application_development/code_relocation/linker_arm_sram2.ld:37
CONFIG_UART_NS16550_PORT_2_BAUD_RATE soc/arc/snps_arc_iot/dts_fixup.h:36
CONFIG_UART_NS16550_PORT_2_IRQ_FLAGS drivers/serial/uart_ns16550.c:911, soc/arc/snps_emsk/soc.h:53
CONFIG_UART_NS16550_PORT_2_IRQ_PRI soc/arc/snps_arc_iot/dts_fixup.h:38
CONFIG_UART_NS16550_PORT_2_NAME soc/arc/snps_arc_iot/dts_fixup.h:37
CONFIG_UART_NS16550_PORT_3_BASE_ADDR drivers/serial/uart_ns16550.c:943, soc/x86/apollo_lake/soc.c:52
CONFIG_UART_NS16550_PORT_3_BAUD_RATE drivers/serial/uart_ns16550.c:944, soc/arc/snps_arc_iot/dts_fixup.h:44
CONFIG_UART_NS16550_PORT_3_CLK_FREQ drivers/serial/uart_ns16550.c:925
CONFIG_UART_NS16550_PORT_3_IRQ drivers/serial/uart_ns16550.c:962, drivers/serial/uart_ns16550.c:966
CONFIG_UART_NS16550_PORT_3_IRQ_FLAGS drivers/serial/uart_ns16550.c:965
CONFIG_UART_NS16550_PORT_3_IRQ_PRI drivers/serial/uart_ns16550.c:963, soc/arc/snps_arc_iot/dts_fixup.h:46
CONFIG_UART_NS16550_PORT_3_NAME drivers/serial/uart_ns16550.c:952, soc/arc/snps_arc_iot/dts_fixup.h:45
CONFIG_UART_QMSI_0_BAUDRATE boards/x86/arduino_101/arduino_101_mcuboot_defconfig:10
CONFIG_UART_QMSI_1_BAUDRATE boards/x86/arduino_101/doc/index.rst:446
CONFIG_XUK_64_BIT_ABI arch/x86_64/core/xuk-config.h:22, arch/x86_64/core/xuk.c:111Pinning for a while to hopefully get rid of a few more. Will dump the rest into a whitelist and add a CI check after that.
Maybe the CI check should be restricted to just code too.
galak
commented
5 years ago Pinning for a while to hopefully get rid of a few more. Will dump the rest into a whitelist and add a CI check after that.
Maybe the CI check should be restricted to just code too.
Did you update your script?
ulfalizer
commented
5 years ago @galak Hacked up a new version that works. Can get a reference for how to set the environment variables from scripts/check_compliance.py in the ci-tools repo.
import collections
import os
import subprocess
import sys
sys.path.insert(0, "scripts/kconfig")
import kconfiglib
os.environ["SOC_DIR"] = "soc/"
os.environ["ARCH_DIR"] = "arch/"
os.environ["BOARD_DIR"] = "boards/*/*"
os.environ["ARCH"] = "*"
os.environ["GENERATED_DTS_BOARD_CONF"] = "dummy"
defined_syms = set(sym.name for sym in kconfiglib.Kconfig().unique_defined_syms)
# gitglossary "magic signature: /"
grep_cmd = ("git", "grep", "--only-matching", "--line-number",
"--extended-regexp", "--word-regexp", "-I", "--null",
"CONFIG_[A-Z0-9_]+")
grep_process = subprocess.Popen(grep_cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
stdout, stderr = grep_process.communicate()
if grep_process.returncode:
# stderr will be 'bytes' here, which prints as b'oops'
sys.exit("error: '{}' failed with exit code {} and wrote {} to stderr"
.format(" ".join(grep_cmd), grep_process.returncode, stderr))
stdout = stdout.decode("utf-8")
undef_to_locs = collections.defaultdict(list)
for line in stdout.splitlines():
path, lineno, sym_name = line.split("\0")
if sym_name[7:] not in defined_syms and not sym_name.endswith("_"):
undef_to_locs[sym_name].append("{}:{}".format(path, lineno))
for name in sorted(undef_to_locs):
print("{:35} {}".format(name, ", ".join(undef_to_locs[name])))Ah... maybe you meant re. false positives in samples/, etc. Nope, not updated for that yet.
galak
commented
5 years ago Ah... maybe you meant re. false positives in
samples/, etc. Nope, not updated for that yet.
that was one question. Another is when running:
Kconfig.zephyr:43: '/Kconfig.modules' does not exist. Perhaps the $srctree
environment variable (unset or blank) is set incorrectly. Note that the current
value of $srctree is saved when the Kconfig instance is created (for consistency
and to cleanly separate instances).@galak
Oh, right, I also removed the source at Kconfig.zephyr:43. CI just writes a blank Kconfig.modules. It's for including extra Kconfig files in external projects I think.
andrewboie
commented
5 years ago Do we have PRs addressing all of these? High-priority bug, gating release.
ulfalizer
commented
5 years ago PR that adds a CI check: https://github.com/zephyrproject-rtos/ci-tools/pull/21. Runs as part of ci-tools/scripts/check_compliance.py.
A more up-to-date list of references to undefined symbols can be found in the PR (some stuff got fixed over the last day that I need to remove from the whitelist too).
ulfalizer
commented
5 years ago @andrewboie IMO, not all of them need to be fixed before release (someone else set it to high priority).
Once the CI check is there, it's easy to kill off the stragglers, without new stuff getting introduced.
nashif
commented
5 years ago reduced priority as most of the issue were fixed.
galak
commented
5 years ago @ulfalizer did you open issues or fixes for these two:
CONFIG_DCD_DSR samples/net/wpan_serial/src/main.c:600
CONFIG_NET_DHCPV6 samples/net/nats/src/main.c:135, samples/net/nats/src/main.c:174, samples/net/nats/src/main.c:269
Submitted PRs #14153 #14152 and #14151 to deal with last few Kconfig symbols that we don't have explicit issues for.
galak
commented
5 years ago Closing this out as we have an issue open for the last cleanup related to this.
Here's a list of Kconfig symbols that are referenced somewhere but not defined in the (main) Kconfig tree, sorted by name and with the locations where they're referenced. If you see something you recognize, then please submit a PR to remove it.
git grepis nice for finding references. If you want to see when something was removed,git log -p -S'config FOO'often works (search history for commits that add/removeconfig FOOand show the diff).There's probably some false positives in there, but I'm dumping it mostly as is. Once it's been pruned a bit, I'll add some easy whitelisting scheme(s) (could allow undefined symbol names in comments if they're surrounded by single quotes, etc.)
I'm planning to make it an automatic check later.