An open-source XBRL processor for business rules, rendering and custom data reporting. See https://xbrl.us/xule for documentation and https://xbrl.us/xule-editor for a VS Code syntax highlighter.
Apache License 2.0
24
stars
9
forks
source link
Add argument for compiling rule files in parallel #24
This adds an argument (--xule-compile-workers) which can be used to specify the number of processes for compiling rules in parallel. The default value is 1 and a value of 0 will use the number of CPUs available on the system.
Precise performance improvements depend on the system and number of workers used. Using all cores (--xule-compile-workers=0) on my M1 MacBook Pro drops compile time for US GAAP 2023 from 1:32 mins to 0:27 and compiling all rule sets for all years of IFRS, US GAAP, and ESEF from 7:18 to 2:46.
We introduced code in Arelle 2.17.3 to facilitate multiprocessing plugins, multiple workers requires at least that version of Arelle. Additionally, due to bugs in the cx_Freeze library which is used to produce the frozen Arelle builds (exe on windows, dmg on macOS, and tgz on Linux) multiple workers requires running Arelle and XULE from Python source. cx_Freeze is looking to resolve their multiprocessing bugs in the next minor release, so we may be able to remove this restriction in the future.
This adds an argument (
--xule-compile-workers) which can be used to specify the number of processes for compiling rules in parallel. The default value is1and a value of0will use the number of CPUs available on the system.Precise performance improvements depend on the system and number of workers used. Using all cores (
--xule-compile-workers=0) on my M1 MacBook Pro drops compile time for US GAAP 2023 from 1:32 mins to 0:27 and compiling all rule sets for all years of IFRS, US GAAP, and ESEF from 7:18 to 2:46.We introduced code in Arelle 2.17.3 to facilitate multiprocessing plugins, multiple workers requires at least that version of Arelle. Additionally, due to bugs in the cx_Freeze library which is used to produce the frozen Arelle builds (
exeon windows,dmgon macOS, andtgzon Linux) multiple workers requires running Arelle and XULE from Python source. cx_Freeze is looking to resolve their multiprocessing bugs in the next minor release, so we may be able to remove this restriction in the future.@campbellpryde @davidtauriello @marcward @derekgengenbacher-wf