fortify / fcli

fcli is a command-line utility for interacting with various Fortify products
https://fortify.github.io/fcli/
Other
31 stars 17 forks source link

performance beta vs devel version #340

Closed xakrurychle closed 1 year ago

xakrurychle commented 1 year ago

$ time fcli -V fcli version 0.20230609.101848-dev_2.0.0-beta, built on 2023-06-09 10:19:16 real 0m0.781s user 0m0.610s sys 0m0.149s

$ time fcli -V fcli version 0.20230629.082654-dev_develop, built on 2023-06-29 08:27:42 real 0m7.271s user 0m0.902s sys 0m1.219s

Using devel branch version takes multiple times more to run commands.

for type in $(fcli ssc token-definition list --session notEncryptedSession -o 'expr={type} \n'); do fcli ssc token create $type --user --password --session notEncryptedSession; done real 2m7.441s user 0m16.319s sys 0m20.341s

Using devel version and not beta version is 7.5x slower. real 0m16.593s user 0m11.913s sys 0m2.296s

rsenden commented 1 year ago

Thanks for reporting, good find! As there's only a relatively small difference in changes between develop and 2.0.0-beta, I'm now running a script to identify which commit introduced the performance degradation.

rsenden commented 1 year ago

As we can see from the performance figures below, performance was good up to commit c8c12c68065eab0d4bdc0f133d03a8d600084e42, with real averaging ~15 seconds (some higher, probably due to higher system load), but then increase to >45 seconds with commit b1471ef1788b8d827065b8c5ef3e139ca02b10a9.

fe2dd798b318d8c984575080780398979a4d2cf9 docs: Update license years
real    0m14.330s
user    0m36.674s
sys 0m2.487s

923f22482f872b7987f4af9a7115025c374da148 chore: Add/update copyright statements
real    0m14.612s
user    0m37.892s
sys 0m2.169s

c02dba78bbd0b8266ac58f7f707d628efb8d0e36 feat!: Refactoring & improve `fcli util all-commands`
real    0m26.208s
user    1m2.000s
sys 0m5.102s

7ed1a5593197d8c49d66687728d89c06d6d34a3b chore: Add documentation link to usage footer
real    0m15.835s
user    0m40.500s
sys 0m2.353s

c8c12c68065eab0d4bdc0f133d03a8d600084e42 chore: Fix top-level fcli command not included in all-commands
real    0m16.911s
user    0m43.206s
sys 0m2.886s

b1471ef1788b8d827065b8c5ef3e139ca02b10a9 chore: Update resource bundles & handling, minor other updates
real    0m50.070s
user    0m52.648s
sys 0m4.836s

30bc9022869f94a5bf04483a42b872a96779cadf fix: `fcli ssc session login`: Show proper error message if no credentials provided (fixes #326)
real    0m47.767s
user    0m47.520s
sys 0m4.080s

e25638aa677370735942319e8d55a4c80c3b466d fix: Properly handle required options in exclusive ArgGroups (fixes #327)
real    0m48.129s
user    0m47.819s
sys 0m4.482s

9682b1cfbba3e9f2bd32edab176f43fa8b2652ce chore: Refactor patch for #327 for better readability & performance
real    0m47.338s
user    0m45.908s
sys 0m4.157s

For reference, the above was generated by the following command (timing 5 Java-based executions of fcli.jar -V for each commit), with manually adjusted line breaks to better indicate to which commit each timing block corresponds:

for c in $(git log --reverse --format="%H" 2.0.0-beta..develop); do date ; git reset --hard > /dev/null ; git clean -fd > /dev/null ; git checkout -q $c && git log --format="%H %s" -n 1 && ./gradlew clean build 2>&1 > /dev/null && git log --format="%H %s" -n 1 >> ~/Downloads/times.txt && { time for i in {1..5}; do java -jar build/libs/fcli.jar -V ; done; } 2>> ~/Downloads/times.txt;  done
rsenden commented 1 year ago

This was caused by a work-around for header and description of the top-level fcli command being inherited by sub-commands that don't explicitly define their own header or description. See https://github.com/remkop/picocli/issues/1465#issuecomment-1621881085 for details.