Open ppalaga opened 4 years ago
@ppalaga Please, assign to me. I'll investigate it.
Hi @ppalaga, I've looked into this problem and I see 2 items:
[INFO] Camel Quarkus :: Integration Tests :: AS2 .......... SUCCESS [25:41 min]
On my local computer it takes usually about 5-6 mins or similar. So I'm not sure what could be wrong.
[camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner:14157] classlist: 9,643.39 ms, 1.68 GB ... [camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner:14157] [total]: 304,349.07 ms, 12.07 GB [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Execute [objcopy, --strip-debug, /home/jondruse/git/camel-quarkus/integration-tests/as2/target/camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner] [INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 308998ms ... [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 05:34 min
[camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner:16243] classlist: 8,989.76 ms, 1.17 GB ... [camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner:16243] [total]: 154,163.27 ms, 5.72 GB [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Execute [objcopy, --strip-debug, /home/jondruse/git/camel-quarkus/integration-tests/as2/target/camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner] [INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 157363ms ... INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 03:00 min
I've tried one optimization by registering only those algorithms/signatures, which are really used. WIP can be seen here https://github.com/apache/camel-quarkus/pull/1686, but CI job failed with unrelated issue. It would be nice to compare results from this test with the reported results.
[camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner:13100] classlist: 9,949.54 ms, 0.93 GB ... [camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner:13100] [total]: 278,546.97 ms, 10.07 GB [INFO] [io.quarkus.deployment.pkg.steps.NativeImageBuildStep] Execute [objcopy, --strip-debug, /home/jondruse/git/camel-quarkus/integration-tests/as2/target/camel-quarkus-integration-test-as2-1.1.0-SNAPSHOT-runner] [INFO] [io.quarkus.deployment.QuarkusAugmentor] Quarkus augmentation completed in 284173ms ... [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 05:11 min
I can see an improvement (but not so big), what do you think?
I think that it could be possible to add some as2 quarkus properties to define really needed bouncycastle services. In case of the current integration test it would look like:
quarkus.as2.digest=sha1,md5,sha512 quarkus.as2.asymmetric=rsa,x509
- Highlighted log line shows that it took 25 mins to finish as2 integration test
[INFO] Camel Quarkus :: Integration Tests :: AS2 .......... SUCCESS [25:41 min]
On my local computer it takes usually about 5-6 mins or similar. So I'm not sure what could be wrong.
Yeah, it's fast on my computer too. There are two factors to take into account:
ci
profile that sets <quarkus.native.native-image-xmx>6g</quarkus.native.native-image-xmx>
https://github.com/apache/camel-quarkus/blob/master/integration-tests/as2/pom.xml#L146 ?The 25mins on the CI are real and too much at the same time. We should try to do something about that.
Limiting the bouncy castle algos sounds like a good idea, and I'd be interested in seeing numbers in the CI. BTW, we have camel-quarkus-support-bouncycastle
extension, so I think the code should go there and the new config options should be namespaced under quarkus.camel.bouncycastle.*
If reducing bouncy castle algos is going to help here, we may try experimenting with it in other tests too where BC is included.
Thanks, reducing bouncy castle ciphers seems to be "doable". I'll try to prepare PR with it and yes, properties will be quarkus.camel.bouncycastle.*
I'll also try to limiting my RAM to see local impact .
BTW, is as2 using always the same fixed set of BC algos? If so, we could perhaps expose a dedicated BuildItem
in the BC extension and let as2 processor produce it, so that the users do not need to bother.
This idea also crossed my mind. Unfortunately ciphers depends entirely on configuration. So values like
quarkus.camel.bouncycastle.digests=sha1,md5,sha512 quarkus.camel.bouncycastle.asymmetricCiphers=rsa,x509
depends on the security configuration defined by user (integration test in this case)
As seen in this job, https://github.com/apache/camel-quarkus/pull/1670/checks?check_run_id=1061031500#step:9:1345 the native compilation of the AS2 itest takes too much RAM and too much time. I wonder whether there is any potential to strip down the amount of compiled code somehow so that less RAM and less time is needed. @JiriOndrusek would you mind having a look?