Slower run-times

[boev]

Your environment (OS/HW): Ubuntu 18.04, GMP 6.1.2, NTL 11.3.2

Hi, has anyone noticed performance decrease with newer changes? I re-ran my benchmarks multiple times and encryption, addition, multiplication, binary circuits have become noticeably slower (x1.5-2). I have not changed anything on my side/machine.

I believe my environment remained unchanged, but cannot be 100% sure. The benchmarks also show a slight increase in reported remaining capacity, even in fresh encryptions, which is good.

[victorshoup]

That’s interesting. It’s good if we try to track such issues. If you can, please try to drill down on some of these, and provide detailed feedback with program outputs that contain as much debug info as possible. Or, at least, send some programs and sample inputs and I can try to dig into it.

I made some big changes with respect to bootstrapping. That may be affecting your run times. For some parameter settings, the code may make some more conservative settings for bootstrapping parameters. There’s really not much that can be done about that: with the old code, the probability of recryption errors could be unacceptably high, and the new code is actually fairly optimal in choosing acceptable parameters.

But...there could be “performance bugs” that were introduced and should be investigated. Certainly, anything not involving bootstrapping should not have been affected.

[boev]

Hi, thanks for the reply. The benchmarks did not involve bootstrapping. Previously I often ran into problem using it, except for some predefined tested contexts. I will investigate further, probably I changed a config on my part.

[victorshoup]

OK. Well that's good to know. The one other thing that I am concerned about is the fact that we are now calling a floating point FFT with some regular frequency. However, my timing tests indicate that (depending on the mix of operations) this should not have to big of an impact (less than 10%).

Like I said, if you can, you should drill down on one or two examples, and if there is an issue, I'm sure we can sort it out.

[boev]

Hi, I ran some benchmark on the current version and on reverted commit https://github.com/homenc/HElib/commit/61f453bf0daf4fce51f0f48a817184e233a48d22. Since I used the same machine and code, environment should not be the reason.

bench_old_vs_new.tar.gz

I think multiplication is a bit slower and this affects the runtime of the comparison procedure. This would be my first guess. I will look further into the changes, try other contexts.

[victorshoup]

Thanks. I think it is worth focusing on this:

old: MUL OPERATION: 0.15761630 0.24245502 new: MUL OPERATION: 0.17520786 0.35988877

I don't know of any changes we made that should have such an impact on the average running time of a MUL operation.

To help us out, you could do the following.

1. Write a program that just does this MUL test.
2. Instrument it so that is also prints out HElib timers...I don't know if you are familiar with those, but most of the test programs contain examples of this. I think you just have to call primtAllTimers(). Also, you should call resetAllTimers() just before you start the MUL tests. Also, call the CheckCtxt function on the ciphertexts right before you start the MUL tests, just to verify that the "sizes" of everything are the same.

Thanks!

[boev]

Yes I also thought the multiplication can be the cause. Will try to produce a performance test. Thanks.

[qianlou]

Hi, boev. I am so interested in benchmarking tasks you did. Could you mind sharing your benchmark script? my email address is louqian95@gmail.com Or could you tell me how I can benchmark those kinds of computations? Thanks very much.