Compile time issues

[liwei1024]

I used xorstr to encrypt every string in my project, and now it takes hours to compile at a time

[liwei1024]

Is there any way to speed it up

[JustasMasiulis]

I know people that have a thousand+ strings that are encrypted and it takes them a few minutes max.

Do you perhaps have a large blob of binary data or some sort of really long strings?

[liwei1024]

我知道人们有超过一千个已加密的字符串，最多只需要几分钟。

您是否可能有大量的二进制数据或某种_很_长的字符串？

I probably used hundreds of them too, all very short strings

[liwei1024]

Using x64 will be much faster

[JustasMasiulis]

Can't really help you. The number of template instantiations is already quite optimized.

[nikitabuyevich]

Same issue here. Takes hours to compile when optimizations are turned on (Release builds).

If optimization is turned off, it compiles normally.

[JustasMasiulis]

Hmm. Now that this is second report I'll take this a bit more seriously. Could you provide me with this info:

• Approximate size of project
• The time it takes to compile
• Number of xorstr uses
• Approximate average size of the string you are trying to encrypt
• Size of longest string you are trying to encrypt
• Compiler version
• Linker version if you are using something that's not really default to the toolchain

There are also some commands to measure what is bottlenecking the compiler the most, but I don't remember so hopefully those will be enough. As mentioned before - I personally know a person that had around 1000 uses of xorstr in his project and it didn't take anywhere near an hour to compile, so this isn't really something I'd expect.

[rraggerr]

same issue, it takes forever to compile, here is a simple example how to re-create:

#include <iostream>
#include "xorstr.hh"

#define _print {printf(xorstr_("String test! Linecount: %d\n"), linecount);linecount++;}
#define print10lines {_print;_print;_print;_print;_print;_print;_print;_print;_print;_print;}
#define print100lines {print10lines;print10lines;print10lines;print10lines;print10lines;print10lines;print10lines;print10lines;print10lines;print10lines;}
#define print1000lines {print100lines;print100lines;print100lines;print100lines;print100lines;print100lines;print100lines;print100lines;print100lines;print100lines;}

int main( )
{
    static unsigned int linecount = 0;
    print1000lines;
}

tested on msvc2019 ver. 16.6.5 msvc toolset 14.26.28801

issue comes up when i'm encrypting around ~200 strings 2 times print100lines) it takes about a second with 150 strings being encrypted

want to mention that fact, this issue can be fixed if you compile your program with /Od parameter(Disabled C++ optimization) it takes a second even with 1000 lines being encrypted

[JustasMasiulis]

Gonna try out https://devblogs.microsoft.com/cppblog/introducing-c-build-insights/

Should have some results tomorrow evening

[JustasMasiulis]

@rraggerr It takes around 1 minute 30 seconds to compile your example.

Spent a few hours rewriting everything, however it only got down to 1 minute with pretty much every single point of overhead removed.

Can't really make it any better if the bottleneck is the compiler having to optimize a 10k lines function.

[rraggerr]

Are you sure you have enabled msvs c++ optimization? I've been waiting for 20 minutes to compile 1000 lines and i just got bored and canceled compile

[JustasMasiulis]

@rraggerr I am experiencing completely different outcomes. You said that debug build without optimization takes 1 sec, for me it's like 20 seconds; while release takes around 1 minute unlike your 20+ minutes

Can you send me the full solution with the source files and your compiler options.

I should also mention that I am using newest visual studio 16.8 preview 2.0; MSVC v14.28

[rraggerr]

xorstr_tes.zip

[JustasMasiulis]

So the difference was that you were compiling for x86 while I was testing x64. I will fix this tomorrow hopefully

[beranos]

So the difference was that you were compiling for x86 while I was testing x64. I will fix this tomorrow hopefully

hi, any news? i have the same issue as soon i switched to x86

[JustasMasiulis]

So the difference was that you were compiling for x86 while I was testing x64. I will fix this tomorrow hopefully

hi, any news? i have the same issue as soon i switched to x86

So I had made some improvements which helped a little bit but didn't invest enough time to make a version that's releasable

[249383]

ok

Justas Masiulis notifications@github.com 于 2020年9月24日周四 上午12:38写道：

So the difference was that you were compiling for x86 while I was testing x64. I will fix this tomorrow hopefully

hi, any news? i have the same issue as soon i switched to x86

So I had made some improvements which helped a little bit but didn't invest enough time to make a version that's releasable

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JustasMasiulis/xorstr/issues/28#issuecomment-697661894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVYBZQRFIL2O7HMXVOVC2DSHIQARANCNFSM4NGTLROQ .

[249383]

But I use C++

Thurston Abraham thurstonabraham@gmail.com 于 2020年10月3日周六 下午7:28写道：

ok

Justas Masiulis notifications@github.com 于 2020年9月24日周四 上午12:38写道：

So the difference was that you were compiling for x86 while I was testing x64. I will fix this tomorrow hopefully

hi, any news? i have the same issue as soon i switched to x86

So I had made some improvements which helped a little bit but didn't invest enough time to make a version that's releasable

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JustasMasiulis/xorstr/issues/28#issuecomment-697661894, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQVYBZQRFIL2O7HMXVOVC2DSHIQARANCNFSM4NGTLROQ .

[Flawww]

I'm having the exact same issue, I band-aid fixed it by using optimize pragma to turn off optimizations for the heavy functions.

I had one function with about 1000 strings in it, and it compiled to about 1.2mb of machine code (without optimizations) so it makes sense why the compiler had a really hard time optimizing it.

[JustasMasiulis]

I have made some changes that should speed up compilation. Would be nice if you could tell me if this speeds stuff up.

[Flawww]

It seems heavily improved, it could still not handle my behemoth function in 15 minutes (aborted at about 15). Other than that it does seem a lot faster though

[JustasMasiulis]

Did more or less the last "optimization" possible.

• If it's link.exe that's taking ages I can't exactly help much more besides recommending to turn off LTCG.
• If it's cl.exe that's taking 15 minutes I can then optimize the template/constexpr shit a bit more but IIRC that is not the case.

[rraggerr]

Yes, now it's a lot more faster than before, i can compile 1000 lines for ~2 minutes and im ok with that. Thanks!

[JustasMasiulis]

I suppose this can be closed, as I don't think I can do much more to make it compile faster.

[rraggerr]

want to mention KN4CK3R's XorStr, it works perfectly with any project settings

[Flawww]

KN4CK3R's has the same issue on the really big functions though, albeit a bit quicker. I still had to disable optimizations for the functions in question or it would never compile in a timely manner. It is what it is, not an issue with the libraries themselves.