Closed adamf88 closed 7 years ago
Thanks @adamf88.
Please forgive my ignorance... Its not clear to me what the problem is.
On Windows under MSVC compiler (_MSC_VER
defined), CRYPTOPP_NO_VTABLE
expands to __declspec(novtable)
. Under GCC, ICC, Comeau, and Clang, it expands to nothing. Also see config.h, line 330.
Would you be able to build a Debug build (-DDEBUG
) with no optimizations (-O0
), provide a minimal test program to exhibit the behavior, and provide a backtrace of the sore spot?
Other useful information would include where you got Crypto++, what version it is, and if you used config.recommend
in place of config.h
.
During compilation there is few warnings
I'd also be interested in hearing about the warnings. We've gotten very aggressive about squashing them. See, for example, cryptest.sh
, line 1590 or so.
To be precise for tests I used such tools:
I'm not very familiar with clang on Windows, I just wanted to do some first test of clang on windows with my project and then I noticed such crash.
Cryptlib and sample test project I compiled using Platform Toolset "LLMV-vs2010". Configuraton Debug Win32. I changed one CryptoPP project setting: Whole Program Optimization to "No Whole Program Optimization"
To compile the project with this llvm toolset I had to do one more modification: At the begining of config.h file I added:
#define CRYPTOPP_DISABLE_X86ASM
(otherwise I got some compilation errors like: "Error 56 error : register %rbx is only available in 64-bit mode gcm.cpp 601" )
I also checked the macro definitions. "LLMV-vs2010" toolset imitate Visual Studio. This toolset defines _MSC_VER as 1600 clang macro is not defined I suppose there are problems, because the library can't identify clang properly on Windows. And this toolset is not fully compatible with vc100. Finally using Platform Toolset = "LLMV-vs2010" CRYPTOPP_NO_VTABLE expands to __declspec(novtable) And clang doesn't implement this feature in the same way as Visual Studio. Of course I can change CRYPTOPP_NO_VTABLE to empty definition and then the program works properly.
My code to expose the problem:
#include <md5.h>
void computeMD5()
{
CryptoPP::MD5 md5;
printf("%d\n", md5.DigestSize());
}
int main()
{
computeMD5();
puts("test");
return 0;
}
When I run this code then I get error message: Unhandled exception at 0x001f1a34 in CryptoppTest.exe: 0xC0000005: Access violation reading location 0x00000048.
Generated assembly: (code around the error):
protected:
IteratedHashWithStaticTransform() {this->Init();}
001F1A04 mov dword ptr [ebp-24h],ecx
001F1A07 mov dword ptr [ebp-28h],eax
001F1A0A call CryptoPP::ClonableImpl<CryptoPP::Weak1::MD5,CryptoPP::AlgorithmImpl<CryptoPP::IteratedHash<unsigned int,CryptoPP::EnumToType<enum CryptoPP::ByteOrder,0>,64,CryptoPP::HashTransformation>,CryptoPP::Weak1::MD5> >::ClonableImpl<CryptoPP::Weak1::MD5,CryptoPP::AlgorithmImpl<CryptoPP::IteratedHash<unsigned int,CryptoPP::EnumToType<enum CryptoPP::ByteOrder,0>,64,CryptoPP::HashTransformation>,CryptoPP::Weak1::MD5> > (1F112Ch)
001F1A0F mov ecx,dword ptr [ebp-24h]
001F1A12 add ecx,60h
001F1A15 mov dword ptr [ebp-10h],0
001F1A1C mov dword ptr [ebp-2Ch],ecx
001F1A1F mov dword ptr [ebp-30h],eax
001F1A22 call CryptoPP::FixedSizeAlignedSecBlock<unsigned int,16,0>::FixedSizeAlignedSecBlock<unsigned int,16,0> (1F1136h)
001F1A27 mov dword ptr [ebp-34h],eax
001F1A2A jmp CryptoPP::IteratedHashWithStaticTransform<unsigned int,CryptoPP::EnumToType<enum CryptoPP::ByteOrder,0>,64,16,CryptoPP::Weak1::MD5,0,0>::IteratedHashWithStaticTransform<unsigned int,CryptoPP::EnumToType<enum CryptoPP::ByteOrder,0>,64,16,CryptoPP::Weak1::MD5,0,0>+5Fh (1F1A2Fh)
protected:
IteratedHashWithStaticTransform() {this->Init();}
001F1A2F mov eax,dword ptr [ebp-28h]
001F1A32 mov ecx,dword ptr [eax]
001F1A34 mov ecx,dword ptr [ecx+48h]
001F1A37 mov dword ptr [ebp-10h],1
001F1A3E mov dword ptr [ebp-38h],ecx
001F1A41 mov ecx,eax
001F1A43 mov edx,dword ptr [ebp-38h]
001F1A46 call edx
001F1A48 jmp CryptoPP::IteratedHashWithStaticTransform<unsigned int,CryptoPP::EnumToType<enum CryptoPP::ByteOrder,0>,64,16,CryptoPP::Weak1::MD5,0,0>::IteratedHashWithStaticTransform<unsigned int,CryptoPP::EnumToType<enum CryptoPP::ByteOrder,0>,64,16,CryptoPP::Weak1::MD5,0,0>+7Dh (1F1A4Dh)
The access violation comes from: 0x001F1A34.
I can post all the warnings, but I suppose the warnings I get are because there is no clang macro definition. Some warnings I received:
Warning 168 warning : function definition with pure-specifier is a Microsoft extension [-Wmicrosoft-pure-definition] algebra.h 155
Warning 17 warning : private field 'm_padding' is not used [-Wunused-private-field] basecode.h 103
Warning 14 warning : unused variable 'cryptopp_assert_46' [-Wunused-variable] algparam.h 46
Warning 11 warning : unused variable 'cryptopp_assert_1991' [-Wunused-variable] misc.h 1991
Man, what are they doing defining _MSC_VER
....
We'll try to support this configuration. According to Getting Started with the LLVM System using Microsoft Visual Studio, you do the following:
C:\..> clang hello.c -o hello.exe
Can you post the output of:
clang++ -dM -E < NUL
I've also got an open question on CFE-Users for detecting this configuration. See "How to detect LLVM Visual Studio Toolset on Windows?" But there's no guarantee we will get an answer.
I reported it as bug in LLVM: https://llvm.org/bugs/show_bug.cgi?id=26905, but it is probably bug in the library.
That's their modus operandi. They don't fully implement something, and then they claim its a bug in a program or library. The problem is they claim to be a Microsoft compiler (_MSC_VER
defined) but they don't consume the same programs MSVC can.
They did the same thing with the GNU C compiler, and it caused pain and misery across the web. See, for example, clang __GNUC__ bug and libgcrypt's Identify theft by Clang. It caused us to workaround at least six of their problems. The most painful was their inability to consume inline assembly with Intel syntax.
clang_3_8_defines.txt clang_trunk_defines.txt
There was also my fault during tests in previous post. (I will be more careful next time) Clang on Windows defines both macros: _MSCVER and __clang_\ . (I also double checked it with simple tests) So fortunately it should be possible to distinguish clang from vc++.
In attachment macro definitions returned by (clang 3.8 and clang trunk version) with command:
clang++ -dM -E
If I can do something more to help you (tests, reporting/fixing warnings), please let me know.
Clang on Windows defines both macros: MSC_VER and _clang . (I also double checked it with simple tests)
OK, good. Thanks for that.
It sounds like the bigger problem is Clang defining __MSC_VER
. That takes us into a number of code paths intended for Microsoft compilers. I fear making this change won't suffice:
331 #if defined(_MSC_VER) && !defined(__clang__)
332 #define CRYPTOPP_NO_VTABLE __declspec(novtable)
333 #else
334 #define CRYPTOPP_NO_VTABLE
335 #endif
Going back to your second comment:
To compile the project with this llvm toolset I had to do one more modification: At the begining of config.h file I added:
#define CRYPTOPP_DISABLE_X86ASM
So I'm clear... You had to... _One_:
// config.h around line 330
#if defined(_MSC_VER) && !defined(__clang__)
#define CRYPTOPP_NO_VTABLE __declspec(novtable)
#else
#define CRYPTOPP_NO_VTABLE
#endif
And _Two_:
// config.h around line 380
#if defined(_MSC_VER) && defined(__clang__)
# define CRYPTOPP_DISABLE_ASM
#endif
Is that correct?
If the two items above are correct, then I think the following patch should suffice at the moment. I just tested it against Linux and OS X, but I don't have the test rig for Clang on Windows.
Can you provide feedback on the patch below?
$ git diff config.* > config.patch
$ cat config.patch
diff --git a/config.h b/config.h
index 41103fe..3627e5f 100644
--- a/config.h
+++ b/config.h
@@ -328,7 +328,7 @@ NAMESPACE_END
#define CRYPTOPP_VC6_INT64
#endif
-#ifdef _MSC_VER
+#if defined(_MSC_VER) && !defined(__clang__)
#define CRYPTOPP_NO_VTABLE __declspec(novtable)
#else
#define CRYPTOPP_NO_VTABLE
@@ -378,7 +378,8 @@ NAMESPACE_END
#endif
// Apple's Clang prior to 5.0 cannot handle SSE2 (and Apple does not use LLVM Clang numbering...)
-#if defined(CRYPTOPP_APPLE_CLANG_VERSION) && (CRYPTOPP_APPLE_CLANG_VERSION < 50000)
+// Clang on Windows defines _MSC_VER but it cannot handle about half the ASM.
+#if (defined(CRYPTOPP_APPLE_CLANG_VERSION) && (CRYPTOPP_APPLE_CLANG_VERSION < 50000)) || (defined(_MSC_VER) && defined(__clang__))
# define CRYPTOPP_DISABLE_ASM
#endif
diff --git a/config.recommend b/config.recommend
index 5a237fd..4a84afb 100644
--- a/config.recommend
+++ b/config.recommend
@@ -328,7 +328,7 @@ NAMESPACE_END
#define CRYPTOPP_VC6_INT64
#endif
-#ifdef _MSC_VER
+#if defined(_MSC_VER) && !defined(__clang__)
#define CRYPTOPP_NO_VTABLE __declspec(novtable)
#else
#define CRYPTOPP_NO_VTABLE
@@ -378,7 +378,8 @@ NAMESPACE_END
#endif
// Apple's Clang prior to 5.0 cannot handle SSE2 (and Apple does not use LLVM Clang numbering...)
-#if defined(CRYPTOPP_APPLE_CLANG_VERSION) && (CRYPTOPP_APPLE_CLANG_VERSION < 50000)
+// Clang on Windows defines _MSC_VER but it cannot handle about half the ASM.
+#if (defined(CRYPTOPP_APPLE_CLANG_VERSION) && (CRYPTOPP_APPLE_CLANG_VERSION < 50000)) || (defined(_MSC_VER) && defined(__clang__))
# define CRYPTOPP_DISABLE_ASM
#endif
Uri and I spent a couple of weeks removing CRYPTOPP_DISABLE_ASM
for Clang under OS X and Linux. It became quite painful because there were effectively 3 different Clangs in play:
* Apple Clang on OS X
* LLVM Clang on OS X
* LLVM Clang on Linux
That's when those 6 or 8 Clang/LLVM bugs surfaced (and LLVM kept telling us it was our problem). I shudder to think there's another one to add to the mix :)
If I can get a test rig setup under Windows 7 with Visual Studio {2010|1012|2013}, then I should be able to bring back some of the ASM. I have Windows 7 and Visual Studio {2010|1012|2013}; I'm missing the Clang bits.
I did some tests (finally I was able to ran all tests using clang compiler):
This patch should be added (otherwise program crashes):
// config.h around line 330
#if defined(_MSC_VER) && !defined(__clang__)
#define CRYPTOPP_NO_VTABLE __declspec(novtable)
#else
#define CRYPTOPP_NO_VTABLE
#endif
Unfortunately it is not enough
// config.h around line 380
#if defined(_MSC_VER) && defined(__clang__)
# define CRYPTOPP_DISABLE_ASM
#endif
For now to properly compile crypto++ I had to add:
#if defined(_MSC_VER) && defined(__clang__)
# define CRYPTOPP_DISABLE_X86ASM
#endif
To remove a lot of warnings I also modified misc.h: Line 115 (CRYPTOPP_COMPILE_ASSERT_INSTANCE)
# if defined(__GNUC__) || defined(__clang__)
# define CRYPTOPP_COMPILE_ASSERT_INSTANCE(assertion, instance) \
static CompileAssert<(assertion)> \
CRYPTOPP_ASSERT_JOIN(cryptopp_assert_, instance) __attribute__ ((unused))
# else
I spend some time and tried to enable SSE2. Configuration Debug|Win32: First of all to remove error:
Error always_inline function '_mm_setzero_si128' requires target feature 'sse2', but would be inlined into function 'TrySSE2' that is compiled without support for 'sse2' cryptlib cpu.cpp 130
In project settings I enabled SSE2: Enable Enhanced Instruction Set: /arch:SSE2 Then my fix CRYPTOPP_DISABLE_X86ASM I changed to CRYPTOPP_DISABLE_ASM Finally I got error in clang compiler backend:
CL : fatal error : error in backend: assembler label 'L?TrySSE2@CryptoPP@@YA_NXZ$parent_frame_offset' can not be undefined
In configuration Debug|x64 in Visual Studio 2015 I received:
cryptlib.lib(integer.obj) : error LNK2001: unresolved external symbol __shiftleft128
Unfortunately it is not enough...
OK, it sounds like I am going to have to setup the test rig for this. Give me a few days on it.
Shooting from the hip, I suspect the pattern below is causing some troubles. We can easily fix CRYPTOPP_MSC_VERSION
in a single place config.h
, line 215:
#ifdef _MSC_VER
#define CRYPTOPP_MSC_VERSION (_MSC_VER)
#endif
But the islands of #ifdef _MSC_VER
is going to be a little more involved.
$ grep _MSC_VER *.h *.cpp | grep '#if' | grep -v 'CRYPTOPP_MSC_VERSION'
algparam.h:#if defined(_MSC_VER) && (_MSC_VER < 1300)
config.h:#if defined(_MSC_VER) || defined(__BORLANDC__)
config.h:#ifdef _MSC_VER
config.h:#if (defined(_MSC_VER) && (!defined(__INTEL_COMPILER) || __INTEL_COMPILER >= 1000) && (defined(_M_X64) || defined(_M_IA64))) || (defined(__DECCXX) && defined(__alpha__)) || (defined(__INTEL_COMPILER) && defined(__x86_64__)) || (defined(__SUNPRO_CC) && defined(__x86_64__))
config.h:#if defined(_MSC_VER)
config.h: #if _MSC_VER == 1200
config.h: #if _MSC_VER > 1200 || defined(_mm_free)
config.h:#if defined(_MSC_VER) || defined(__fastcall)
config.h:#if defined(_MSC_VER) && (_MSC_VER < 1300)
config.h:#if defined(_MSC_VER) && !defined(CRYPTOPP_MSVC6PP_OR_LATER)
config.h:#if defined(_MSC_VER) && !defined(__clang__)
config.h:#ifdef _MSC_VER
config.h:#if (defined(_MSC_VER) && _MSC_VER <= 1300) || defined(__MWERKS__) || (defined(_STLPORT_VERSION) && ((_STLPORT_VERSION < 0x450) || defined(_STLP_NO_UNCAUGHT_EXCEPT_SUPPORT)))
config.h:#if (defined(CRYPTOPP_APPLE_CLANG_VERSION) && (CRYPTOPP_APPLE_CLANG_VERSION < 50000)) || (defined(_MSC_VER) && defined(__clang__))
config.h:#if !defined(CRYPTOPP_DISABLE_ASM) && ((defined(_MSC_VER) && defined(_M_IX86)) || (defined(__GNUC__) && (defined(__i386__) || defined(__x86_64__))))
config.h: #if !defined(CRYPTOPP_DISABLE_SSSE3) && (_MSC_VER >= 1400 || CRYPTOPP_GCC_VERSION >= 40102 || defined(__SSSE3__) || defined(__SSE3__))
config.h:#if !defined(CRYPTOPP_DISABLE_ASM) && defined(_MSC_VER) && defined(_M_X64)
config.h:#if defined(_MSC_VER) && _MSC_VER >= 1300
config.h:#if (defined(_MSC_VER) && _MSC_VER <= 1300) || defined(__INTEL_COMPILER) || defined(__BORLANDC__)
config.h:#if (_MSC_VER >= 1600) || (__cplusplus >= 201103L)
dll.h:#if !(defined(_MSC_VER) && (_MSC_VER < 1300))
files.h:#if defined(CRYPTOPP_UNIX_AVAILABLE) || _MSC_VER >= 1400
files.h:#if defined(CRYPTOPP_UNIX_AVAILABLE) || _MSC_VER >= 1400
files.h:#if defined(CRYPTOPP_UNIX_AVAILABLE) || _MSC_VER >= 1400
misc.h:#ifdef _MSC_VER
misc.h: #if _MSC_VER >= 1400
misc.h:#if (_MSC_VER >= 1400)
misc.h:#if (_MSC_VER >= 1400 || defined(__GNUC__)) && (CRYPTOPP_BOOL_X64 || CRYPTOPP_BOOL_X86)
misc.h:#endif // #if (_MSC_VER >= 1400 || defined(__GNUC__)) && (CRYPTOPP_BOOL_X64 || CRYPTOPP_BOOL_X86)
misc.h:#ifdef _MSC_VER
misc.h:#endif // #ifdef _MSC_VER
misc.h:#if _MSC_VER >= 1300 && !defined(__INTEL_COMPILER)
misc.h:#endif // #if _MSC_VER >= 1310
misc.h:#if _MSC_VER >= 1400 && !defined(__INTEL_COMPILER)
misc.h:#endif // #if _MSC_VER >= 1400
osrng.h:#if defined(_MSC_VER)
pubkey.h:#if (defined(_MSC_VER) && _MSC_VER < 1300)
secblock.h:#if _MSC_VER >= 1500
stdcpp.h:#if _MSC_VER >= 1500
stdcpp.h:#if _MSC_VER >= 1600
stdcpp.h:#ifdef _MSC_VER
vmac.h:#if !(defined(_MSC_VER) && _MSC_VER < 1300) // can't use function template here with VC6
cpu.cpp:#if _MSC_VER >= 1400 && CRYPTOPP_BOOL_X64
des.cpp:#if (_MSC_VER >= 1600) || (__cplusplus >= 201103L)
dll.cpp:#if !(defined(_MSC_VER) && (_MSC_VER < 1300))
files.cpp:#if defined(CRYPTOPP_UNIX_AVAILABLE) || _MSC_VER >= 1400
files.cpp:#if _MSC_VER >= 1400
files.cpp:#if defined(CRYPTOPP_UNIX_AVAILABLE) || _MSC_VER >= 1400
fipstest.cpp:#if defined(_MSC_VER) && _MSC_VER >= 1400
fipstest.cpp:#if (_MSC_VER >= 1400 && !defined(_STLPORT_VERSION)) // ifstream doesn't support wide filename on other compilers
fipstest.cpp:#if defined(_MSC_VER) && _MSC_VER >= 1400
gcm.cpp:#if defined(_MSC_VER) && (_MSC_VER >= 1300 && _MSC_VER < 1400)
ida.cpp:#if defined(_MSC_VER) && !defined(__MWERKS__) && (_MSC_VER <= 1300)
integer.cpp:#if (_MSC_VER >= 1400) && !defined(_M_ARM)
integer.cpp: #ifdef _MSC_VER
integer.cpp: #if _MSC_VER >= 1400 && !defined(__INTEL_COMPILER) && !defined(_M_ARM)
integer.cpp:#if _MSC_VER < 1300
integer.cpp:#if _MSC_VER != 1200 || defined(NDEBUG)
misc.cpp:#if !(defined(_MSC_VER) && (_MSC_VER < 1300))
rijndael.cpp:#if (CRYPTOPP_BOOL_AESNI_INTRINSICS_AVAILABLE && (!defined(_MSC_VER) || _MSC_VER >= 1600 || CRYPTOPP_BOOL_X86 || CRYPTOPP_BOOL_X32))
rijndael.cpp:#if defined(_MSC_VER) && CRYPTOPP_BOOL_X86
sha.cpp:#if defined(_MSC_VER) && (_MSC_VER == 1200)
sha.cpp:#if defined(_MSC_VER) && (_MSC_VER == 1200)
sha.cpp: #if !defined(_MSC_VER) || (_MSC_VER < 1400)
sha.cpp: #if !defined(_MSC_VER) || (_MSC_VER < 1400)
sosemanuk.cpp:#ifndef _MSC_VER
sosemanuk.cpp:#ifdef _MSC_VER
test.cpp:#if (_MSC_VER >= 1000)
tiger.cpp: #if _MSC_VER < 1300
vmac.cpp:#if defined(_MSC_VER) && !CRYPTOPP_BOOL_SLOW_WORD64
vmac.cpp: #if _MSC_VER < 1300 || defined(__INTEL_COMPILER)
vmac.cpp: #if _MSC_VER >= 1400 && !defined(__INTEL_COMPILER) && !defined(_M_ARM)
vmac.cpp:#if !(defined(_MSC_VER) && _MSC_VER < 1300)
vmac.cpp:#if (defined(_MSC_VER) && _MSC_VER < 1300)
vmac.cpp:#if defined(_MSC_VER) && _MSC_VER < 1300
whrlpool.cpp: #if _MSC_VER < 1300
whrlpool.cpp:#if defined(__GNUC__) || (defined(_MSC_VER) && _MSC_VER < 1300)
zdeflate.cpp:#if defined(_STDEXT_BEGIN) && !(defined(_MSC_VER) && (_MSC_VER < 1400 || _MSC_VER >= 1600)) && !defined(_STLPORT_VERSION)
zdeflate.cpp:#if _MSC_VER >= 1600
zdeflate.cpp:#if defined(_MSC_VER) && !defined(__MWERKS__) && (_MSC_VER <= 1300)
I successfully compiled Cryptopp (master branch a21b777) under Windows with last Clang version.
Here's a summary of the change I made:
For the __shiftleft128 unresolved symbol problem, inside integer.cpp line 146:
- #ifdef _MSC_VER
+ #if defined(_MSC_VER) && !defined(__clang__)
For the __crt_debugger_hook unresolved symbol, inside fipstest.cpp line 31:
-#if defined(_MSC_VER) && _MSC_VER >= 1400
+#if defined(_MSC_VER) && _MSC_VER >= 1400 && !defined(__clang__)
and line 382:
-#if defined(_MSC_VER) && _MSC_VER >= 1400
+#if defined(_MSC_VER) && _MSC_VER >= 1400 && !defined(__clang__)
On Windows 7, for the "GetOverlappedResulEx not found" problem, seems like WINVER or/and _WIN32_WINNT macro contains the wrong Windows version in winpipes.cpp. I don't know why and where it is defined, so because I'm on Windows 7, I added the "-DWINVER=0x0601 -D_WIN32_WINNT=0x0601" flag based on the windows faq.
For the crash related to vtable, I modified config.h at line 353:
-#ifdef _MSC_VER
+#if defined(_MSC_VER) && !defined(__clang__)
Now, just run:
cmake -G "Ninja" -DCMAKE_C_COMPILER="E:/build-master/bin/clang-cl.exe" -DCMAKE_CXX_COMPILER="E:/build-master/bin/clang-cl.exe" -DCMAKE_C_FLAGS="-DWINVER=0x0601 -D_WIN32_WINNT=0x0601 -msse3 -msse4.1 -msse4.2 -mpclmul -maes /EHsc" -DCMAKE_CXX_FLAGS="-DWINVER=0x0601 -D_WIN32_WINNT=0x0601 -msse3 -msse4.1 -msse4.2 -mpclmul -maes /EHsc" -DCMAKE_MAKE_PROGRAM=E:/ninja ..
Then:
E:\ninja.exe
And finally:
cryptest.exe v
All tests passed except the 2 tests of the SymmetricCipher algorithm Sosemanuk.
@adamf88 - to help others, we are going to break the build when we run into Clang pretending to be VC++, and point people to the bug report:
// See http://github.com/weidai11/cryptopp/issues/147
#if defined(_MSC_VER) && defined(__clang__)
# error: "Unsupported configuration"
#endif
Ultimately, I want to recommend a Clang switch that tells Clang to "stop pretending to be other compilers". We have code paths for Clang, and it should be using them.
As an experiment, you might try:
// See http://github.com/weidai11/cryptopp/issues/147
#if defined(_MSC_VER) && defined(__clang__)
# warning: "Unsupported configuration; undefining _MSC_VER"
# undef _MSC_VER
#endif
For what its worth, I want to get back on this issue because its the sort of thing that interests me. But I'm having a hard time finding the spare cycles needed to spend on it.
@adamf88 - I'm not going to have time to work on this in the foreseeable future. Based on past experience with Clang specific support when the compiler pretended to be GCC by defining ___GNUC__
, I know this can be a major time sink. I don't want to risk a bottomless pit.
I'd prefer Clang top pretending to be other compilers and use the code paths we provide for it. We made a genuine effort to support both LLVM Clang and Apple Clang. Its kind of frustrating they break things the way they do with no option to disable the behavior.
I'm going to close the issue. Is there anything else we can do?
@noloader - For now I think the error is completely fine. I just made a test with clang compiler (for other people it might be more important in future).
Hello,
I've just hit this error when updating Crypto++ from 5.6.1 to 5.6.5 in my project.
I'd prefer Clang top pretending to be other compilers and use the code paths we provide for it.
I think there is some missing perspective here. Clang has a "cl.exe driver", clang-cl
, which emulates the behavior of Microsoft's cl.exe
, from command-line syntax to C++ syntax and extensions. The reason why clang-cl
defines _MSC_VER
is in order to support Microsoft's Visual C++ and platform headers. What's not obvious is that clang-cl
is available not just on Windows, but other platforms, too. This allows cross-compiling MFC applications from Linux to Windows, something which is not possible otherwise, and something I rely on in my project.
Another valid use case is using a modern compiler backend (Clang's LLVM) to target legacy platforms that are no longer supported by Microsoft's newer compilers. So, it is unfortunate that Crypto++ refuses to support this configuration, because there actually are valid use cases for it.
I will see if I can patch around this using the information in this thread, but it would be nice if it were officially supported, especially considering Microsoft's recent efforts of integrating Clang into Visual Studio.
Is it still an issue with latest clang ?
Is it still an issue with latest clang ?
I'm not sure, sorry.
If you want to try it, then remove the #error
around line 275.
We don't do anything predatory with Clang, like making changes that we know will break it. So you don't have to worry about us spiking the code. However, Clang does not consume programs like MSVC compilers do, so be careful.
You should be OK if the self tests pass.
Yes, it's still an issue with at least LLVM 11.0.0 using clang-cl for MSVC emulation. Defining CRYPTOPP_NO_VTABLE
to an empty value for this case still works (not sure why you didn't go with this solution), as does avoiding the virtual call in constructor altogether by applying something like
diff --git a/iterhash.h b/iterhash.h
index 5a65b5ba..1d77b4cd 100644
--- a/iterhash.h
+++ b/iterhash.h
@@ -191,7 +191,7 @@ public:
unsigned int DigestSize() const {return DIGESTSIZE;}
protected:
- IteratedHashWithStaticTransform() {this->Init();}
+ IteratedHashWithStaticTransform() {IteratedHashWithStaticTransform::Init();}
void HashEndianCorrectedBlock(const T_HashWordType *data) {T_Transform::Transform(this->m_state, data);}
void Init() {T_Transform::InitState(this->m_state);}
FYI, Clang doesn't really pretend to be MSVC these days (not sure about before) unless specifically asked to. In particular, for it to start pretending that it's MSVC, one needs to either run it via clang-cl frontend (e.g. setting CXX=clang-cl) or by passing it something from this set:
-fms-compatibility-version=<value>
Dot-separated value representing the Microsoft compiler version number to report in _MSC_VER (0 = don't define it (default))
-fms-compatibility Enable full Microsoft Visual C++ compatibility
-fms-extensions Accept some non-standard constructs supported by the Microsoft compiler
-fmsc-version=<value> Microsoft compiler version number to report in _MSC_VER (0 = don't define it (default))
Is there no workaround for this yet? I mean it's been 5 years.
I see there's #1039 opened recently, if by workaround you mean something that doesn't require patching.
Doesn't work for me! I thought, I could get away with only using AES, but just realized that AutoSeededRandomPool is also broken. I moved to LLVM to be able to write inline asm. So unlucky.
Any chance of this soon being properly fixed or should I consider other options?
Edit: SHA256 also crashes. Can't believe this.
Cheers
Thanks @mikedld.
I applied your patch at Commit 26a819d5c26c.
@noloader @mikedld Thank you guys so much.
I also had toruble using Crypto++ in a project built using clang 12.0 bundled with MSVC 2022 (_MSC_VER
= 1930). However, the problem seems to only be the hard stop in config_os.h
. I tested the master candidate version of Crypto++ as of today with the hard stop released and crypttest
built without issues (except for some warnigs). All tests passed.
As a result I would propose the following mod: https://github.com/jarzec/cryptopp/commit/6820f46a710799d40c9801868814ece08118684b
Hello @jarzec , i am implementing an encrytion code with c++, when i am trying to compile it in Visual Studio using llvm clang-cl i get "C++ #error directive: : config_os.h unsupported configuration" ; i tried modifying the config_os.h
as you suggested by adding _MSC_VER < 1930 &&
to line 32 but it doesn't seem to solve the issue, i am using visual studio 2019, any idea to bypass the issue? Do i need to compile the library it self using llvm clang?
few months passed and I'm still hitting this error. Does it mean that crypto++ doesn't work with msvc?
Hello @jarzec , i am implementing an encrytion code with c++, when i am trying to compile it in Visual Studio using llvm clang-cl i get "C++ #error directive: : config_os.h unsupported configuration" ; i tried modifying the
config_os.h
as you suggested by adding_MSC_VER < 1930 &&
to line 32 but it doesn't seem to solve the issue, i am using visual studio 2019, any idea to bypass the issue? Do i need to compile the library it self using llvm clang?
@SIvellofback I am sorry for the delayed answer, I must have overlooked the notification email about our question. I will respond in case anyone stumbled upon this. As I mentioned in my earlier post the version _MSC_VER
= 1930 corresponds to VS 2022, so the mod wouldn't work directly for VS 2019. You can find the exact value of _MSC_VER
for your exact version of VS e.g. here.
That being said you still need to check if for your particular version the modified condition is the only reason of failure. I have only tested with 2022 (I am not even sure which one exactly) and I haven't tested all the features, only the ones we actually use.
few months passed and I'm still hitting this error. Does it mean that crypto++ doesn't work with msvc?
@miro-ka I have not proposed my mod as PR because I don't have the time to find the minimum working _MSC_VER
nor to test robustly enough even for the version I've used. However, with the clarification above you can try your own version of the patch using _MSC_VER
of your VS and check if the functionality you need works.
I think we should stop using _MSC_VER
and start using CRYPTOPP_MSC_VERSION
instead. CRYPTOPP_MSC_VERSION
should work as the proper guard. Also see https://github.com/weidai11/cryptopp/blob/master/config_ver.h#L85 .
You should also file a bug report against Clang. It should not pretend to be other compilers when it cannot consume a program written for the other compiler.
Isn't the problem really that Crypto++ is depending on an implementation detail of the Microsoft compiler? Meaning, Microsoft may release a version of their compiler tomorrow that changes this behavior, breaking Crypto++, while still being completely and perfectly compliant with the language specification.
@CyberShadow ,
If Clang cannot handle Microsoft specific compiler extensions like __declspec(novtable)
, then Clang has no business defining _MSC_VER
.
OK, but does the documentation of __declspec(novtable)
describe the exact behavior that Crypto++ is relying on? Reading the bug report linked from the original post, the LLVM developer seems to imply that Crypto++ is using the extension not in a way that it was designed to be used.
@CyberShadow ,
If Clang cannot handle Microsoft specific compiler extensions like
__declspec(novtable)
, then Clang has no business defining_MSC_VER
.
Well, to be precise it is not clang that declares that but clang-cl - the MSVC clang "wrapper"/"adapter" that is shipped with VisualStudio.
@jarzec,
cl.exe
is the MSVC compiler. Or it used to be. Please post the output of cl.exe
.
Here's what I see from an older version of Visual Studio:
C:\Program Files (x86)\Microsoft Visual Studio 14.0>cl.exe
Microsoft (R) C/C++ Optimizing Compiler Version 19.00.23026 for x86
Copyright (C) Microsoft Corporation. All rights reserved.
And from the latest version of Visual Studio:
C:\Users\Jeffrey Walton\Desktop>cl.exe
Microsoft (R) C/C++ Optimizing Compiler Version 19.36.32535 for x86
Copyright (C) Microsoft Corporation. All rights reserved.
@jarzec,
Please compile and run this from a Developer Prompt:
#include <iostream>
// Compile with:
// g++ cryptopp-compiler.cpp -o cryptopp-compiler.exe
// Or:
// cl.exe /EHs cryptopp-compiler.cpp /Fe:cryptopp-compiler.exe
int main(int argc, char* argv[])
{
#if defined (_MSC_VER)
std::cout << "_MSC_VER is defined" << std::endl;
#else
std::cout << "_MSC_VER is not defined" << std::endl;
#endif
#if defined (__GNUC__)
std::cout << "__GNUC__ is defined" << std::endl;
#else
std::cout << "__GNUC__ is not defined" << std::endl;
#endif
#if defined (__clang__)
std::cout << "__clang__ is defined" << std::endl;
#else
std::cout << "__clang__ is not defined" << std::endl;
#endif
#if defined (__INTEL_COMPILER)
std::cout << "__INTEL_COMPILER is defined" << std::endl;
#else
std::cout << "__INTEL_COMPILER is not defined" << std::endl;
#endif
#if defined (__xlC__)
std::cout << "__xlC__ is defined" << std::endl;
#else
std::cout << "__xlC__ is not defined" << std::endl;
#endif
#if defined (__SUNPRO_CC)
std::cout << "__SUNPRO_CC is defined" << std::endl;
#else
std::cout << "__SUNPRO_CC is not defined" << std::endl;
#endif
return 0;
}
I just installed Clang via the SDK. clang-cl.exe
cannot compile the library. We don't even get to the runtime error. It dies during compilation.
Clang really needs to stop pretending to be other compilers.
Please file a bug with the LLVM project. This is not our problem.
> nmake /f cryptest.nmake CXX=clang-cl.exe
Microsoft (R) Program Maintenance Utility Version 14.36.32535.0
Copyright (C) Microsoft Corporation. All rights reserved.
******************************
Platform is x64
******************************
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /Yc"pch.h" /Fp"pch.pch" /c pch.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c cryptlib.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c cpu.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c integer.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c 3way.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c adler32.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c algebra.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c algparam.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c allocate.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c arc4.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c aria.cpp
clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c aria_simd.cpp
aria_simd.cpp(181,6): error: always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3', but would be
inlined into function 'ARIA_ProcessAndXorBlock_SSSE3' that is compiled without support for 'ssse3'
_mm_shuffle_epi8(_mm_load_si128(CONST_M128_CAST(rk)), MASK)))
^
aria_simd.cpp(188,5): error: always_inline function '_mm_shuffle_epi8' requires target feature 'ssse3', but would be
inlined into function 'ARIA_ProcessAndXorBlock_SSSE3' that is compiled without support for 'ssse3'
_mm_shuffle_epi8(_mm_load_si128(CONST_M128_CAST(rk)), MASK)));
^
2 errors generated.
NMAKE : fatal error U1077: 'clang-cl.exe /nologo /W4 /wd4231 /wd4511 /wd4156 /D_MBCS /Zi /TP /GR /EHsc /DNDEBUG /D_NDEBUG /Oi /Oy /O2 /MT /FI sdkddkver.h /FI winapifamily.h /c aria_simd.cpp' : return code '0x1'
Stop.
clang-cl is not the official compiler from LLVM.
https://blog.conan.io/2022/10/13/Different-flavors-Clang-compiler-Windows.html
No, clang-cl is a clang driver developed as part of the Clang/LLVM project. It's available on all platforms, not just Windows. See --driver-mode=cl
.
Clang really needs to stop pretending to be other compilers.
I am repeating myself but this is a very short-sighted take. Doing so will break many projects as the feature enables use cases that are impossible otherwise. You are missing perspective.
I've played a bit with clang-cl installed together with VS2022. I was able to compile cryptopp (so far static library only) with clang-cl after few fixes. cryptest passes all tests. It is very early Proof of concept and requires a lot further work.
You can check my changes here: https://github.com/adamf88/cryptopp/tree/a7c88748b2dd0e4f7c1149446a55d8dcf524b24c
My changes:
There is many warnings generated by clang (on Linux they are disabled) CRYPTOPP_GCC_DIAGNOSTIC_AVAILABLE is not defined so far. Compiling the project as dynamic library does not work yet. I fixed first clang error: explicit specialization of '' after instantiation. https://github.com/adamf88/cryptopp/commit/dfcb1a969a1c9e0b4b6bf70c4fdef7d5319e7a31
If you like this direction then I can continue work on that.
And a related bug report: https://github.com/llvm/llvm-project/issues/63492 .
@adamf88,
The problem is with CRYPTOPP_NO_VTABLE usage in IteratedHashWithStaticTransform class.
This should be cleared by tightening up our internal defines in config_ver.h (and a few related changes).
However, it does not fix the underlying problems with Clang. For example, Clang still cannot compile a program written for MSVC, like using __declspec(novtable)
and https://github.com/llvm/llvm-project/issues/63492 . The LLVM devs have to fix their problems.
@noloader I see the problem with novtable has already workaround applied: https://github.com/weidai11/cryptopp/commit/26a819d5c26c
The SIMD issue we can also workaround as you already moved SIMD code to separate cpp files with C API. Previously I have adjusted vcxproj only and I see you use nmake. Please see attached patch. It adjusts vcxproj and nmake to pass correct -m flags when compiling SIMD files. After applying this patch I'm able to run tests without issues.
Patch generated from: https://github.com/adamf88/cryptopp/tree/ClangOnWindows
nmake /f cryptest.nmake CXX=clang-cl.exe
cryptest.exe v
...
All tests passed!
Seed used was 1687900310
Test started at Tue Jun 27 23:11:50 2023
Test ended at Tue Jun 27 23:11:56 2023
Any updates on the situation?
just noticed that on windows with clang 15.0.7 on Windows 10 it gives this error ( noticed it in my Jenkins when I checked logs ) and compiles fine on linux with same clang version
And I do know that I use the cmake repo but I feel its because of what's listed in here
Windows 10 Pro
C:\Users\tanja>clang --version
clang version 15.0.7
Target: x86_64-pc-windows-msvc
Thread model: posix
InstalledDir: C:\Program Files\LLVM\bin
C:/Jenkins/workspace/Cpp-Tanja84dk-Crypt/build/_deps/cryptopp-cmake-build/cryptopp/config_os.h:34:3: error: : "Unsupported configuration"
# error: "Unsupported configuration"
^
1 error generated.
ninja: build stopped: subcommand failed.
Build step 'CMake Build' marked build as failure
Finished: FAILURE
console log is here Log
师傅,我也是这个报错,您解决了吗?
Would an extra preprocessor define to externally disable the #error be accepted? Something like:
// The problems with Clang pretending to be other compilers is
// discussed at http://github.com/weidai11/cryptopp/issues/147.
#if (defined(_MSC_VER) && defined(__clang__) && \
- !(defined( __clang_analyzer__)) && !defined(__INTEL_LLVM_COMPILER))
+ !(defined( __clang_analyzer__)) && !defined(__INTEL_LLVM_COMPILER) && \
+ !defined(CRYPTOPP_ENABLE_UNSUPPORTED_CONFIG))
# error: "Unsupported configuration"
#endif
It's annoying to have to manually edit the crytopp source in order to get it to compile.
I found the problem when I compiled the cryptopp on Windows using llvm toolchain. During compilation there is few warnings, but finally I was able to compile the library successfully. Unfortunately the program crashes. In my case during constructing md5 class. The problem is with CRYPTOPP_NO_VTABLE usage in IteratedHashWithStaticTransform class.
The function Init() in IteratedHashWithStaticTransform() is virtual (Defined as virtual in IteratedHashBase class).
I reported it as bug in LLVM: https://llvm.org/bugs/show_bug.cgi?id=26905 But it is probably bug in the library.