weidai11 / cryptopp

free C++ class library of cryptographic schemes
https://cryptopp.com
Other
4.66k stars 1.47k forks source link

An invalid parameter was passed to a function that considers invalid parameters fatal. Only when building in debug mode. #1243

Closed artvabasDev closed 8 months ago

artvabasDev commented 8 months ago

Like to report a debug build error, hopefully you can solve this, its a bit unhandy to develop an application when you're not able to build it for debugging,

Using cryptopp version 8.9 with Visual Studio 2022 in a MFC application.

When I build this piece of code in debug mode:

void SomeClass::EncryptString(CString& str) {

AutoSeededRandomPool rng;

// Generate private key RSA::PrivateKey privateKey; privateKey.GenerateRandomWithKeySize(rng, 2048); RSA::PublicKey publicKey(privateKey);

std::string plain = CW2A(str), cipher, recovered;

// Encryption RSAES_OAEP_SHA_Encryptor e(publicKey);

StringSource ss1(plain, true, new PK_EncryptorFilter(rng, e, new StringSink(cipher) ) // PK_EncryptorFilter ); // StringSource }

When executing line: privateKey.GenerateRandomWithKeySize(rng, 2048); I got the following debug message:

Unhandled exception at 0x00007FF73F1F31FC An invalid parameter was passed to a function that considers invalid parameters fatal.

And Visual Studio 2022 is showing:

extern "C" declspec(noreturn) void cdecl _invoke_watson( wchar_t const const expression, wchar_t const const function_name, wchar_t const* const file_name, unsigned int const line_number, uintptr_t const reserved ) { UNREFERENCED_PARAMETER(expression ); UNREFERENCED_PARAMETER(function_name); UNREFERENCED_PARAMETER(file_name ); UNREFERENCED_PARAMETER(line_number ); UNREFERENCED_PARAMETER(reserved );

    if (IsProcessorFeaturePresent(PF_FASTFAIL_AVAILABLE))
    {
        __fastfail(FAST_FAIL_INVALID_ARG);
    }

    // Otherwise, raise a fast-fail exception and termintae the process:
    __acrt_call_reportfault(
        _CRT_DEBUGGER_INVALIDPARAMETER,
        STATUS_INVALID_CRUNTIME_PARAMETER,
        EXCEPTION_NONCONTINUABLE);

    TerminateProcess(GetCurrentProcess(), STATUS_INVALID_CRUNTIME_PARAMETER);
}

And stop at: __fastfail(FAST_FAIL_INVALID_ARG);.

While the same piece of code when building in release mode is working just fine.

I use the debug version of cryptlib.lib for building in debug mode and the release version for building in release mode.

noloader commented 8 months ago

I can't duplicate with the following program built with -DDEBUG -g3 -O0. The library was also built with the same flags.

$ cat test.cxx 
#include "cryptlib.h"
#include "osrng.h"
#include "rsa.h"

int main(int argc, char* argv[])
{
  using namespace CryptoPP;

  AutoSeededRandomPool prng;
  RSA::PrivateKey privateKey;
  privateKey.GenerateRandomWithKeySize(prng, 2048);
  RSA::PublicKey publicKey(privateKey);

  return 0;
}

There's probably something wrong with your program.

artvabasDev commented 8 months ago

Still strange that the same code in release is working just fine and in debug not. Have a workaround now, in debug using plain text in release encrypted.

Thanks for you effort.

noloader commented 8 months ago

I experienced this issue about 8 or 10 years ago on OS X. If I recall correctly, there was a global static, and for some reason, it was triggering the unused parameter exception. I never figured out what was sideways.