Problem in debugging Encoder in Visual Studio 2019

[aaroncuilu]

Hi

I tried to generate STEP format file directly from the rpk files. I did two tests with examples in CityEngine SDK 2.6.8300. The first one was successful and the second was failed.

First test: I compiled the prt4cmd and stlenc separately in release mode. After copy the new the STEP encoder to the prt4cmd's lib folder and modify the parameter to use the new encoder, prt4cmd can run successfully and generate correct STEP format file from the given rpk file.

Second test: I compiled the prt4cmd and stlenc in Debug mode separately and repeat the First test step, prt4cmd cannot run. The exception was as following: [2022-09-25 15:11:47] [error] prt::createEncoderInfo failed, exception caught: 'Encoder not found.' [2022-09-25 15:11:47] [error] Status code = 20 = Encoder not found.

It seems that prt4cmd cannot find the STEP encoder correctly.

To compile the program in Debug mode, I modify the following CMakeList.txt as following: set(CMAKE_BUILD_TYPE "Debug" CACHE STRING "Choose one of: Debug Release RelWithDebInfo MinSizeRel")

In order to figure out the problem, I merged the two projects into one project. The stlenc was put into a sub directory inside prt4cmd. And use Visual Studio 2019 to debug. It failed at following line. encoderInfoBuilder.setExtension(STL_EXT);

I am working on a pilot project to use CGA technology. I would like to know to how to debug the customized Encoder using Visual Studio 2019.

Thanks for the kind help.

Best regards Aaron

[mistafunk]

Hi & thanks for your interest in PRT.

I believe the problem lies in wrong compiler flags for "debug" mode which leads to C++ ABI incompatibility between the PRT core dll and your codec extension dll.

Please note that the default CMake compiler flags for CMAKE_BUILD_TYPE="Debug" are not compatible with the PRT release, i.e. you cannot generally mix dlls compiled with Release and Debug.

We recommend to use a "non-optimized release" build for your custom extension. This will provide exact debugger breakpoints while still beeing ABI compatible with the released PRT. In the following CMake snippet TGT is your custom extension target, but you can of course also set these flags in Visual Studio or similar IDE). These flags are in addition to the ones mentioned in the README:

target_compile_options(${TGT} PRIVATE /MD)           # runtime library: multi-threaded, dynamically linked
target_compile_options(${TGT} PRIVATE /Od)           # disable optimization
target_compile_options(${TGT} PRIVATE /Ob0)          # disable inlining
target_compile_options(${TGT} PRIVATE /GS-)          # disable buffer security checks
target_compile_definitions(${TGT} PRIVATE -DNDEBUG)  # do NOT use ABI-incompatible debug tools like iterator checks etc

Background information: PRT consists of two namespaces prt:: and prtx::. The first one uses a C-style API where you can mix different compiler configurations, e.g. Debug and Release. We call this a "compiler firewall". The second one (prtx) is the lower-level API you use to write e.g. custom encoders. Here the compiler flags need to match because we are passing C++ objects through the interface boundary and e.g. class memory layout needs to match. Otherwise you get erratic errors like you described.

PS: funnily enough, I just today entered a task in our backlog to improve our documentation with debug guidelines :-)

[aaroncuilu]

Hi Simon

Thanks a lot for you fast reply. My problem was solved as I followed your advice to modify those compile options. For Visual Studio 2019, I found the following link which works. https://learn.microsoft.com/en-us/cpp/build/how-to-debug-a-release-build?view=msvc-170

Best regards Aaron

[mistafunk]

That's great to hear.