AcademySoftwareFoundation / OpenShadingLanguage

Advanced shading language for production GI renderers
BSD 3-Clause "New" or "Revised" License
2.05k stars 348 forks source link

fix: Handle mixed string casting that plagues optix codegen #1718

Closed lgritz closed 11 months ago

lgritz commented 11 months ago

The string situation is still a bit tricky on the OptiX side. There's a full overhaul of that coming separately as we continue to shift to an all-ustringhash world.

But in the mean time, we were getting some testsuite failures that were because of a mismatch between times we think of a gpu-side ustring as a char* and when it's a uint64. IN FACT, they are always the hashes, but we backed ourselves into a corner where sometimes that 64 bit pattern is passed around as a pointer, ugh.

This small patch attempts to catch this case and just do the appropriate cast to compensate. It's a band-aid; a more comprehensive fix is coming, as I said.

This gets us down to just ONE failing optix test case, which I will tackle separately.

The change necessitated adding a new ptr_to_int64_cast() method to LLVM_Util.