Unresolved external symbols when building with batch mode on Windows

ericmehl commented 1 year ago

Problem

Building with USE_BATCHED options on Windows fails due to unresolved external symbols.

Expected behavior: Successful build with USE_BATCHED option.

Actual behavior: There are quite a few unresolved symbols, here are a few representative ones when using USE_BATCHED=b8_AVX :

wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "public: class Imath_2_4::Color3<float> __cdecl OSL_v1_12::pvt::ColorSystem::to_rgb(class OpenImageIO_v2_4::ustring,class Imath_2_4::Color3<float> const &,class OSL_v1_12::ShadingContext *)const " (?to_rgb@ColorSystem@pvt@OSL_v1_12@@QEBA?AV?$Color3@M@Imath_2_4@@Vustring@OpenImageIO_v2_4@@AEBV45@PEAVShadingContext@3@@Z) referenced in function osl_b8_AVX_prepend_color_from_vs
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "public: class OSL_v1_12::Dual<class Imath_2_4::Color3<float>,2> __cdecl OSL_v1_12::pvt::ColorSystem::transformc(class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,class OSL_v1_12::Dual<class Imath_2_4::Color3<float>,2> const &,class OSL_v1_12::ShadingContext *)const " (?transformc@ColorSystem@pvt@OSL_v1_12@@QEBA?AV?$Dual@V?$Color3@M@Imath_2_4@@$01@3@Vustring@OpenImageIO_v2_4@@0AEBV43@PEAVShadingContext@3@@Z) referenced in function osl_b8_AVX_transform_color_vss
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "public: class Imath_2_4::Color3<float> __cdecl OSL_v1_12::pvt::ColorSystem::transformc(class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,class Imath_2_4::Color3<float> const &,class OSL_v1_12::ShadingContext *)const " (?transformc@ColorSystem@pvt@OSL_v1_12@@QEBA?AV?$Color3@M@Imath_2_4@@Vustring@OpenImageIO_v2_4@@0AEBV45@PEAVShadingContext@3@@Z) referenced in function osl_b8_AVX_transform_color_vss
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "public: class OSL_v1_12::Dual<class Imath_2_4::Color3<float>,2> __cdecl OSL_v1_12::pvt::ColorSystem::ocio_transform(class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,class OSL_v1_12::Dual<class Imath_2_4::Color3<float>,2> const &,class OSL_v1_12::ShadingContext *)const " (?ocio_transform@ColorSystem@pvt@OSL_v1_12@@QEBA?AV?$Dual@V?$Color3@M@Imath_2_4@@$01@3@Vustring@OpenImageIO_v2_4@@0AEBV43@PEAVShadingContext@3@@Z) referenced in function "public: void __cdecl OSL_v1_12::Mask<8>::foreach<0,8,class <lambda_10c3e66d8f149be7f137bbae132c7c2d> >(class <lambda_10c3e66d8f149be7f137bbae132c7c2d>)const " (??$foreach@$0A@$07V<lambda_10c3e66d8f149be7f137bbae132c7c2d>@@@?$Mask@$07@OSL_v1_12@@QEBAXV<lambda_10c3e66d8f149be7f137bbae132c7c2d>@@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "public: class Imath_2_4::Color3<float> __cdecl OSL_v1_12::pvt::ColorSystem::ocio_transform(class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,class Imath_2_4::Color3<float> const &,class OSL_v1_12::ShadingContext *)const " (?ocio_transform@ColorSystem@pvt@OSL_v1_12@@QEBA?AV?$Color3@M@Imath_2_4@@Vustring@OpenImageIO_v2_4@@0AEBV45@PEAVShadingContext@3@@Z) referenced in function "public: void __cdecl OSL_v1_12::Mask<8>::foreach<0,8,class <lambda_45cf92685dd24c045df80eeea74fdbeb> >(class <lambda_45cf92685dd24c045df80eeea74fdbeb>)const " (??$foreach@$0A@$07V<lambda_45cf92685dd24c045df80eeea74fdbeb>@@@?$Mask@$07@OSL_v1_12@@QEBAXV<lambda_45cf92685dd24c045df80eeea74fdbeb>@@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::rgb" (__imp_?rgb@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::RGB" (__imp_?RGB@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "public: 
void __cdecl OSL_v1_12::Mask<8>::foreach<0,8,class <lambda_45cf92685dd24c045df80eeea74fdbeb> >(class <lambda_45cf92685dd24c045df80eeea74fdbeb>)const " (??$foreach@$0A@$07V<lambda_45cf92685dd24c045df80eeea74fdbeb>@@@?$Mask@$07@OSL_v1_12@@QEBAXV<lambda_45cf92685dd24c045df80eeea74fdbeb>@@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::hsv" (__imp_?hsv@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::hsl" (__imp_?hsl@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::YIQ" (__imp_?YIQ@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::XYZ" (__imp_?XYZ@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::xyY" (__imp_?xyY@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::linear" (__imp_?linear@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)
wide_opcolor_b8_AVX.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) class OpenImageIO_v2_4::ustring const OSL_v1_12::Strings::sRGB" (__imp_?sRGB@Strings@OSL_v1_12@@3Vustring@OpenImageIO_v2_4@@B) referenced in function "void __cdecl OSL_v1_12::b8_AVX_pvt::`anonymous namespace'::wide_transformc<class Imath_2_4::Color3<float> >(class OSL_v1_12::pvt::ColorSystem,class OpenImageIO_v2_4::ustring,class OpenImageIO_v2_4::ustring,struct OSL_v1_12::Masked<class Imath_2_4::Color3<float>,8>,struct OSL_v1_12::Wide<class Imath_2_4::Color3<float> const ,8>,class OSL_v1_12::ShadingContext *)" (??$wide_transformc@V?$Color3@M@Imath_2_4@@@?A0x24e98508@b8_AVX_pvt@OSL_v1_12@@YAXVColorSystem@pvt@2@Vustring@OpenImageIO_v2_4@@1U?$Masked@V?$Color3@M@Imath_2_4@@$07@2@U?$Wide@$$CBV?$Color3@M@Imath_2_4@@$07@2@PEAVShadingContext@2@@Z)

Steps to Reproduce

Build with MSVC 2019
Add -D USED_BATCHED=b8_AVX build flag

Versions

OSL branch/version: 1.12.9.0
OS: Windows 10
C++ compiler: MSVC 2019
LLVM version: 11.1.0
OIIO version: 2.4.8.0

johnfea commented 2 months ago

I can reproduce the same or similar issue. This is with MSVC 2022 either with clang compiler supplied with it or with intel one api compiler.

liboslexec/wide/wide_opcolor.cpp:

__OSL_MASKED_OP2(blackbody, Wv, Wf)
..
OSL_OMP_PRAGMA(omp simd simdlen(__OSL_WIDTH))
..
cs.lookup_blackbody_rgb(temperature); <-- This function is unresolved symbol:
wide_opcolor_b8_AVX2.cpp.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) public: class Imath_3_1::Color3<float> __cdecl OSL_v1_14_1::pvt::ColorSystem::lookup_blackbody_rgb(float)const " (__imp_?lookup_blackbody_rgb@ColorSystem@pvt@OSL_v1_14_1@@QEBA?AV?$Color3@M@Imath_3_1@@M@Z) referenced in function osl_b8_AVX2_blackbody_vf

This function is inline function implemented in included opcolor_impl.h.

johnfea commented 2 months ago

It was a few different issues. Unnecessary dllimport function attribute was causing the issue I showed above and many similar issues. Then, functions definitions in headers followed by unused code that called those functions seemed to be generating unresolved external symbol errors. And then there were some functions like to_rgb() mentioned above which was called from wide/wide_opcolor.cpp and implemented in opcolor.cpp, but opcolor.cpp didn't seem to be compiled into the batched dll. Not sure why those only gave issues in MSVC environment, maybe they are just never called in any tests and would generate errors at runtime if they were. I just commented those out for now. I might make a patch for this at some point.

AcademySoftwareFoundation / OpenShadingLanguage