search

RenderKit / openvkl

Intel(R) Open Volume Kernel Library
Apache License 2.0
200 stars 26 forks source link

openvkl tests cant find libraries on windows. #21

Closed semeraro closed 7 months ago

semeraro commented 1 year ago

I am trying to build and use openvkl on a windows system (forgive me father... ). I got the superbuild to complete but when I try to run one of the examples in the bin directory I get

C:\Users\Dave Semeraro>vklMinimal_03.exe [openvkl] Load of openvkl_module_cpu_device_4 failed due to: 'could not open module lib >openvkl_module_cpu_device_4: The specified module could not be found. ' [openvkl] Load of openvkl_module_cpu_device_8 failed due to: 'could not open module lib >openvkl_module_cpu_device_8: The specified module could not be found. ' [openvkl] Load of openvkl_module_cpu_device_16 failed due to: 'could not open module lib >openvkl_module_cpu_device_16: The specified module could not be found. ' [openvkl] INITIALIZATION ERROR: Could not find device of type: cpu_8. Make sure you have the >correct VKL libraries linked.

I have added the full path to the device dll files to my windows path environment but still no luck. Setting the Path works for the tbb and rkcommon dll files that are also needed by the openvkl examples but not for the cpu device itself. I am running on windows 11 with v msvc 2022 release. Any windows gurus out there?

This isnt really a bug but I didnt know where else to ask the question.

kraszkow commented 1 year ago

Hi @semeraro, which version of OpenVKL are you using (which branch/tag/commit sha)?

semeraro commented 1 year ago

I cloned the master branch of version 1.3.2 from github. Tag 15 I suppose. I just pulled it down two days ago. I suspect it has something to do with rkcommon which I believe handles loading of the dlls. I did the superbuild straight out of the box. Release build type and left the cmake settings as they were.

Dave

From: kraszkow @.> Sent: Tuesday, May 9, 2023 12:48 PM To: openvkl/openvkl @.> Cc: Dave Semeraro @.>; Mention @.> Subject: Re: [openvkl/openvkl] openvkl tests cant find libraries on windows. (Issue #21)

Hi @semerarohttps://github.com/semeraro, which version of OpenVKL are you using (which branch/tag/commit sha)?

— Reply to this email directly, view it on GitHubhttps://github.com/openvkl/openvkl/issues/21#issuecomment-1540607375, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACKHPJ7E27RLMO3W62DSGX3XFJ7PVANCNFSM6AAAAAAX3T4KRM. You are receiving this because you were mentioned.Message ID: @.***>

kraszkow commented 1 year ago

You're right about this "something to do with rkcommon which I believe handles loading of the dlls". Since latest release (1.3.2) OpenVKL core library (openvkl.dll) expects all module libraries (openvkl_module_cpu_device*.dll) to be in the same directory as core lib. You can't override this behavior anymore by using PATH env variable.

Is this something which makes your life harder? If so - could you better explain your use case?

semeraro commented 1 year ago

The device files and openvkl.dll are all placed in install/openvkl/bin. They are all in the same directory on my build. The openvkl_module_cpu_device dll files still dont load.

kraszkow commented 1 year ago

Could you try to do same experiment with prebuilt binaries form latest release: https://github.com/openvkl/openvkl/releases/download/v1.3.2/openvkl-1.3.2.x86_64.windows.zip ?

I want to understand if this is a problem connected to how you're building it or it's something else.

semeraro commented 1 year ago

Happy to help. Im betting its something I did in the build. Standby.

semeraro commented 1 year ago

The binary distro seems to work. All the vklMinimal# tests worked. vklTests seems to be hung testing max iterator size but it passed several tests before that.

kraszkow commented 1 year ago

Thanks Dave. So it's something with how you're building it. Could you attach here binaries (all dlls) which you've built? I will try to narrow down this issue.

semeraro commented 1 year ago

I can do that. There are lots of them including all the dependencies. Are you sure you want all of them or just the openvkl stuff?

kraszkow commented 1 year ago

openvkl*.dll + rkcommon.dll should be sufficient.

semeraro commented 1 year ago

how do I attach dll files. File type not supported errors.

kraszkow commented 1 year ago

zip them? and then follow: https://docs.github.com/en/get-started/writing-on-github/working-with-advanced-formatting/attaching-files

semeraro commented 1 year ago

openvkl_module_cpu_device_16.zip the openvkl bits.

semeraro commented 1 year ago

rkcommondll.zip and rkcommon

semeraro commented 1 year ago

thanks. I have cloned a fresh copy of rkcommon. Configure and built with no cmake changes. Failed all of the ctests.

kraszkow commented 1 year ago

Thanks, but I can't reproduce this problem. My steps:

1) Download & unzip https://github.com/openvkl/openvkl/releases/download/v1.3.2/openvkl-1.3.2.x86_64.windows.zip 2) Download & unzip & overwrite rkcommon.dll & openvkl*.dll 3) Run examples, tutorial etc. - all works fine on my side:

image

If above steps can give you same initial error then most likely only difference is that I'm using Win10 (vs win11 you're using).

semeraro commented 1 year ago

Interesting. Ill give that a try and see what happens on my side.

kraszkow commented 1 year ago

There is one tool which may help you debug this issue (when you have repro described initially in this issue): https://learn.microsoft.com/en-us/sysinternals/downloads/procmon

Then you should set filters: "Process Name" contains "vkl" + "Path" contains ".dll":

image

Then you'll be able to see from where it tries to look for those dll files.

semeraro commented 1 year ago

Thanks, I will check that out. I had been using dumpbin from the command line but this looks like a better option.

From: kraszkow @.> Sent: Tuesday, May 9, 2023 6:36 PM To: openvkl/openvkl @.> Cc: Dave Semeraro @.>; Mention @.> Subject: Re: [openvkl/openvkl] openvkl tests cant find libraries on windows. (Issue #21)

There is one tool which may help you debug this issue (when you have repro described initially in this issue): https://learn.microsoft.com/en-us/sysinternals/downloads/procmon

Then you should set filters: "Process Name" contains "vkl" + "Path" contains ".dll":

[image]https://user-images.githubusercontent.com/31285540/237242392-31feda6d-fdcb-4c49-8177-f8eee407bc66.png

Then you'll be able to see from where it tries to look for those dll files.

— Reply to this email directly, view it on GitHubhttps://github.com/openvkl/openvkl/issues/21#issuecomment-1541021226, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACKHPJ3VNVV4NXBD2OIDI4DXFLIJXANCNFSM6AAAAAAX3T4KRM. You are receiving this because you were mentioned.Message ID: @.***>

semeraro commented 1 year ago

I did some experiments with dll shuffling. I found that if I swapped the rkcommon dll that I built from source for the rkcommon dll that comes with the openvkl binary distribution for windows the openvkl tests in the binary distribution dont work. If I instead swap in my home built openvkl dlls for the openvkl dlls in the binary distro, the openvkl tests in the binary distribution continue to work. It seems I have done something wrong when building the rkcommon package.

kraszkow commented 1 year ago

Did you found solution? We recommend to build rkcommon using "superbuild", here you can see how it's done for older MSVC:

https://github.com/openvkl/openvkl/blob/v1.3.2/.github/workflows/ci.windows.yml#L23

semeraro commented 1 year ago

Yes, I was finally able to compile openvkl and all of the dependencies. I first removed all traces of openvkl and dependencies from my laptop. I had become so confused about which version of which was where that I had to remove it all. I then started with a fresh clone of openvkl 1.3.2. I proceeded with the superbuild. I did change a couple of things. I turned off parallel compile via the cmake settings in my vscode extensions. I have had problems with linux builds of things that trip over themselves when parallel builds are done. The other change I made was to the "default build command" in the superbuild cmakelists file. I changed the config from release to debug. I had been trying to make the change through the cmake settings but found some of the dependencies picked up on it and some did not. I wanted everything compiled debug in anticipation of having to do some tracing through everything. I left everything else alone. Configure and build went smoothly. And best of all the example applications worked fine. As has often been the case I cant say what it was I was doing wrong. I wish I could pinpoint it but I cant. Thanks for indulging me and for the help.

regards, Dave

From: kraszkow @.> Sent: Friday, May 12, 2023 8:11 AM To: openvkl/openvkl @.> Cc: Dave Semeraro @.>; Mention @.> Subject: Re: [openvkl/openvkl] openvkl tests cant find libraries on windows. (Issue #21)

Did you found solution? We recommend to build rkcommon using "superbuild", here you can see how it's done for older MSVC:

https://github.com/openvkl/openvkl/blob/v1.3.2/.github/workflows/ci.windows.yml#L23

— Reply to this email directly, view it on GitHubhttps://github.com/openvkl/openvkl/issues/21#issuecomment-1545724567, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACKHPJ7XHAN5SI5SI6UNPVTXFYZHXANCNFSM6AAAAAAX3T4KRM. You are receiving this because you were mentioned.Message ID: @.***>