Error when exectuting lightfieldPlayer

[emmanuelwilson]

Hello, I've built the 3DApps following the instructions provided without a problem within a Docker Container on Ubuntu, however when I go to execute lightfieldPlayer within the build folder using "./lightfieldPlayer" it gives me the following error:

./lightfieldPlayer: error while loading shared libraries: libimguiVarsd.so.1: cannot open shared object file: No such file or directory

Have you ever encountered this before? Is there a step that I am missing to add this library? I couldn't find any reference to it online.

[ichlubna]

Hi @emmanuelwilson maybe you can provide us with the output of ldd ./lightfieldPlayer | grep libimguiVarsd It should be located in the folder with the supporting libs, like: pathToTheLibs/install/linux/lib/libimguiVarsd.so.1

[emmanuelwilson]

Thank you for the reply! When I run the command I get the following: libimguiVarsd.so.1 => not found

However looking into the path that you mentioned I actually see the library libimguiVarsd.so.1 there! It is in path: /install/linux2/lib/

Is it due to the fact that it is in the linux2 directory? There isn't any other linux directory that was created in the building process.

[dormon]

Edit this file on your system: sudo gedit /etc/ld.so.conf add line containing the "......./install/linux2/lib" directory and then run: sudo ldconfig

[emmanuelwilson]

I am not very familiar with conf files, so just to be sure, what should it look like after I add directory? When I run the ldconfig command with or without the modifications to the ld.so.conf file I get the following message:

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-allocator.so.1 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so.1 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-compiler.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-allocator.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-opencl.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-opencl.so.1 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-cfg.so.1 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.1 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-cfg.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ml.so.1 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ml.so.470.86 is empty, not checked.

[ichlubna]

@emmanuelwilson This might help. You can also use the export version without editing any files. I am not sure why you are getting the output above. Does it work despite that? Note that our implementation uses GPU decoder so the Nvidia drivers along with a H265 decoding capable GPU are necessary. Sorry for the inconvenience. ^_^"

[emmanuelwilson]

Apologies for the delay, I've been trouble shooting things on my end. I've been able to reduce the errors by half, you were right that I was indeed missing the Nvidia drivers! However it is still giving me: /sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-compiler.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-allocator.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-opencl.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ptxjitcompiler.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libcuda.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-cfg.so.470.86 is empty, not checked.

/sbin/ldconfig.real: File /lib/x86_64-linux-gnu/libnvidia-ml.so.470.86 is empty, not checked.

and I have not been able to fix it. I believe that it may be an issue with the Docker image that I am using. So I have been working on building a new Dockerfile which will hopefully solve the issue. Namely the Nvidia base image for openGL will solve any cuda/GPU driver missmatch I may have. If you guys have any recommendations or tips for properly installing all the dependencies I am all ears! I appreciate both of your feedback!

[ichlubna]

I have no experience with Docker so I can't help you much with that, sorry. Installing the NV drivers should be enough. Not sure why you have those missing dependencies. It looks like the drivers are incomplete or something. Maybe there are some additional packages containing the OpenCL, Cuda etc.? Those things are not used in our LF app though. They are probably used in FFMPEG for the GPU support that we use as well.

[emmanuelwilson]

I'll also be trying to set this up outside of Docker to avoid any challenges around that. I believe that most of these problems arise when building the apps, so you might be onto something about CMake? If it is not too much to ask, would it be possible to share what drivers and libraries packages you guys use for this software? Otherwise just some key drivers that I may need, (as you mentioned, NV, openCL, FFMPEG, etc). I would like to replicate your environment as closely as I possibly can to get this running! Just in case I am missing something! Thanks again for your help.

On Tue, Feb 22, 2022 at 5:09 AM ichlubna @.***> wrote:

I have no experience with Docker so I can't help you much with that, sorry. Installing the NV drivers should be enough. Not sure why you have those missing dependencies. It looks like the drivers are incomplete or something. Maybe there are some additional packages containing the OpenCL, Cuda etc.? Those things are not used in our LF app though. Hmm maybe they are included by accident in CMake @dormon https://github.com/dormon?

— Reply to this email directly, view it on GitHub https://github.com/dormon/3DApps/issues/2#issuecomment-1047630973, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGAQGEV6F2OM5ZZYPWMDOYLU4NOHNANCNFSM5NDTNGXA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

You are receiving this because you were mentioned.Message ID: @.***>

[ichlubna]

share what drivers and libraries packages you guys use for this software?

I am using Arch Linux with:

nvidia 510.54-1
nvidia-sdk 9.0.20-1
nvidia-settings 510.54-1
nvidia-utils 510.54-1
opencl-nvidia 510.54-1
ffmpeg 2:4.4.1-1
cuda 11.6.0-1
cuda-tools 11.6.0-1

This might, however, be a problem with your Ubuntu packages which might be different.

[emmanuelwilson]

Hi guys! Been working away on my end to troubleshoot things, however I have been stuck with getting the right ffmpeg version. Everything else as far as I am aware matches what you have shown above. However when I try to downgrade ffmpeg or even try to build the earlier package from archive I get the following warning: warning: cannot resolve "libx264.so=163-64", a dependency of "ffmpeg" : : The follwing package cannot be upgraded due to unresolvable dependencies: ffmpeg

: : Do you want to skip the above package for this upgrade? [y/N] error: failed to prepare transaction (could not satisfy dependencies) : : unable to satisfy dependency 'libx264.so=163-64' required by ffmpeg

How did you guys go about getting your earlier version of ffmpeg? Have you guys run into this before? Thanks!

[ichlubna]

Sorry, I have noticed right now that there might be a problem during compilation which was not there a few weeks ago when I tested this. Maybe it's the new FFmpeg like you said. The current version 76af89f62bf708ea81f6caadca50beeb525c65e3 should have that fixed. (problem with av_find_best_stream?) Now I can compile that with ffmpeg 2:5.0-5.

[emmanuelwilson]

That fixed it! Thank you! Now it crashes when it is trying to build "stupidScheduler.cpp.o" The error it gives me is:

/git/3DApps/src/stupidScheduler.cpp: in member function 'virtual float OpenGLContext : :measure(const IsActive&, float) const' :
/git/3DApps/src/stupidScheduler.cpp:162:15 error: call of overloaded 'glFinish()' is ambiguous
  162 | glFinish();
         |
In file included from /opt/cuda/include/CL/cl.hpp:176,
                         from /git/3DApps/src/stupidScheduler.cpp:9:
/usr/include/GL/gl,h828:23: note: candidate: 'void glFinish()'
  828 | GLAPI void GLAPIENTRY glFinish( void );
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls.h:1822:22: note: candidate: 'void ge::gl::glFinish()'
  1822  | GEGL_EXPORT void glFinish();
            |
/git/3DApps/src/stupidScheduler.cpp:166:15: error: call of overloaded 'glFinish()' is ambiguous
166  |     glFinish();
        |
In file included from /opt/cuda/include/CL/cl.hpp:176,
                         from /git/3DApps/src/stupidScheduler.cpp:9:
/usr/include/GL/gl,h828:23: note: candidate: 'void glFinish()'
  828 | GLAPI void GLAPIENTRY glFinish( void );
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls.h:1822:22: note: candidate: 'void ge::gl::glFinish()'
  1822  | GEGL_EXPORT void glFinish();
            |
/git/3DApps/src/stupidScheduler.cpp:170:17: error: call of overloaded 'glFinish()' is ambiguous
170   |        glFinish();
         |
In file included from /opt/cuda/include/CL/cl.hpp:176,
                         from /git/3DApps/src/stupidScheduler.cpp:9:
/usr/include/GL/gl,h828:23: note: candidate: 'void glFinish()'
  828 | GLAPI void GLAPIENTRY glFinish( void );
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls.h:1822:22: note: candidate: 'void ge::gl::glFinish()'
  1822  | GEGL_EXPORT void glFinish();
            |
/git/3DApps/src/stupidScheduler.cpp:172:15: error: call of overloaded 'glFinish()' is ambiguous
172   |        glFinish();
         |
In file included from /opt/cuda/include/CL/cl.hpp:176,
                         from /git/3DApps/src/stupidScheduler.cpp:9:
/usr/include/GL/gl,h828:23: note: candidate: 'void glFinish()'
  828 | GLAPI void GLAPIENTRY glFinish( void );
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls.h:1822:22: note: candidate: 'void ge::gl::glFinish()'
  1822  | GEGL_EXPORT void glFinish();
            |
/git/3DApps/src/stupidScheduler.cpp: In member function 'virtual void OpenGLContext::printInfo() const':
/git/3DApps/src/stupidScheduler.cpp:186:66: error: call of overloaded 'glGetString(int)' is ambiguous
186  |   std::cout <<"GL_RENDERER          : " << glGetString(GL_RENDERER                 ) <<std::endl;
        |
In file included from /opt/cuda/include/CL/cl.hpp:176,
/usr/include/GL/gl,h826:34: note: candidate: 'const GLubyte* glGetString(GLenum)'
  826 | GLAPI const GLubyte * GLAPIENTRY glGetString( GLenum name);
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls..h.366:32 note: candidate: 'const GLubyte* ge::gl::glGetString(GLenum)'
366  |         GEGL_EXPORT const Glubyte* glGetString(GLenum name);
        |
/git/3DApps/src/stupidScheduler.cpp:187:66: error: call of overloaded 'glGetString(int)' is ambiguous
187   |            std::cout << "GL_VENDOR                                 : " <<glGetString(GL_VENDOR          ) << std::endl;
         |
In file included from /opt/cuda/include/CL/cl.hpp:176,
                         from /git/3DApps/src/stupidScheduler.cpp:9:
/usr/include/GL/gl,h826:34: note: candidate: 'const GLubyte* ge::gl::glGetString(GLenum)'
  826 | GLAPI const GLubyte* GLAPIENTRY glGetString(GLenum);
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls..h.366:32 note: candidate: 'const GLubyte* ge::gl::glGetString(GLenum)'
366  |         GEGL_EXPORT const Glubyte* glGetString(GLenum name);
        |
/git/3DApps/src/stupidScheduler.cpp:188:66: error: call of olverloaded 'glGetString(int)' is ambiguous
188   |        std::cout   <<"GL_VERSION                             : " << glGetString(GL_VERSION                  )<<std::endl;
         |
In file included from /opt/cuda/include/CL/cl.hpp:176,
                         from /git/3DApps/src/stupidScheduler.cpp:9:
/usr/include/GL/gl,h826:34: note: candidate: 'const GLubyte* ge::gl::glGetString(GLenum)'
  826 | GLAPI const GLubyte* GLAPIENTRY glGetString(GLenum);
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls..h.366:32 note: candidate: 'const GLubyte* ge::gl::glGetString(GLenum)'
366  |         GEGL_EXPORT const Glubyte* glGetString(GLenum name);
        |
/git/3DApps/src/stupidScheduler.cpp:189:66: error: call of olverloaded 'glGetString(int)' is ambiguous
189   |        std::cout   <<"GL_SHADING_LANGUAGE_VERSION                             : " << glGetString(GL_SHADING_LANGUAGE_VERSION                 )<<std::endl;
         |
In file included from /opt/cuda/include/CL/cl.hpp:176,
                         from /git/3DApps/src/stupidScheduler.cpp:9:
/usr/include/GL/gl,h826:34: note: candidate: 'const GLubyte* ge::gl::glGetString(GLenum)'
  826 | GLAPI const GLubyte* GLAPIENTRY glGetString(GLenum);
         |
In file included from /git/3DApps/src/stupidScheduler.cpp:3:
/install/linux/include/geGL/StaticCalls..h.366:32 note: candidate: 'const GLubyte* ge::gl::glGetString(GLenum)'
366  |         GEGL_EXPORT const Glubyte* glGetString(GLenum name);
        |
make[2]: *** [CMakeFiles/stupidScheduler.dir/build.make:76: CMakeFiles/studpidScheduler.dir/src/stupidScheduler.cpp.o] Error 1
make[1]: *** [CMakeFiles/MakeFile2:1091: CMakeFiles/stupidScheduler.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

Not sure how to interpret this! Any suggestions?

[dormon]

Well, you can disable applications, that you don't need. In cmake-gui, under section build, you can disable any application. BUILD_APP_stupidScheduler

[emmanuelwilson]

You guys have been very helpful! I appreciate your patience! The project has been built! Now I am running into problems when I run it however. When running the command: [emman@emman build]$ ./lightfieldPlayer I get the following error:

terminate called after throwing an instance of 'std::runtime_error' 
   what(): Cannot open the video file
Aborted (core dumped)

Followed by freezing the pc.

Am I doing something wrong? Is there a specific way to call this app (ie. feed it a video/input) ?

[ichlubna]

@emmanuelwilson Nice! Hmm the freezing is weird, that should not happen at all. Anyway, you can use standard -h argument to see the usage. You can, for example, run it as: ./lightfieldPlayer --video pavilion.mkv Here is one sample video file we use.

[emmanuelwilson]

Thank you for providing video! Ran into another problem which I hope you could help narrow down. When I run the command you provided above I get:

[emman@emman build]$ ./lightfieldPlayer --video pavilion.mkv
[hevc @ 0x7f11f0489400] Failed setup up for format vdpau: hwaccel initialisation returned error.
terminate called after trhowing and instance of 'std::runtime_error'
    what(): Cannot receive frame
Aborted (core dumped ) 

Have you dealt with this kind of problem before? There are some forums I have found that touch on this although most of them have varying solutions for similar problems.

I checked that FFMPEG does have the vdpau hardware acceleration option available by default and doesn't seem like I need to activate accelerated hardware manually via VLC. I don't think I am missing anything else at this point however there is always the possibility.

Based on this forum however: https://bbs.archlinux.org/viewtopic.php?id=251361 There is a possibility that my hardware is simply not up to the task. Although I would have thought that I have enough to run such a short video.
My hardware is: Processor: AMD Ryzen™ 7 1700X Processor 8 corres 3.4GHz GPU: Nvidia GeForce GTX 1060 6GB and 32 GB of RAM.

Looking at the vdpauinfo output I see that some decoder capabilities are not supported for:

VP9_PROFILE_1
VP9_PROFILE_2
VP9_PROFILE_3
HEVC_MAIN_STILL
HEVC_MAIN_444
HEVC_MAIN_444_10
HEVC_MAIN_444_12

Along with any of the HIGH QUALITY SCALING Video mixer features. Let me know if any of these decoders are essential for the process! If that is the case then I will have to look into swapping out some hardware. I can share more details but didn't want to swamp the posting. What do you guys think?

[ichlubna]

@emmanuelwilson Hmm...Well maybe you also need to install some vdpau drivers or something? But it seems to be OK. You can also try to encode the video in a different format if the one I provided is not supported. I think that the HW should be sufficient. Aaah but look at this. Seems like it is what you wrote above, the 444 profile might not be supported by your GPU. The video of mine is actually in that format. It doesn't matter, you can encode the video using ffmpeg into a different pixel format using, for example, -pix_fmt yuv420p. Hope this helps!

Unfortunately, the GPU video decoding is quite problematic and this was the only combination I made working. I plan to rewrite this in Vulkan but it will take some time.

[emmanuelwilson]

Great! Really appreciate the feedback! Wouldn't have made if far without either of your help! Just to be clear, the only way to get the decoding to work is with a GPU which supports the 444 profile correct? Its not just for the video you sent? Or more importantly, what format does this program require? For any input I give it do I need to make a video similar to what you shared or can it take a series of pictures similar to that of the Standford Camera array dataset? Are there any instructions or guide I should know about so I don't have to constantly take up your time? I am eager to put my own camera array to good use I am sure you can tell!

[ichlubna]

Hah I am sorry for the whole process being so cumbersome. This code was used mainly for experiments and measurements so it's not a final user-friendly product. You can also use different profiles on your GPU. For example on my RTX 2070 I can decode 444 but also 420. Basically, all the formats that your GPU can decode should work. If I convert the provided video like this: ffmpeg -i pavillion.mkv -c:v libx265 -pix_fmt yuv420p test.mkv I can play it as well like: ./lightfieldPlayer --video test.mkv Yea, this app supports only video input. It's just the LF rows encoded sequentially as a video. You can take a look at the provided video and its frames using: ffmpeg -i pavillion.mkv test/%04d.png and also encode your own LF dataset, like that one from Stanford (if you name it like 0000.png, 0001.png...): ffmpeg -i %04d.png -c:v libx265 -pix_fmt yuv420p neverEndingStory.mkv You can also specify the size of the grid with --grid parameter in our app. Hope this helps!

[emmanuelwilson]

This is incredibly helpful thank you!!! The problems I am running into now may be hardware related, however whenever I open the app, whether its to run the reformatted video or even when looking at the "-h" help option, I am no longer able to type anything. The screen is normal and my terminal typing square still flashes and the mouse is still able to move around. There isn't any message or error produced, it just sits there taunting me. Any reason for this you think? As a follow up to this, what should I expect for the output of the app? I'm thinking perhaps the app is doing what it is supposed to but maybe I don't have the right capacity to actually see it? Or maybe it is taking its time to process (which considering the speed in your paper I doubt it)

[ichlubna]

Hmmm that is weird. The output is an OpenGL windows with the lightfield view. To be honest, I am not sure about this problem. Maybe it's something with the Docker you use? The -h should not do anything else, just print the help and exit (try --help also just in case...).