Compiling custom python wrapping error.

LuletterSoul commented 3 years ago

First of all, thank you for your amazing contribution for video streaming and analysis ! I am building on Ubuntu 18.04 with Python 3.8.3 in a virtual environment. Everything seems to be ok before it throws an error /usr/bin/ld: cannot find -lpython3.8.

/home/ndww/Downloads/valkka-core/src/muxer.cpp: In destructor ‘virtual MuxFrameFilter::~MuxFrameFilter()’:
/home/ndww/Downloads/valkka-core/src/muxer.cpp:87:25: warning: ‘void av_free_packet(AVPacket*)’ is deprecated [-Wdeprecated-declarations]
     av_free_packet(avpkt);
                         ^
In file included from /home/ndww/Downloads/valkka-core/include/avdep.h:41:0,
                 from /home/ndww/Downloads/valkka-core/include/frame.h:42,
                 from /home/ndww/Downloads/valkka-core/include/framefilter.h:36,
                 from /home/ndww/Downloads/valkka-core/include/muxer.h:44,
                 from /home/ndww/Downloads/valkka-core/src/muxer.cpp:34:
/home/ndww/Downloads/valkka-core/ext/ffmpeg/libavcodec/avcodec.h:4650:6: note: declared here
 void av_free_packet(AVPacket *pkt);
      ^~~~~~~~~~~~~~
/home/ndww/Downloads/valkka-core/src/muxer.cpp: In member function ‘void MuxFrameFilter::initMux()’:
/home/ndww/Downloads/valkka-core/src/muxer.cpp:130:25: warning: ignoring return value of ‘int avformat_init_output(AVFormatContext*, AVDictionary**)’, declared with attribute warn_unused_result [-Wunused-result]
     avformat_init_output(av_format_context, &av_dict);
     ~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 19%] Building CXX object CMakeFiles/Valkka.dir/src/muxshmem.cpp.o
[ 20%] Building CXX object CMakeFiles/Valkka.dir/src/opengl.cpp.o
[ 21%] Building CXX object CMakeFiles/Valkka.dir/src/openglframefifo.cpp.o
[ 22%] Building CXX object CMakeFiles/Valkka.dir/src/openglobject.cpp.o
[ 23%] Building CXX object CMakeFiles/Valkka.dir/src/openglthread.cpp.o
[ 24%] Building CXX object CMakeFiles/Valkka.dir/src/rawrite.cpp.o
[ 25%] Building CXX object CMakeFiles/Valkka.dir/src/rgbframefifo.cpp.o
[ 26%] Building CXX object CMakeFiles/Valkka.dir/src/shader.cpp.o
[ 27%] Building CXX object CMakeFiles/Valkka.dir/src/sharedmem.cpp.o
[ 28%] Building CXX object CMakeFiles/Valkka.dir/src/testthread.cpp.o
[ 29%] Building CXX object CMakeFiles/Valkka.dir/src/tex.cpp.o
[ 30%] Building CXX object CMakeFiles/Valkka.dir/src/thread.cpp.o
[ 31%] Building CXX object CMakeFiles/Valkka.dir/src/tools.cpp.o
[ 32%] Building CXX object CMakeFiles/Valkka.dir/src/usbthread.cpp.o
[ 33%] Building CXX object CMakeFiles/Valkka.dir/src/valkkafs.cpp.o
[ 34%] Building CXX object CMakeFiles/Valkka.dir/src/valkkafsreader.cpp.o
[ 35%] Linking CXX shared library lib/libValkka.so
/usr/bin/ld: cannot find -lpython3.8
collect2: error: ld returned 1 exit status
CMakeFiles/Valkka.dir/build.make:952: recipe for target 'lib/libValkka.so.1.0.3' failed
make[3]: *** [lib/libValkka.so.1.0.3] Error 1
make[3]: Leaving directory '/home/ndww/Downloads/valkka-core/build_dir'
CMakeFiles/Makefile2:906: recipe for target 'CMakeFiles/Valkka.dir/all' failed
make[2]: *** [CMakeFiles/Valkka.dir/all] Error 2
make[2]: Leaving directory '/home/ndww/Downloads/valkka-core/build_dir'
Makefile:151: recipe for target 'all' failed
make[1]: *** [all] Error 2
make[1]: Leaving directory '/home/ndww/Downloads/valkka-core/build_dir'
debian/rules:29: recipe for target 'build' failed
make: *** [build] Error 2

Do I make a mistake? It will be very helpful if you can provide some guidelines.

elsampsa commented 3 years ago

This means it can't find python libraries:

/usr/bin/ld: cannot find -lpython3.8

Let's see..

Are you using the "easybuild.bash" command or not?

Anyway, if you take a look at here

https://github.com/elsampsa/valkka-core/blob/master/CMakeLists.txt#L147

When you run cmake in your (custom?) build directory, it uses the command "python3-config" to find the correct switches for compilation and linkin. So please tell us what is is the output of

python3-config --includes
python3-config --libs --embed

Since you are in ubuntu 18.04, whose default python version is 3.6, I assume you have compiled python yourself..?

Then, depending where your compiled python librares are siting, linuxes' shared library system may or may not find it.

So, please, locate where your custom-compiled python libraries are & make sure that your ubuntu can find them. Ubuntu searches for all shared libraries in a certain set of directories, you can find using this command:

ldconfig -v 2>/dev/null | grep -v ^$'\t'

To make it find more directories, you may play around with environmental variables (google is your friend), or add the directory to the global search list with:

sudo -i
echo "/PATH/TO/YOUR/DIRECTORY" > /etc/ld.so.conf.d/my_python.conf
ldconfig
exit

Hope this helps.

P. S. Ubuntu's standard python shared library is here:

./usr/lib/x86_64-linux-gnu/libpython3.6m.so

LuletterSoul commented 3 years ago

@elsampsa Thanks for your instant response and kind reply.

Yes, I really run easybuild.bash to complete build process.
I don't compiled python myself, but I create a virtual python environment with Conda. So, while the CMake executes python3-config, it will get corresponding configurations under this env. As you suggest, when I run:
```
python3-config --includes
python3-config --libs --embed
```

It shows:

  -I/home/ndww/anaconda3/envs/dol2/include/python3.8 -I/home/ndww/anaconda3/envs/dol2/include/python3.8

and:

  -lpython3.8 -lcrypt -lpthread -ldl  -lutil -lrt -lm -lm

When I run ldconfig -v 2>/dev/null | grep -v ^$'\t', it prints:

/usr/local/cuda-10.1/targets/x86_64-linux/lib:
/usr/lib/x86_64-linux-gnu/libfakeroot:
/lib/i386-linux-gnu:
/usr/lib/i386-linux-gnu:
/usr/local/lib:
/lib/x86_64-linux-gnu:
/usr/lib/x86_64-linux-gnu:
/lib:
/usr/lib:

What do you mean /PATH/TO/YOUR/DIRECTORY? Is it represents the python virtual env path ?

elsampsa commented 3 years ago

PATH/TO/YOUR/DIRECTORY would be the directory where you have the file python 3.8 shared object file: libpython3.8m.so

Could you please locate the whereabouts of the file libpython3.8m.so ? .. it must be somewhere there, under /home/ndww/anaconda3/envs/ maybe?

.. after that we can proceed.

I'm not an anaconda user myself..

LuletterSoul commented 3 years ago

@elsampsa Hi, thanks for you warm heart. I have solved this problem by install python3.8-dev successfully. The corresponding libpython3.8m.so file is install in /usr/lib/x86_64-linux-gnu, so Cmake could find this file finally.

But, I encountered the other problem, I will open the other issue for bug report.

elsampsa / valkka-core

Compiling custom python wrapping error. #21