Open stef opened 5 years ago
Hi @stef,
I'm trying to package gr-gsm for alpine linux,
Cool. Where I can find your pacakges?
and i had great success previously against gr3.7, but now that gr3.8 is out it seems to me that gr-gsm is not yet ready for this. i get the following errors during cmake:
-- The CXX compiler identification is GNU 8.3.0 -- The C compiler identification is GNU 8.3.0 -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Detecting CXX compile features -- Detecting CXX compile features - done -- Check for working C compiler: /usr/bin/gcc -- Check for working C compiler: /usr/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Detecting C compile features -- Detecting C compile features - done -- Build type not specified: defaulting to release. -- Found Git: /usr/bin/git (found version "2.22.0") -- Found Boost: /usr/include (found suitable version "1.69.0", minimum required is "1.35") found components: filesystem system -- -- Checking for module SWIG -- Found SWIG version 3.0.12. -- Found SWIG: /usr/bin/swig -- Found LOG4CPP: /usr/lib/liblog4cpp.so -- Found PkgConfig: /usr/bin/pkg-config (found version "1.6.1") -- Checking for module 'gmp' -- Package 'gmp', required by 'virtual:world', not found -- Found GMP: /usr/lib/libgmpxx.so -- Checking for module 'mpir >= 3.0' -- Package 'mpir', required by 'virtual:world', not found -- Could NOT find MPIR (missing: MPIRXX_LIBRARY MPIR_LIBRARY MPIR_INCLUDE_DIR) -- Found MPLIB: /usr/lib/libgmpxx.so -- Looking for pthread.h -- Looking for pthread.h - found -- Performing Test CMAKE_HAVE_LIBC_PTHREAD -- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success -- Found Threads: TRUE -- Found Boost: /usr/include (found suitable version "1.69.0", minimum required is "1.69.0") found components: date_time program_options filesystem system regex thread unit_test_framework chrono atomic -- Checking for module 'volk' -- Found volk, version 2.0 -- Found VOLK: /usr/lib/libvolk.so CMake Warning at CMakeLists.txt:135 (find_package): Found package configuration file: /usr/lib/cmake/gnuradio/GnuradioConfig.cmake but it set Gnuradio_FOUND to FALSE so package "Gnuradio" is considered to be NOT FOUND. Reason given by package: Gnuradio could not be found because dependency Volk could not be found. -- Checking for module 'cppunit' -- Found cppunit, version 1.14.0 -- Found CPPUNIT: /usr/lib/libcppunit.so;dl -- Checking for module 'libosmocore' -- Found libosmocore, version UNKNOWN-dirty -- Found libosmocore: /usr/lib/libosmocore.so -- Checking for module 'libosmocodec' -- Found libosmocodec, version UNKNOWN-dirty -- Found libosmocodec: /usr/lib/libosmocodec.so -- Checking for module 'libosmocoding' -- Found libosmocoding, version UNKNOWN-dirty -- Found libosmocoding: /usr/lib/libosmocoding.so -- Checking for module 'libosmogsm' -- Found libosmogsm, version UNKNOWN-dirty -- Found libosmogsm: /usr/lib/libosmogsm.so CMake Error at CMakeLists.txt:150 (message): GnuRadio Runtime required to compile gr-gsm -- Configuring incomplete, errors occurred!
gnuradio-3.8 and it's dev environment is installed, as is libvolk-2.0 and also it's headers, as can be seen from the cmake later finding volk.
We are testing gr-gsm
with Docker containers in TravisCI and we are using the gnuradio packages that comes from the distributions (Debian/Ubuntu/Kali and Fedora).
Could you provide a Dockerfile for Alpine for both gnuradio 3.7 and 3.8 ?
do you guys have an idea what might go wrong here if i'm messing up something, or if indeed there is no gr3.8 support yet, then what is your roadmap?
I doubt that we have a roadmap :smile:. I guess that no one has spent any time testing and adding support for gnuradio 3.8 as it is still quite new, it has been release 3 day ago.
Cool. Where I can find your pacakges?
https://github.com/aports-ugly/aports/tree/master/ugly/gr-gsm these are APKBUILD files only, you have to build them yourself, with this repo we currently do not provide binary builds.
We are testing gr-gsm with Docker containers in TravisCI and we are using the gnuradio packages that comes from the distributions (Debian/Ubuntu/Kali and Fedora). Could you provide a Dockerfile for Alpine for both gnuradio 3.7 and 3.8 ?
sorry, i don't do docker. but alpine runs well in docker i heard so it should be possible based on these packages to set up something like you ask for. gnuradio 3.7 and 3.8 both work fine though. if really needed i can provide some packages, but since they require also boost it is quite a hassle to also provide the right boost packages and other binary deps, it really is better to build these yourself - it is quite easy an abuild -r
will do this for you. but i guess that's out of question for you guys. i guess i have to wait until one of the mentioned distros package gnuradio so you can start porting. that's alright.
I doubt that we have a roadmap smile. I guess that no one has spent any time testing and adding support for gnuradio 3.8 as it is still quite new, it has been release 3 day ago.
true, but release candidates were also available also before that.
Cool. Where I can find your pacakges?
https://github.com/aports-ugly/aports/tree/master/ugly/gr-gsm
Thanks.
these are APKBUILD files only, you have to build them yourself, with this repo we currently do not provide binary builds.
OK.
We are testing gr-gsm with Docker containers in TravisCI and we are using the gnuradio packages that comes from the distributions (Debian/Ubuntu/Kali and Fedora). Could you provide a Dockerfile for Alpine for both gnuradio 3.7 and 3.8 ?
sorry, i don't do docker. but alpine runs well in docker i heard so it should be possible based on these packages to set up something like you ask for. gnuradio 3.7 and 3.8 both work fine though.
I'm not familiar with Alpine Linux and its tools and ecosystem.
if really needed i can provide some packages, but since they require also boost it is quite a hassle to also provide the right boost packages and other binary deps,
I'm confused, you are talking about packaging gr-gsm
and then you are saying you can't provide packages.
it really is better to build these yourself - it is quite easy an
abuild -r
will do this for you.
Could you give us all commands that have to be executed to install gr-gsm
together withgnuradio 3.8
, starting from a minimal VM installation?
but i guess that's out of question for you guys.
Nothing is out of question but without knowing how to reproduce your problem it's unlikely that we could help you resolve it.
i guess i have to wait until one of the mentioned distros package gnuradio so you can start porting. that's alright.
Instead of waiting someone else to to the work maybe you could try implementing the needed changes and contribute them back.
I doubt that we have a roadmap smile. I guess that no one has spent any time testing and adding support for gnuradio 3.8 as it is still quite new, it has been release 3 day ago.
true, but release candidates were also available also before that.
True, but unless someone is interested enough and willing to spend the needed time it's unlikely to happen.
ohi,
if really needed i can provide some packages, but since they require also boost it is quite a hassle to also provide the right boost packages and other binary deps,
I'm confused, you are talking about packaging gr-gsm and then you are saying you can't provide packages.
yeah, sorry. i can provide packages for gnuradio, but those have to match the correct boost libs, which come from the main/edge alpine repo which is a quite fast moving target. so it is always best to build gnuradio against whatever is installed from main/edge. it's a bit of a hassle.
it really is better to build these yourself - it is quite easy an abuild -r will do this for you.
Could you give us all commands that have to be executed to install gr-gsm together with gnuradio 3.8, starting from a minimal VM installation?
yes, of course. i stripped down my build vm for you (sadly it's still 2.8GB expanded): http://links.ctrlc.hu/mirror/gr-gsm-dbg.qcow2.tar.zst - it's a sparse tar file compressed with zstd. you can login with an empty password as user s. I put a very simple "script" in /home/s/gr-gsm-dbg. the script installs all the dependencies (you need internet connectivity for that), then unpacks and prepares gr-gsm-0.42.2 and drops you of in the root directory of gr-gsm sources. from here you don't need to use abuild or other alpine specific tools, just the usual cmake/make stuff you would do on other systems, if you succeed in building against gr3.8 we have a win.
tell me when you downloaded the vm image so i can free up the space on the server. thanks so much!
tell me when you downloaded the vm image so i can free up the space on the server.
Done, you could remove it.
I just found GNU Radio 3.8 OOT Module Porting Guide
It turns out they have implemented a lot of changes to their CMake build system that are not backward compatible and recommend the Out Of Tree (OOT) modules to adopt a branching model similar to theirs - master
, maint-3.8
, maint-3.7
branches compatible with gnuradio master
, 3.8
and 3.7
versions.
@ptrkrysik What do you think about adopting such a branching model?
They have also replaced XML with YAML
Hi @stef,
I just pushed a new branch maint-3.8
to my fork. It builds successfully in your VM image and make test
passes after adding /usr/lib/python3.7/dist-packages/
to PYTHONPATH
(ATM not sure why this is needed) and installing some additional packages. I haven't tested in with an SDR devices or any cfiles yet. For the record here are the commands from the history file.
sudo apk add py3-yaml
sudo apk add py3-gobject3
sudo apk add py3-setuptools
sudo apk add py3-click
sudo apk add py-click-plugins
sudo apk add py3-docutils
sudo apk add talloc-dev
export PYTHONPATH=/usr/lib/python3.7/dist-packages/
TODO:
grcc
grgsm_livemon.grc
and grgsm_livemon_headless.grc
to YAML format.I got the maint-3.8 branch of yours and built it on FreeBSD, grgsm_decode and grgsm_scanner works fine but grgsm_livemon not working i guess due to blocks being in xml format instead of yml .
this might help in converting the xml to yaml: https://gist.github.com/haakov/4228ff6a14486641add538483093e86b
Got them converted but now it says,
Traceback (most recent call last): File "/root/Downloads/gr-gsm-maint-3.8/apps/grgsm_livemon.py", line 44, in <module> class grgsm_livemon(gr.top_block, Qt.QWidget): File "/root/Downloads/gr-gsm-maint-3.8/apps/grgsm_livemon.py", line 46, in grgsm_livemon def __init__(self, args="", collector=localhost, collectorport=4729, fc=941.8e6, gain=30, osr=4, ppm=0, samp_rate=2000000.052982, serverport=4729, shiftoff=400e3): NameError: name 'localhost' is not defined
Conversion for gsm_block_tree.xml and gsm_block_tree.block.yml failed respectively with the messages below
Traceback (most recent call last):
File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/lib/python2.7/site-packages/gnuradio/grc/converter/cmdline_converter.py", line 32, in <module>
input = block.from_xml(sys.argv[-1])
File "grc/converter/block.py", line 43, in from_xml
data = convert_block_xml(element)
File "grc/converter/block.py", line 100, in convert_block_xml
('${ value }' if block_id.startswith('variable') else no_value)
AttributeError: 'NoneType' object has no attribute 'startswith'
Traceback (most recent call last):
File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/usr/local/lib/python2.7/site-packages/gnuradio/grc/converter/cmdline_converter.py", line 32, in <module>
input = block.from_xml(sys.argv[-1])
File "grc/converter/block.py", line 40, in from_xml
element, version_info = xml.load(filename, 'block.dtd')
File "grc/converter/xml.py", line 52, in load_lxml
raise ValueError("Failed to parse or validate {}".format(filename))
ValueError: Failed to parse or validate /usr/local/share/gnuradio/grc/blocks/gsm_block_tree.block.yml
i confirm, your maint-3.8 branch in your fork builds fine! great work, thanks for your kind support, APKBUILDs for alpine will soon be available.
Hi @ptrkrysik,
While you are at the CCCamp 2019 event, would it be it be possible to discuss the gnuradio 3.8 support in gr-osmosdr and gr-gsm with the other Osmocom developers and specifically the development/branching model described in https://wiki.gnuradio.org/index.php/GNU_Radio_3.8_OOT_Module_Porting_Guide?
The gr-osmosdr issue is https://osmocom.org/issues/3855
Thanks.
Hi @velichkov and @stef,
I exchanged few words with @noc0lour from GNU Radio and his suggestion is to use master branch for GNU Radio 3.8 and use separate branch for 3.7 compatible stuff.
I'm thinking what will be consequences.
For packaging for distributions that have 3.7 probably the maintainer could switch branch that he fetches the code from to the 3.7 branch. For the regular user who builds gr-gsm from source it might mean slightly more complicated installation description.
3.8 compatible gr-gsm will be probably available to people who build GNU Radio from source (or use distribution that already have GNU Radio 3.8 like alpine linux).
I don't see at the moment possibility to have a single branch that supports both versions of GNU Radio - it seems overly complicated.
Regarding gr-osmosdr - I'll ask Dimitri Stolnikov (horiz0n) if he has interest in taking care of gr-osmosdr migration.
Hi @ptrkrysik,
I talked exchanged few words with @noc0lour from GNU Radio and his suggestion is to use master branch for GNU Radio 3.8 and use separate branch for 3.7 compatible stuff.
Thanks.
I'm thinking what will be consequences.
For packaging for distributions that have 3.7 probably the maintainer could switch branch that he fetches the code from to the 3.7 branch. For the regular user who builds gr-gsm from source it might mean slightly more complicated installation description.
Yes, it's going to complicate a little installation from source for regular users that are not familiar with git or gnuradio. We are going to update the documentation but for sure there will be a lot of questions and github issues.
3.8 compatible gr-gsm will be probably available to people who build GNU Radio from source (or use distribution that already have GNU Radio 3.8 like alpine linux).
And installations with pybombs
.
I don't see at the moment possibility to have a single branch that supports both versions of GNU Radio - it seems overly complicated.
I agree.
Also the development of new features is going to be affected (at least initially) as we will need to push the changes to two branches which means more commits, more reviews in gerrit, more testing, etc...
Regarding gr-osmosdr - I'll ask Dimitri Stolnikov (horiz0n) if he has interest in taking care of gr-osmosdr migration.
Thanks.
Regarding gr-osmosdr - possibility of having a supported version for GNU Radio 3.8 is low. Most probably we'll have to switch to Soapy, but I never tried it yet and don't know how well/bad it might work.
Hi Everyone,
I just force pushed several changes to my maint-3.8
branch (the old version is in maint-3.8-backup
branch). I successfully built gr-gsm
with gnuradio 3.8 installed using pybombs on Fedora 29 using the receipts from maint-3.8
branch of my fork as well as in Debian Testing container with gnuradio installed from the apt repository and manually installing gr-iqbal
and gr-osmosdr
packages from maint-3.8
branches of my fork (See https://github.com/velichkov/gr-gsm/commit/bd2b81f1433e743e0217ee8cb7d94309344406d3)
When testing locally all unit tests, decode.sh and decrypt.sh succeed, both in container and on the host. I don't know why but decode.sh and decrypt.sh fail somehow randomly in travis (https://travis-ci.org/velichkov/gr-gsm/jobs/579927694, https://travis-ci.org/velichkov/gr-gsm/jobs/579927695), it seems like some random non-deterministic behavior despite the fact that "GSM Clock Offset Control" block is disabled by not specifying a frequency or arfcn in those two scripts.
I've also tested grgsm_livemon
, grgsm_livemon_headless
and grgsm_capture
with usrp and rtl devices and they seems to mostly work.
One incompatible change that I had to do is 013d4c258c6ad31c2581f0caa4eee3aa609fd9de. I've tried many different versions but it always fails when a python function is used in this array, it may be a bug in gnuradio, hope to resolve this in the future as well.
Any testing (especially with different SDR devices) and feedback is appreciated.
That's all for now. Cheers.
Hi, thanks for your work! I tried your recipes and it seems it works fine, but then pybombs tried to install package qwt5... This action fails on my Machine... it says:
PyBOMBS.install_manager - INFO - Installing package: qwt5
01372 kB / 01372 kB (100%)
Configuring: (100%) [=====================================================================================================================]
Building: (100%) [=====================================================================================================================]
cd src/ && ( test -e Makefile || /usr/lib/qt5/bin/qmake -o Makefile /root/prefix/default/src/qwt5/src/src.pro ) && make -f Makefile
make[1]: Entering directory '/root/prefix/default/src/qwt5/src'
g++ -c -pipe -O2 -D_REENTRANT -Wall -W -fPIC -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -I. -isystem /usr/include/i386-linux-gnu/qt5 -isystem /usr/include/i386-linux-gnu/qt5/QtGui -isystem /usr/include/i386-linux-gnu/qt5/QtCore -Imoc -isystem /usr/include/libdrm -I/usr/lib/i386-linux-gnu/qt5/mkspecs/linux-g++ -o obj/qwt_abstract_scale_draw.o qwt_abstract_scale_draw.cpp
In file included from qwt_abstract_scale_draw.cpp:19:
qwt_scale_map.h:87:5: error: ‘QT_STATIC_CONST’ does not name a type
87 | QT_STATIC_CONST double LogMin;
| ^~~~~~~
qwt_scale_map.h:88:5: error: ‘QT_STATIC_CONST’ does not name a type
88 | QT_STATIC_CONST double LogMax;
| ^~~~~~~
make[1]: [Makefile:951: obj/qwt_abstract_scale_draw.o] Error 1
make[1]: Leaving directory '/root/prefix/default/src/qwt5/src'
make: [Makefile:47: sub-src-make_first] Error 2
PyBOMBS.Packager.source - ERROR - Build failed. See output above for error messages.
PyBOMBS.Packager.source - ERROR - Problem occurred while building package qwt5:
Build failed.
PyBOMBS.install_manager - ERROR - Error installing package qwt5. Aborting.
Is there a workaround to get this installed to proceed with the gnuradio and gr-gsm installation? I work with a brand new upgraded Kali 2019.3 Machine, just for your interest. Thanks in advance
Hi @msfu101,
I work with a brand new upgraded Kali 2019.3 Machine, just for your interest.
For Kali and Debian Testing my advice would be to install gnuradio and the other dependencies from that apt repository and then install gr-iqbal, gr-osmosdr and gr-gsm following the steps from Kali docker file
I tried your recipes and it seems it works fine, but then pybombs tried to install package qwt5... This action fails on my Machine... it says:
PyBOMBS.install_manager - INFO - Installing package: qwt5 01372 kB / 01372 kB (100%) Configuring: (100%) [=====================================================================================================================] Building: (100%) [=====================================================================================================================] cd src/ && ( test -e Makefile || /usr/lib/qt5/bin/qmake -o Makefile /root/prefix/default/src/qwt5/src/src.pro ) && make -f Makefile make[1]: Entering directory '/root/prefix/default/src/qwt5/src' g++ -c -pipe -O2 -D_REENTRANT -Wall -W -fPIC -DQT_NO_DEBUG -DQT_GUI_LIB -DQT_CORE_LIB -I. -isystem /usr/include/i386-linux-gnu/qt5 -isystem /usr/include/i386-linux-gnu/qt5/QtGui -isystem /usr/include/i386-linux-gnu/qt5/QtCore -Imoc -isystem /usr/include/libdrm -I/usr/lib/i386-linux-gnu/qt5/mkspecs/linux-g++ -o obj/qwt_abstract_scale_draw.o qwt_abstract_scale_draw.cpp In file included from qwt_abstract_scale_draw.cpp:19: qwt_scale_map.h:87:5: error: ‘QT_STATIC_CONST’ does not name a type 87 | QT_STATIC_CONST double LogMin; | ^
~~~~~~ qwt_scale_map.h:88:5: error: ‘QT_STATIC_CONST’ does not name a type 88 | QT_STATIC_CONST double LogMax; | ^~~~~~~ make[1]: [Makefile:951: obj/qwt_abstract_scale_draw.o] Error 1 make[1]: Leaving directory '/root/prefix/default/src/qwt5/src' make: [Makefile:47: sub-src-make_first] Error 2 PyBOMBS.Packager.source - ERROR - Build failed. See output above for error messages. PyBOMBS.Packager.source - ERROR - Problem occurred while building package qwt5: Build failed. PyBOMBS.install_manager - ERROR - Error installing package qwt5. Aborting.Is there a workaround to get this installed to proceed with the gnuradio and gr-gsm installation?
Most probably you are trying to install gnuradio
receipt that will install gnuradio 3.7 and not 3.8. Try installing gnuradio38
instead but note that the receipts for gnuradio 3.8 are still not finished and on a clean system installation most probably is going to fail and you need to search and install dependencies manually. Consider reporting any pybombs failures in the gr-receipts project
okay i tried it but the last step don't work
RUN mkdir /src/build --->>> here it says mkdir: cannot create directory ‘/src/build’: No such file or directory WORKDIR /src/build RUN cmake .. && \
# and .gnuradio directories do not exist
mkdir $HOME/.grc_gnuradio/ $HOME/.gnuradio/ && \
make -j $(nproc) && \
make install && \
ldconfig && \
make CTEST_OUTPUT_ON_FAILURE=1 test
do you have a solution for that or i am too dumb xD
@msfu101, Please avoid posting here messages not directly related to gnuradio 3.8 support. In the other issue I've already told you to post questions and installation problems to our mailing list.
RUN mkdir /src/build --->>> here it says mkdir: cannot create directory ‘/src/build’: No such file or
You don't have permissions to create directories in root partition (/), clone the git repositories and create build sub-directories in your home directory instead.
Regarding gr-osmosdr - possibility of having a supported version for GNU Radio 3.8 is low. Most probably we'll have to switch to Soapy, but I never tried it yet and don't know how well/bad it might work.
why? does that mean that gr-osmosdr is orphaned and not maintained anymore?
this must be a joke for shure ... mkdir /src/build mkdir: cannot create directory ‘/src/build’: No such file or directory
peapol must make a more friendly build install of this repo ...
Is there still a Debian distro out there with gnuradio package 3.7 and thus not affected by this issue?
Hi @jsa91,
Is there still a Debian distro out there with gnuradio package 3.7 and thus not affected by this issue?
Yes, debian buster has gnuradio-3.7.13.4-4
Debian's gr-gsm package also needs updating for GNURadio 3.8. Could we have a new release please?
Any progress with the "Gnuradio could not be found because dependency Volk could not be found." mess?
@msfu101, Please avoid posting here messages not directly related to gnuradio 3.8 support. In the other issue I've already told you to post questions and installation problems to our mailing list.
RUN mkdir /src/build --->>> here it says mkdir: cannot create directory ‘/src/build’: No such file or
You don't have permissions to create directories in root partition (/), clone the git repositories and create build sub-directories in your home directory instead.
Hello my friend, would you be so kind to tell me if is this fixed now ? thanks in advance !
I'm trying to package gr-gsm for alpine linux, and i had great success previously against gr3.7, but now that gr3.8 is out it seems to me that gr-gsm is not yet ready for this. i get the following errors during cmake:
gnuradio-3.8 and it's dev environment is installed, as is libvolk-2.0 and also it's headers, as can be seen from the cmake later finding volk.
do you guys have an idea what might go wrong here if i'm messing up something, or if indeed there is no gr3.8 support yet, then what is your roadmap?
thanks a lot for all your awesome work!