robotastic / trunk-recorder

Records calls from a Trunked Radio System (P25 & SmartNet)
GNU General Public License v3.0
868 stars 195 forks source link

Docker status #365

Closed robbiet480 closed 4 years ago

robbiet480 commented 4 years ago

Hi @robotastic,

Thought I should create a issue to continue work from https://github.com/robotastic/docker-gnuradio/issues/2. Now that I've figured out the path to get multiple architectures building properly, i'm working on improving the entire trunk-recorder Docker ecosystem. This includes updating the two repos Dockerfile to follow best practices, slimming them down as much as possible and improving the Docker Compose story. Unrelated to trunk-recorder specifically, I'm also working on containerizing @chuot's rdio-scanner so that users can get a whole stack running with a single Docker Compose file and much faster too, especially for those on Raspberry Pi.

I've created this issue because I was hoping to get the current lay of the land from you and @EricTendian. I see a lot of Docker issues in multiple repos but am unsure where my time is best spent so as to not duplicate work. So, a few questions to start:

  1. Based on #349 it appears we need a new Dockerfile to support GNURadio 3.8. Is this correct? I see https://github.com/robotastic/trunk-recorder-docker-gr3.8, is this actually working? Should we start using GNURadio 3.8 as the base for Docker?
  2. What's the issue with Locales? I see you are setting everything to en_US.UTF-8. Why? Can we stop doing that and just use C or C.UTF-8?
  3. Is there any way to build a static binary so that we can just have the GNURadio and other deps from docker-gnuradio and none of the extra cruft of this repo? I like keeping Docker images nice and small so its quick for users to get going.
  4. Are there any magic cmake or make flags I can set to optimize cross compiling? As mentioned in https://github.com/robotastic/docker-gnuradio/issues/2 compile times are quite long right now. I'd like to speed those up if possible.

Here's the current Dockerfile I'm using which works nicely and hews to expected Docker standards:

FROM robotastic/docker-gnuradio:latest

WORKDIR /src

COPY . .

RUN cmake . && make -j`nproc` && cp recorder /recorder

CMD ["/recorder"]

And the new and improved docker-compose.yaml, featuring rdio-scanner support too:

version: '3'
volumes:
  rdio_database:
services:
  recorder:
    image: robotastic/trunk-recorder:latest
    container_name: trunk-recorder
    restart: always
    devices:
      - /dev/bus/usb/001/003:/dev/bus/usb/001/003
    volumes:
      - ./media:/app/media
      - ./config:/app/config
      - ./ebrcs.csv:/app/ebrcs.csv
  rdio:
    image: robbiet480/rdio-scanner:latest
    container_name: rdio-scanner
    volumes:
      - ./rdio_config.json:/app/server/config.json
      - rdio_database:/app/server/database.sqlite
      - ./media:/calls

The big improvements to the Compose file are no more having to build a second image just for Compose, not having to run privileged mode and explicitly mounting CSVs instead of just copying everything over.

Once Docker is improved I strongly recommend that you suggest it as the premiere installation method for end users, especially those on Raspberry Pi, so that they don't have to deal with deps and compile times.

Excited to hear your thoughts and contribute to this fantastic project!

ghost commented 4 years ago

I started looking at the docker-compose file yesterday as well (but I'm not a home-assistant software developer so my skillset is substantially more limited :smile: ). I didn't like that the existing docker-compose.yml ran as privileged. I know is needs access to the sdr dongles, but I didn't want privileged. So like you, I've been using:

devices:
      - "/dev/bus/usb:/dev/bus/usb"

I have multiple dongles, so pass the whole bus so they can all be accessed. Not what I'd want ideally but I'm guessing the goal behind the docker-compose is to have a "run and go" method.

I like that your version doesn't require a docker-build as it doesn't copy the talkgroups file in. In terms of keeping with the "run and go" would it make sense to move the talkgroups .csv files into their own /csv directory (or whatever) and then pass that as a volume and update the configuration file to source them from there? Then we could have multiple .csv and config.json could just pull them in from a directory.

My final thought was that I wasn't sure if we wanted to continue to run the docker container as root. That's generally not considered best practice as far as I know, but without that the user would have to configure permissions on the host to allow the docker user rw access to the sdr dongles. So it's not "fire and forget".

I would agree docker is the "easy" way to go with running trunk-recorder; mine is running on my Ryzen desktop but I've bought a NUC and will just copy a folder between the machines and docker-compose up when I move it to it's dedicated server. In the past I've tried to build it natively on Pi's but was converted over to being a docker person since my last run at trunk-recorder. Going docker this time was substantially easier.

robotastic commented 4 years ago

@robbiet480 Thanks so much for looking into this! Improving Docker support has been on my list of things to do for a long time. I agree - once this is buttoned up, this should be the primary way of running Trunk Recorder. I can look at setting up an automated build pipeline so it automatically spits out new multi-arch images when there is a push to the repo. It may take a little EC2 time, but totally worth it. I actually just started messing around with buildx too, so all in on this approach.

More detailed: 1) The Dockerfile in TR needs to be updated for GR3.8. It can't hurt to goto 3.8. It will probably have the latest SDR drivers. 2) I am flexible on Locales, I think I just went with that because I was getting an error and needed to specify some Repo. 3) Do you mean from TR? I think we just need the recorder binary and can delete everything after we are done building. Def worth a shot. 4) Yea, I am seeing this in another project I am working on and compiling OpenCV. I don't think there are any good shortcuts. If we move it to an AWS EC2 instance though, it shouldn't be too big a problem. There are not too many commits to Trunk Recorder.

For the Pi - It could be interesting to look at https://www.balena.io as a way to push and manage Docker images on the Pi.

Thanks again for digging into this!

robbiet480 commented 4 years ago

@jquagga Thanks for your feedback here. I knew I forgot to do something when building the new Dockerfile... will push a fix to run as non-root. I don’t believe the user will need to do anything with permissions to pass in the device but will check it out. As for the CSV thing, I like it this way, only because it’s the most flexible option. If users want a folder, they can mount a folder and update their config to point to the mounted path.

@robotastic I’d suggest using a GitHub action to do builds since it’s just so easy to use. This one looks perfect and is plug and play.

As for your responses, a few things:

robotastic commented 4 years ago

Cool - I didn't realize the GitHub actions included some compute time. I will work on getting that going, and that action you linked to looks like a good starting point. since it is running on Githubs compute, the long build time isn't the end of the world, esp since there are not that many commits.

I will go checkout the openmhz code. It was suppose to check for duplicate system shortnames... clearly broken though.

robbiet480 commented 4 years ago

@robotastic Did you get anywhere with determining if we can do static builds? That's probably the second biggest thing holding up progress & optimization, after getting GNURadio 3.8 building.

robbiet480 commented 4 years ago

https://github.com/robotastic/docker-gnuradio/pull/3

robbiet480 commented 4 years ago

366 is ready for review. We should get that GNURadio Docker image merged before merging #366. Honestly though, I was thinking if its a good idea to continue building GNURadio and trunk-recorder separate, since we really sure be making sure they are versioned together I think. This is fine for now though.

robbiet480 commented 4 years ago

Oh yeah, we should absolutely attempt to get static builds going still. The latest image size (with GNURadio 3.8) is 1.92GB. If we got static builds going, we could probably get to under 100mb. Just tried to get static builds going following the path of #226 but no go thus far. Probably going to take serious work from @robotastic but will be well worth it I think. I sadly don't know enough about C compilers to figure out how to build a static binary that also includes all the deps.

robotastic commented 4 years ago

Thanks for patching up the Docker script - I just got something similar working on GitLab and was going to turn to this. I will go work on Static linking right now. I am not sure if we need to even worry about building GnuRadio. I think I was doing that because the version of gnuradio that used to be in the apt-get repo was so old. It is probably better to just install GR using apt-get. Maybe we could keep a bleeding edge version that would compile gnuradio from src nightly to test against.

robbiet480 commented 4 years ago

Would also agree that we should continue doing apt-get for GNURadio.

robotastic commented 4 years ago

Ok... slight wrinkle... I have everything building staticly, except when it gets to all the gnuradio libs. and if you install them using apt-get, you only get shared libraries. It looks like if you want static libraries, you have to build grnuadio from source. You can use this flag: ENABLE_STATIC_LIBS ( from here: https://www.gnuradio.org/doc/doxygen/build_guide.html ) Check out the static branch.... I could also try to make everything static, except for libgnuradio. Not sure how much space that would save.

robbiet480 commented 4 years ago

Not the worst idea to build GNURadio from scratch if it means we also can drop any GUI requirements. That would be a HUGE size savings and almost certainly get things starting up faster too.

robbiet480 commented 4 years ago

Just doing apt-get install gnuradio is the largest space hog by far, almost certainly due to installing QT. Here's all the packages it attempts to install:

adwaita-icon-theme
at-spi2-core
blt
dbus-user-session
dconf-gsettings-backend
dconf-service
fontconfig
fontconfig-config
fonts-dejavu-core
fonts-liberation
fonts-lyx
freeglut3
gdal-data
glib-networking
glib-networking-common
glib-networking-services
gnuradio
gnuradio-dev
graphviz
gsettings-desktop-schemas
gtk-update-icon-cache
hicolor-icon-theme
javascript-common
libaec0
libarmadillo9
libarpack2
libasyncns0
libatk-bridge2.0-0
libatk1.0-0
libatk1.0-data
libatspi2.0-0
libavahi-client3
libblas3
libboost-atomic1.67-dev
libboost-atomic1.67.0
libboost-chrono1.67-dev
libboost-chrono1.67.0
libboost-date-time-dev
libboost-date-time1.67-dev
libboost-date-time1.67.0
libboost-filesystem-dev
libboost-filesystem1.62.0
libboost-filesystem1.67-dev
libboost-filesystem1.67.0
libboost-program-options-dev
libboost-program-options1.67-dev
libboost-program-options1.67.0
libboost-regex1.67.0
libboost-serialization1.67-dev
libboost-serialization1.67.0
libboost-system-dev
libboost-system1.62.0
libboost-system1.67-dev
libboost-system1.67.0
libboost-test-dev
libboost-test1.67-dev
libboost-test1.67.0
libboost-thread-dev
libboost-thread1.67-dev
libboost-thread1.67.0
libboost-timer1.67.0
libboost1.67-dev
libcaca0
libcairo-gobject2
libcairo2
libcdt5
libcgraph6
libcharls2
libcodec2-0.8.1
libcolord2
libcomedi0
libcppunit-1.14-0
libcppunit-dev
libcroco3
libcups2
libdap25
libdapclient6v5
libdapserver7v5
libdatrie1
libdconf1
libdouble-conversion1
libdrm-amdgpu1
libdrm-common
libdrm-nouveau2
libdrm-radeon1
libdrm2
libegl-mesa0
libegl1
libepoxy0
libepsilon1
libevdev2
libfftw3-bin
libfftw3-dev
libfftw3-double3
libflac8
libfontconfig1
libfreexl1
libfyba0
libgail-common
libgail18
libgbm1
libgd3
libgdal20
libgdk-pixbuf2.0-0
libgdk-pixbuf2.0-bin
libgdk-pixbuf2.0-common
libgeos-3.7.1
libgeos-c1v5
libgeotiff2
libgfortran5
libgif7
libgl1
libgl1-mesa-dri
libgl1-mesa-glx
libglapi-mesa
libgles2
libglu1-mesa
libglvnd0
libglx-mesa0
libglx0
libgnuradio-analog3.7.13
libgnuradio-atsc3.7.13
libgnuradio-audio3.7.13
libgnuradio-blocks3.7.13
libgnuradio-channels3.7.13
libgnuradio-comedi3.7.13
libgnuradio-digital3.7.13
libgnuradio-dtv3.7.13
libgnuradio-fcd3.7.13
libgnuradio-fec3.7.13
libgnuradio-fft3.7.13
libgnuradio-filter3.7.13
libgnuradio-noaa3.7.13
libgnuradio-pager3.7.13
libgnuradio-pmt3.7.13
libgnuradio-qtgui3.7.13
libgnuradio-runtime3.7.13
libgnuradio-trellis3.7.13
libgnuradio-uhd3.7.13
libgnuradio-video-sdl3.7.13
libgnuradio-vocoder3.7.13
libgnuradio-wavelet3.7.13
libgnuradio-wxgui3.7.13
libgnuradio-zeromq3.7.13
libgraphite2-3
libgsl23
libgslcblas0
libgsm1
libgtk-3-0
libgtk-3-bin
libgtk-3-common
libgtk2.0-0
libgtk2.0-bin
libgtk2.0-common
libgts-0.7-5
libgts-bin
libgudev-1.0-0
libgvc6
libgvpr2
libharfbuzz0b
libhdf4-0-alt
libhdf5-103
libice6
libimagequant0
libinput-bin
libinput10
libjack-jackd2-0
libjbig0
libjs-jquery
libjs-jquery-ui
libjson-glib-1.0-0
libjson-glib-1.0-common
libkmlbase1
libkmlconvenience1
libkmldom1
libkmlengine1
libkmlregionator1
libkmlxsd1
liblab-gamut1
liblapack3
liblcms2-2
libllvm9
liblog4cpp5-dev
liblog4cpp5v5
libltdl7
libmariadb3
libminizip1
libmtdev1
libnetcdf13
libnorm1
libnotify4
libnspr4
libnss3
libodbc1
libogdi3.2
libogg0
libopenjp2-7
libopus0
liborc-0.4-0
libpango-1.0-0
libpangocairo-1.0-0
libpangoft2-1.0-0
libpathplan4
libpcre2-16-0
libpgm-5.2-0
libpixman-1-0
libpoppler82
libportaudio2
libpq5
libproj13
libproxy1v5
libpulse0
libpython-dev
libpython2-dev
libpython2.7
libpython2.7-dev
libqhull7
libqt5core5a
libqt5dbus5
libqt5designer5
libqt5gui5
libqt5help5
libqt5network5
libqt5opengl5
libqt5printsupport5
libqt5sql5
libqt5sql5-sqlite
libqt5svg5
libqt5test5
libqt5widgets5
libqt5xml5
libqwt-qt5-6
librest-0.7-0
librsvg2-2
librsvg2-common
libsdl1.2debian
libsensors-config
libsensors5
libsm6
libsndfile1
libsodium23
libsoup-gnome2.4-1
libsoup2.4-1
libspatialite7
libsuperlu5
libsz2
libtcl8.6
libthai-data
libthai0
libtiff5
libtk8.6
libuhd3.13.1
liburiparser1
libvolk1-bin
libvolk1-dev
libvolk1.4
libvorbis0a
libvorbisenc2
libwacom-bin
libwacom-common
libwacom2
libwayland-client0
libwayland-cursor0
libwayland-egl1
libwayland-server0
libwebp6
libwebpdemux2
libwebpmux3
libwxbase3.0-0v5
libwxgtk3.0-gtk3-0v5
libx11-xcb1
libxaw7
libxcb-dri2-0
libxcb-dri3-0
libxcb-glx0
libxcb-icccm4
libxcb-image0
libxcb-keysyms1
libxcb-present0
libxcb-randr0
libxcb-render-util0
libxcb-render0
libxcb-shape0
libxcb-shm0
libxcb-sync1
libxcb-util0
libxcb-xfixes0
libxcb-xinerama0
libxcb-xkb1
libxcomposite1
libxcursor1
libxdamage1
libxdot4
libxerces-c3.2
libxfixes3
libxft2
libxi6
libxinerama1
libxkbcommon-x11-0
libxkbcommon0
libxmu6
libxpm4
libxrandr2
libxrender1
libxshmfence1
libxslt1.1
libxss1
libxt6
libxtst6
libxxf86vm1
libzmq5
mariadb-common
mysql-common
notification-daemon
odbcinst
odbcinst1debian2
poppler-data
proj-bin
proj-data
python-backports.functools-lru-cache
python-bs4
python-cairo
python-chardet
python-cheetah
python-cycler
python-dateutil
python-decorator
python-dev
python-enum34
python-gdal
python-gobject-2
python-gtk2
python-html5lib
python-kiwisolver
python-lxml
python-matplotlib
python-matplotlib2-data
python-networkx
python-numpy
python-olefile
python-opengl
python-pil
python-pkg-resources
python-pygraphviz
python-pyparsing
python-pyqt5
python-scipy
python-sip
python-six
python-soupsieve
python-subprocess32
python-tk
python-tz
python-webencodings
python-wxgtk3.0
python-wxversion
python-yaml
python-zmq
python2-dev
python2.7-dev
qt5-gtk-platformtheme
qttranslations5-l10n
tk8.6-blt2.5
ttf-bitstream-vera
uhd-host
x11-common
ghost commented 4 years ago

Ok, question on this one. I was trying to help someone in the gitter get docker going on their pi which meant they were trying to use the updated guide in wiki. I've been using the "legacy" trun-recorder-docker image but thought I'd give the new spin I go. Using the default docker-compose now listed on the wiki (the one which runs as nobody and doesn't build a new image) with the :edge image as of today tosses an error that it cannot find config.json.

I think this is because the changes to the docker image now put the recorder binary into /recorder, but does not pass a --config=/app/config/config.json. So when the :edge image starts up, it looks for config.json in /config.json but it's not there in the example docker-compose.yml. So I think either the docker-compose.yml on the wiki may need it's path to config.json adjusted, or the Dockerfile could use the --config=/app/config/config.json of CMD restored. I don't think it matters which way it goes, they just have to line up.

I have another docker image next to this one running audioplayer.php so I have to adjust paths in that to match the new docker setup. Before I do that, I wanted to confirm which way this was going to go.

robbiet480 commented 4 years ago

Frick. Good catch @jquagga. Will submit a PR to hardcode looking for config at /app/config.json (really no reason for it to be /app/config/config.json I think, but open to discussion!)

Good reminder for me btw: @robotastic You still need to cut a new release of trunk-recorder so that the latest tag gets uploaded to Docker Hub. You should do that after merging https://github.com/robotastic/docker-gnuradio/pull/4 and cutting a release there so that it also uploads the latest tag and trunk-recorder Docker gets built with GR3.8. (Also don’t forget to set the same Docker secrets on the docker-gnuradio repo too...) Thanks!

robbiet480 commented 4 years ago

@robotastic To get TR build going, ou just need to cut a 1.0 of gnuradio repo then restart the trunk-recorder build. Its trying to grab :latest tag but that tag doesn't exist yet.

hayden-t commented 3 years ago

wheres the armv7 docker image at ?

robbiet480 commented 3 years ago

wheres the armv7 docker image at ?

It's set to build but for some reason failing when building on top of the GNURadio image. Looking into it now.

robbiet480 commented 3 years ago

We're being bit by this issue in CMake.

hayden-t commented 3 years ago

any old docker images i can get ?

robbiet480 commented 3 years ago

Nope, armv7 was never properly built. Will have a fix late today probably.

hayden-t commented 3 years ago

oh ok, that would be neat ! thanks

robbiet480 commented 3 years ago

@hayden-t @robotastic just merged #448 so in the next hour or so you should see valid armv7 images live on his Docker Hub as v3.3.2.

robbiet480 commented 3 years ago

@hayden-t edge is now live, @robotastic hasn't cut 3.3.2 yet so give edge a try and let me know! https://hub.docker.com/layers/robotastic/trunk-recorder/edge/images/sha256-835abe8b2103b8e8166faa3f63374606749eb5a34d21d8ca19460f1ee12f4280?context=explore

hayden-t commented 3 years ago

ok i got it, and tried to run it, i will open support ticket seperately

hayden-t commented 3 years ago

can you help https://github.com/robotastic/trunk-recorder/issues/450

jp83 commented 3 years ago

Sorry to tack on here, but I can't get docker usb permissions to work with the latest Docker images. I have my own really old Docker build based on previous image that I overlaid and copied in my own config a year or two ago, but something underlying must have changed. I previously had --privileged -v /dev/bus/usb:/dev/bus/usb. I've tried --device /dev/bus/usb and --device-cgroup-rule 'a : rwm'. I keep getting the usual:

Using device #0 Generic RTL2832U OEM usb_open error -3 Please fix the device permissions, e.g. by installing the udev rules file rtl-sdr.rules

I understand from the FAQ to git rtl-sdr and build it, but shouldn't that be built into the image from Dockerfile? What am I missing? I happen to be running on a small thin client machine with Rancher OS, lightweight operating system. For reference I just got openwebrx docker image running with my rtl-sdr out-of-the-box. Please help. Thanks.

leee commented 3 years ago

Even though this issue was about Docker, the purpose and original use of this issue was for growing Docker support, not providing support on Docker issues. The comments in March probably could've been a new thread, so lets keep closed issues closed.

@jp83 as for your issue, I think it deserves a new issue of its own - would you be willing to create one? I also think referencing which openwebrx image you use (and the Dockerfile for it) could also be especially helpful for @robbiet480 or whoever wants to take up the fix.

Thanks!

jp83 commented 3 years ago

Understand, opened #461.