use in production

[frecuencialibre]

Thank you for all this work!!!

After spending many MANY hours trying to get LibreTime working in various different server setups, I was honestly blown away by how quickly I was able to get a working instance spun up using this docker build. Well f***ing done. :)

Using docker seems very promising, and clearly superior to both installing LibreTime directly in the host operating system, and also to the development setup we've been experimenting with (see my forum post for description and diagram ) for the goal of being able to both reliably serve the needs of a real radio station with alpha-stage open source software, and also contribute back to LibreTime. If you read that forum post I linked you'll see how similar our goals are, with the difference that I had no knowlege of what docker even is.

And so we're almost ready to bet on this approach for our station, hopefully contributing documentation, bug work, etc. but need to ask two big questions.

1. I'd love to confirm that using docker for production won't bring unforseen problems down the line. Thoughts?

I obviously don't have the experience to be able to make a call here, and am hearing conflicting accounts. On one hand you and many others ARE using docker for production, and your build readme provides specific, apparently quite simple steps to do so.

• a hacker friend who has used Docker for years gives me ominous but vague warnings, and notes that the tool was born to set up development environments.
• the internet is full of there be dragons type warnings, but they're typically from people who stand to profit from you being scared to deploy with docker yourself, and their articles seem to be primarily related to large-scale deployments on clusters. Our LibreTime production instance (and I'd venture most LibreTime instances) will only need to allow a small (< 20) group of people to log in to run our community radio station. Our specific case is perhaps a little rare in that, internet access here in this part of México is intermittant, so we'll be running our production instance on a cheap server right here locally, next to the FM transmitter. Hence dev, staging, and production will all be on single machines with identical kernels (debian 9) -- no swarming clustered corporate herds, no real need for "container orchestration," right? no dragons? Perhaps this project could also ship with a battle-tested production version of docker-compose.yml?

2. What would a dev-staging-production workflow using ubuntu-multicontainer-libretime look like?

Would this be a job for docker machine? My apologies in advance for my ignorance regarding docker, I'm coming from javascript land...

Again, many thanks. It seems clear that LibreTime is in sore need of a modern production deployment process in order to help folks get up and running quickly in order to not lose as much time as I have haha. I'm excited to get up and running providing documentation in both spanish and english.

ryan

[ned-kelly]

@frecuencialibre - Thanks for your feedback, much appreciated!

After reading both your discourse post and your reply here I've got a few quick points that I'll add re your comments/concerns which should help make your decisions easier:

• Using Docker in production is fast becoming the "normal" across many organisations around the globe so I wouldn't be scared about it, but rather embrace it - here's some key considerations for your deployment (using docker):

  • Your dev/stage/testing environments will now all finally be identical.

  • You're going to use much less resources using a multi-container docker architecture on one single VM than a multi-VM architecture ... Obviously if you were a large commercial station, you may want to split out some of the components (such as the PostgreSQL database, RabbitMQ etc) onto multiple VM's behind a load balancer to ensure the solution is Highly Available -- What's currently there in terms of the architecture on a single-host should be more than sufficient for a community station.

  • It's easily scalable, future proof, and from a security point of view much better than just plonking everything on a single host (if one service is compromised it's much harder to break out of the container if properly setup/configured).

  • Easy to upgrade/deploy!

  • There's loads more pros/cons of docker as I'm sure you're aware, I won't list them all here...

  • Your friends comments re: docker is for 'development environments' are not 100% spot on sorry, In short thousands of organisations rely on Docker around the world for large scale production deployments - In terms of enterprise deployment the product "Kubernetes" (which was born by Google, and is actively used throughout their organisation to orchestrate thousands of Docker containers in production that run many of the Google services you see under the hood) - Hopefully that's proof enough that Docker's fit for production..

  • What would a dev-staging-production workflow using ubuntu-multicontainer-libretime look like?

  • I think it's probably best that we have a quick chat on something like Slack or Skype about this one quickly so I can understand where you want to go (Perhaps send me a PM with your Skype username - if you don't have Skype, I'll send you a Slack invite) - in short you should just be able to stand up the same docker-compose file both locally and on your "stage/prod" VM's If you're going to make customisations to the deployment you will probably want to script them into the actual Docker Build (See the readme for details here), and then Build your own docker container with your customisations for dev/stage/prod rather than pulling the "generic setup" I've got published on Docker Hub directly.

All-in-all, I'm running this setup fine for production (which is why it was built), There's a few issues with Liquidsoap that need to be ironed out - Occasionally a station "Jingle" might cut short 1 second or repeat, but these seem to be occurring regardless of "docker or no docker" - (I'm currently hunting down a fix, and will push one up when We've got it) - And there's also the issue of not easily being able to import thousands of hundreds of thousands of audio files directly from the filesystem (also not a Docker problem, but rather a Libretime feature still needing to be developed: https://github.com/LibreTime/libretime/issues/70)

Looking forward to your comments.

[frecuencialibre]

baddass! i think PM's have been disabled so i emailed you at the address you have in git for your public commits. my skype is associated with that same email address. i'll just post it here if you don't get the email. i'll be busy this afternoon, but am reading a docker beginners tutorial now.

[frecuencialibre]

regarding the "issues with Liquidsoap that need to be ironed out," could this be the known issue with silan? I was hearing tracks be cut off until I ran

wget -qO- http://download.opensuse.org/repositories/home:/hairmare:/silan/Debian_7.0/Release.key   | apt-key add -
echo 'deb http://download.opensuse.org/repositories/home:/hairmare:/silan/xUbuntu_16.04 ./'   > /etc/apt/sources.list.d/hairmare_silan.list
apt-get update
apt-get install silan

within libretime-core.

note that I had to re-import tracks.

[frecuencialibre]

would an additional production docker-compose file, possibly with the security certificate proxy you mentioned, ports, and anything else you see pertinent be a good option? just sharing my newbie research haha.

https://github.com/docker/docker.github.io/blob/master/compose/extends.md#different-environments docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d

ps. docker rocks:

docker exec -it libretime-core bash

docker stop libretime-core
nano .env
docker-compose up --no-deps -d libretime-core

[ned-kelly]

Hi @frecuencialibre - Glad you're figuring it out - No need for another docker compose file for production, you can just add in something like Caddy in another container and point to the internal network name of the container (libretime-core) - The issue currently with Caddy (or any the only other reverse proxy I've tried - Nginx) is that the Login page is not working - I have a feeling it's going to be something not being properly forwarded in the Headers - Feel free to post back any results if you have time to investigate.

Your observations re silan i suspect are correct, however upgrading it did not help in my situation when testing.

[frecuencialibre]

when you tested did you delete and re-import the audio files?

[ned-kelly]

@frecuencialibre No, the audio files that you upload be saved on your filesystem - so as long as you keep the PostgreSQL database and the Audio Files mapped in, you can re-build/upgrade the libretime-core as may times as you want and the config will be kept.

[frecuencialibre]

i found that audio files that i had imported prior to upgrading silan continued to cut off incorrectly even after upgrading silan. not sure if audio duration is somehow saved into the db upon import... but anyway files imported through the web interface after upgrading silan then had correct duration.

[Robbt]

Yeah the database entry is modified. You could zero out the fields it stores or rerun the silan on it. I have s prototype of automatic import as a PR also that would allow you to copy the files to a directory and have them automatically reimported.