locast2tuner in Docker - WIP

eiddor commented 3 years ago

Forked from #7

The Docker version is going to be an interesting challenge for me - I could build it realtime inside of a Rust image, but that would mean a 10 minute process anytime the container is created. There's also the problem of dynamically linked libraries:

roddie@testubuntu:~/dev/locast2tuner/target/debug (main)% ldd locast2tuner
        linux-vdso.so.1 (0x00007ffd43baf000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f19c75e4000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f19c75c1000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f19c7472000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f19c746c000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f19c727a000)
        /lib64/ld-linux-x86-64.so.2 (0x00007f19c8ff5000)

Looking at some options now - A lot of this is new to me, so it'll be learning on the fly.

eiddor commented 3 years ago

This project might be the answer for me: https://github.com/clux/muslrust

Sure enough!

roddie@testubuntu:~/dev/docker/locast2tuner/target/x86_64-unknown-linux-musl/debug (main)% ldd locast2tuner
        not a dynamic executable

I'll try throwing it into an alpine image next to see how it runs.

eiddor commented 3 years ago

Wow - I did it (using the binary built above).

Dockerfile:

FROM alpine:latest

COPY /locast2tuner /

ENTRYPOINT ["locast2tuner", "-U", "<email address>", "-P", "<password>", "-a", "0.0.0.0"]

Building:

[~/docker] roddie@testubuntu% docker build -t locast2tuner .
Sending build context to Docker daemon  152.6MB
Step 1/2 : FROM alpine:3.7
3.7: Pulling from library/alpine
5d20c808ce19: Pull complete 
Digest: sha256:8421d9a84432575381bfabd248f1eb56f3aa21d9d7cd2511583c68c9b7511d10
Status: Downloaded newer image for alpine:3.7
 ---> 6d1ef012b567
Step 2/2 : ENTRYPOINT ["locast2tuner", "-U", "<email>", "-P", "<password>!", "-a", "0.0.0.0"]
 ---> Running in 5055a16f34a6
Removing intermediate container 5055a16f34a6
 ---> bbc995cd0314
Successfully built bbc995cd0314
Successfully tagged locast2tuner:latest

Running:

[~/docker] roddie@testubuntu% docker run -p 6077:6077 locast2tuner
[INFO] - locast2tuner 0.1.0 on Linux 5.4.0-70-generic starting..
[INFO] - UUID: <UUID>
[INFO] - Logging in with <email>
[INFO] - Login succeeded!
[INFO] - Downloading FCC facilities
[INFO] - Cached FCC facilities to /root/.locast2tuner/facilities
[INFO] - Loading stations for Dallas (cache: true, cache timeout: 3600, days: 8)..
[INFO] - Starting http server for Dallas on http://0.0.0.0:6077
[INFO] - Starting 1 workers
[INFO] - Starting "actix-web-service-0.0.0.0:6077" service on 0.0.0.0:6077
[INFO] - Tuners:
[INFO] -  +--------+----------+-----+--------------------------------------+-----------------+---------------------+
[INFO] -  | City   | Zip code | DMA | UUID                                 | Timezone        | URL                 |
[INFO] -  +--------+----------+-----+--------------------------------------+-----------------+---------------------+
[INFO] -  | Dallas |          | 623 | <UUID> | America/Chicago | http://0.0.0.0:6077 |
[INFO] -  +--------+----------+-----+--------------------------------------+-----------------+---------------------+
[INFO] - locast2tuner started..
[INFO] - Stream c9bfabf - ......

It freaking worked on both Plex and Emby.

TODO:

~~Test the binary/Dockerfile built on Ubuntu on my Mac~~ - SUCCESS!
Integrate config file into the Dockerfile
Document the config file format and provide a sample
Document how to run it in Docker with volume/port mapping
Figure out how to package and distribute the binary (it's 150MB), and who owns the build process
Investigate/test this project as an alternative for builds
Make it all ready for prime-time

wouterdebie commented 3 years ago

Awesome! I'm working on Debian/Ubuntu packaging and using Github actions. We can build everything there. We might want to consider using the .deb file that will be built instead of a separate binary.

On Mon, Apr 5, 2021, 21:25 Roddie Hasan @.***> wrote:

Wow - I did it (using the binary built above).

Dockerfile:

FROM alpine:latest

COPY /locast2tuner /

ENTRYPOINT ["locast2tuner", "-U", "", "-P", "", "-a", "0.0.0.0"]

Building:

[~/docker] @.***% docker build -t locast2tuner . Sending build context to Docker daemon 152.6MB Step 1/2 : FROM alpine:3.7 3.7: Pulling from library/alpine 5d20c808ce19: Pull complete Digest: sha256:8421d9a84432575381bfabd248f1eb56f3aa21d9d7cd2511583c68c9b7511d10 Status: Downloaded newer image for alpine:3.7 ---> 6d1ef012b567 Step 2/2 : ENTRYPOINT ["locast2tuner", "-U", "", "-P", "!", "-a", "0.0.0.0"] ---> Running in 5055a16f34a6 Removing intermediate container 5055a16f34a6 ---> bbc995cd0314 Successfully built bbc995cd0314 Successfully tagged locast2tuner:latest

Running:

[~/docker] @.***% docker run -p 6077:6077 locast2tuner [INFO] - locast2tuner 0.1.0 on Linux 5.4.0-70-generic starting.. [INFO] - UUID: [INFO] - Logging in with [INFO] - Login succeeded! [INFO] - Downloading FCC facilities [INFO] - Cached FCC facilities to /root/.locast2tuner/facilities [INFO] - Loading stations for Dallas (cache: true, cache timeout: 3600, days: 8).. [INFO] - Starting http server for Dallas on http://0.0.0.0:6077 [INFO] - Starting 1 workers [INFO] - Starting "actix-web-service-0.0.0.0:6077" service on 0.0.0.0:6077 [INFO] - Tuners: [INFO] - +--------+----------+-----+--------------------------------------+-----------------+---------------------+ [INFO] - | City | Zip code | DMA | UUID | Timezone | URL | [INFO] - +--------+----------+-----+--------------------------------------+-----------------+---------------------+ [INFO] - | Dallas | | 623 | | America/Chicago | http://0.0.0.0:6077 | [INFO] - +--------+----------+-----+--------------------------------------+-----------------+---------------------+ [INFO] - locast2tuner started.. [INFO] - Stream c9bfabf - ......

It freaking worked on both Plex and Emby.

TODO:

Integrate config file into the Dockerfile

Document the config file format and provide a sample

Document how to run it in Docker with volume/port mapping

Figure out how to package and distribute the binary (it's 150MB), and who owns the build process

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wouterdebie/locast2tuner/issues/17#issuecomment-813775810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABKABSYRRO7LQGWDWRKKGTTHJWITANCNFSM42NYBVQQ .

wouterdebie commented 3 years ago

Another thing we might want to consider is having a reverse mount for the config file, so people don't have to build their own image, but can just download one from a registry.

On Tue, Apr 6, 2021, 06:42 Wouter de Bie @.***> wrote:

Awesome! I'm working on Debian/Ubuntu packaging and using Github actions. We can build everything there. We might want to consider using the .deb file that will be built instead of a separate binary.

On Mon, Apr 5, 2021, 21:25 Roddie Hasan @.***> wrote:

Wow - I did it (using the binary built above).

Dockerfile:

FROM alpine:latest

COPY /locast2tuner /

ENTRYPOINT ["locast2tuner", "-U", "", "-P", "", "-a", "0.0.0.0"]

Building:

[~/docker] @.***% docker build -t locast2tuner . Sending build context to Docker daemon 152.6MB Step 1/2 : FROM alpine:3.7 3.7: Pulling from library/alpine 5d20c808ce19: Pull complete Digest: sha256:8421d9a84432575381bfabd248f1eb56f3aa21d9d7cd2511583c68c9b7511d10 Status: Downloaded newer image for alpine:3.7 ---> 6d1ef012b567 Step 2/2 : ENTRYPOINT ["locast2tuner", "-U", "", "-P", "!", "-a", "0.0.0.0"] ---> Running in 5055a16f34a6 Removing intermediate container 5055a16f34a6 ---> bbc995cd0314 Successfully built bbc995cd0314 Successfully tagged locast2tuner:latest

Running:

[~/docker] @.***% docker run -p 6077:6077 locast2tuner [INFO] - locast2tuner 0.1.0 on Linux 5.4.0-70-generic starting.. [INFO] - UUID: [INFO] - Logging in with [INFO] - Login succeeded! [INFO] - Downloading FCC facilities [INFO] - Cached FCC facilities to /root/.locast2tuner/facilities [INFO] - Loading stations for Dallas (cache: true, cache timeout: 3600, days: 8).. [INFO] - Starting http server for Dallas on http://0.0.0.0:6077 [INFO] - Starting 1 workers [INFO] - Starting "actix-web-service-0.0.0.0:6077" service on 0.0.0.0:6077 [INFO] - Tuners: [INFO] - +--------+----------+-----+--------------------------------------+-----------------+---------------------+ [INFO] - | City | Zip code | DMA | UUID | Timezone | URL | [INFO] - +--------+----------+-----+--------------------------------------+-----------------+---------------------+ [INFO] - | Dallas | | 623 | | America/Chicago | http://0.0.0.0:6077 | [INFO] - +--------+----------+-----+--------------------------------------+-----------------+---------------------+ [INFO] - locast2tuner started.. [INFO] - Stream c9bfabf - ......

It freaking worked on both Plex and Emby.

TODO:

Integrate config file into the Dockerfile

Document the config file format and provide a sample

Document how to run it in Docker with volume/port mapping

Figure out how to package and distribute the binary (it's 150MB), and who owns the build process

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/wouterdebie/locast2tuner/issues/17#issuecomment-813775810, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABKABSYRRO7LQGWDWRKKGTTHJWITANCNFSM42NYBVQQ .

eiddor commented 3 years ago

Another thing we might want to consider is having a reverse mount for the config file, so people don't have to build their own image, but can just download one from a registry.

I was actually talking about the config file for locast2tuner itself that would have the username/password etc. I'd rather users didn't have to build their own image, as well.

Awesome! I'm working on Debian/Ubuntu packaging and using Github actions. We can build everything there. We might want to consider using the .deb file that will be built instead of a separate binary.

Have a look at this project, @wouterdebie: https://github.com/emk/rust-musl-builder

I used this to build the binary and then used the binary to build a docker image that was only ~24MB.

Specifically, look at the "Making static releases with Travis CI and GitHub" section, which may allow you to build the releases you're looking for.

What do you think?

eiddor commented 3 years ago

FYI, the binary built from the emk/rust-musl-builder package is only 18MB vs. 150MB from the one that I used initially.

The Travis CI/GitHub release stuff is way above my head as a non-developer, but I'm willing to learn it and try it out if you think this might be a good idea.

wouterdebie commented 3 years ago

I was actually talking about the config file for locast2tuner itself that would have the username/password etc. I'd rather users didn't have to build their own image, as well.

Yes, exactly. We can have a reverse mount that the image can use and where the config is stored. That way, people don't have to build their own.

Have a look at this project, @wouterdebie: https://github.com/emk/rust-musl-builder

Yeah, I found a bunch of examples in order to do that. I'm experimenting on the cargo_deb branch.

FYI, the binary built from the emk/rust-musl-builder package is only 18MB vs. 150MB from the one that I used initially.

this is the difference between a debug and a release build. A lot of debug stuff is stripped away. The binary that I normally build is around 30 Mb, while a release build only takes up about 3 Mb of space.

The Travis CI/GitHub release stuff is way above my head as a non-developer, but I'm willing to learn it and try it out if you think this might be a good idea.

I'm working on this, but I'll let you know if I need help!

Thanks for the engagement!

eiddor commented 3 years ago

Yes, exactly. We can have a reverse mount that the image can use and where the config is stored. That way, people don't have to build their own.

Hmmm.. I think we're saying the same thing here, but I'll let you figure out how you want to go with builds and then I'll Dockerize, test, and document the workflow.

In the meantime, I am thinking about cleaning up and adding my new Dockerfile to the main branch along with some rough instructions on how to use it. It's got the rust-musl-builder stuff built-in and can get some advanced users started running/testing the project in a container while you work on the build process. What do you think?

Here's what it looks like:

# -*- mode: dockerfile -*-
#
# An example Dockerfile showing how to build a Rust executable using this
# image, and deploy it with a tiny Alpine Linux container.

# You can override this `--build-arg BASE_IMAGE=...` to use different
# version of Rust or OpenSSL.
ARG BASE_IMAGE=ekidd/rust-musl-builder:latest

# Our first FROM statement declares the build environment.
FROM ${BASE_IMAGE} AS builder

# Add our source code.
ADD --chown=rust:rust . ./

# Build our application.
RUN cargo build --release

# Now, we need to build our _real_ Docker container, copying in `locast2tuner`.
FROM alpine:latest

COPY --from=builder \
    /home/rust/src/target/x86_64-unknown-linux-musl/release/locast2tuner \
    /usr/local/bin/
ENTRYPOINT ["/usr/local/bin/locast2tuner", "-U", "<username>", "-P", "<password>", "-a", "0.0.0.0"]

I'll color it up slightly and add a config file option (with documentation).

this is the difference between a debug and a release build. A lot of debug stuff is stripped away. The binary that I normally build is around 30 Mb, while a release build only takes up about 3 Mb of space.

Oh, duh - that makes sense. I told you that I'm way out of my depth here, right? I am so not a developer or coder. :-)

eiddor commented 3 years ago

Ok, I think it's ready to go - I haven't updated README.md yet, but will do that if you think this is worth it as a temporary step.

https://github.com/eiddor/locast2tuner/tree/rough_docker

Basically the user has to do three things (to be documented):

1) Build the container image with: docker build -t locast2tuner . 2) Copy config/config.ini.sample to config/config.ini and edit the username/password 3) Run it either with: docker-compose (sample included) or docker run -p 6077:6077 -v ./config/:/app/config -d locast2tuner

It's obviously a little rough and would rely on me getting the documentation done, but it works well on my end!

TheSamDickey commented 3 years ago

Would it be possible to have the dockerfile download straight from the github repo so that the user doesn't have to download it? Also- there is a way for dockerhub to automatically build whenever there is an update on github, it'd be super cool if that worked too

eiddor commented 3 years ago

@TheSamDickey I think an image on Docker hub where we want to end up eventually (unless @wouterdebie disagrees).

What I've got going right now is just sometime to tide Docker users over in the meantime. I'm actually going to rewrite it tonight or tomorrow to build using the new packages that he's just released. That way there's no downloading involved beyond the Dockerfile.

wouterdebie commented 3 years ago

In got Debian packages working today and will do docker next, so we might have this soon. I'd like to host this on ghcr.io, where I already host another docker image.

On Wed, Apr 7, 2021, 20:29 Roddie Hasan @.***> wrote:

@TheSamDickey https://github.com/TheSamDickey I think an image on Docker hub where we want to end up eventually (unless @wouterdebie https://github.com/wouterdebie disagrees).

What I've got going right now is just sometime to tide Docker users over in the meantime. I'm actually going to rewrite it tonight or tomorrow to build using the new packages that he's just released. That way there's no downloading involved beyond the Dockerfile.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wouterdebie/locast2tuner/issues/17#issuecomment-815380433, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABKABSH6V552AULDL3BRE3THUBGVANCNFSM42NYBVQQ .

eiddor commented 3 years ago

I actually just sat down to write the Dockerfile using your Debian packages - Do you want me to get that done? I would kind of appreciate the practice and I should have it working tonight.

(ghcr.io is great)

wouterdebie commented 3 years ago

Sure thing! I'll have to see how this integrates with building the packages. On option is to use apt-get install the packages from the Debian PPA when building the docker image. However, we could also use the artifact that is created during the build step and install it in the image directly.

Regardless, having a proper Dockerfile is a great first step.

You might want to also have a look at .github/actions/cicd.yml to see how docker images are built on GH actions.

On Wed, Apr 7, 2021, 21:54 Roddie Hasan @.***> wrote:

I actually just sat down to write the Dockerfile using your Debian packages - Do you want me to get that done? I would kind of appreciate the practice and I should have it working tonight.

(ghcr.io is great)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/wouterdebie/locast2tuner/issues/17#issuecomment-815407793, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABKABTSKON4X6O44FOHWUDTHULHZANCNFSM42NYBVQQ .

eiddor commented 3 years ago

Excellent, thank you! I'm going to start with an Ubuntu image and basically follow what you've already documented. The good news is that I've already got the user documentation done since it's pretty much the same as the one I built earlier.

After I get it working I'll start learning more about what you're doing with CI and Actions - From what I understand so far, that would probably be a good place to begin using the binary from the build.

I've learned a few new things already just doing some research to get the Cargo Dockerfile built, so this is a nice break from my pure networking day job.

eiddor commented 3 years ago

PR submitted.

Another good learning experience - For now installing the packages into the Ubuntu 20.04 base image works well. I've tested it on a couple of machines and am running it on my PMS.

As bonus when we move this to the GH Actions process It can be a bit smaller if we build the binary and use the alpine image as a base (the binary works well in alpine). The alternative is to base the image on Ubuntu and install from the deb artifact.

I'm up until 3 AM reading. This stuff is kind of cool.

wouterdebie commented 3 years ago

@eiddor your contributions have been super helpful. I just added building and publishing a Docker image from CI/CD and added a bunch of stuff in the README. Let me know what you think.

eiddor commented 3 years ago

Ha! You did it exactly the way I was about to do it. Well done!

I'm glad to know that I wasn't completely off-base with my understanding of the system, too. This is almost exactly what I had in my head.

I really appreciate the learning experience. I had no idea what CI/CD or GitHub Actions were until late last night. I'll never be a developer or creator, but some of these tools are really cool.

I'll update the docker-compose.yml file to reflect the config location to .locast2tuner to match the docs

PMS also has XMLTV support, so I'll update that in the README.

After that I'll work on prettying-up the Plex and EMBY installation section with screenshots if you think that's a good idea.

wouterdebie commented 3 years ago

@eiddor Thanks!

I'll update the docker-compose.yml file to reflect the config location to .locast2tuner to match the docs

Awesome! I have the feeling that people who know how to use docker-compose will probably know they need to make sure paths are correct. However, it's probably a good idea to explicitly mention that.

PMS also has XMLTV support, so I'll update that in the README.

True! Thanks!

After that I'll work on prettying-up the Plex and EMBY installation section with screenshots if you think that's a good idea.

Awesome! Do you think we should do this in the README or move this to the Wiki perhaps and link in the Wiki? Or maybe add additional markdown files to the repo in a docs/ directory?

wouterdebie commented 3 years ago

Moving the documentation discussion to #26

wouterdebie / locast2tuner

locast2tuner in Docker - WIP #17