Dockerfile and instructions

orbitalturtle commented 11 months ago

Thanks to @anmode and @tsjk for starting this Dockerfile work in PR #197 . I just made a couple of changes to get it working on my end, including adding the /.teos directory to the teosd group (otherwise I was getting a permissions error when running the container).

It seems that by only copying over the specific binaries we need (teos-cli and teosd) rather than the full /target folder reduced the build to ~100 MB.
Changing from debian to alpine further reduced the build to ~40 MB

For the tutorials, thanks to @sr-gi, who I believe wrote the original Teos python Docker documentation much of which I borrowed in this PR. (Though I removed the mention of creating a volume since we automatically do that in the current Dockerfile. Thoughts?) I've added a section for how to set up a Tor hidden service pointing to the Docker instance.

Next steps:

One thing that was discussed in PR #197 is to explore how to allow someone to migrate an existing Teos instance to a container. Some changes will be needed to make the existing Tor key compatible and instructions for how to do this need to be added.

(I wrote the Tor Docker instructions after attempting to follow the discussion in #197 . If I got anything wrong, feel free to let me know.)

mariocynicys commented 11 months ago

I pushed in here by mistake, but how could I even do it :o

Feel free to force push on my last commit, I have it backed up.

mariocynicys commented 11 months ago

I did some tweaks in 2ad1d4c to use the default data directory in /root, omit making new user to run the tower from, avoid installing bash (not needed actually).

This is the command I run to boot up a dockered teos daemon:

docker run --network host --name teos -v ./datadir:/root/.teos --rm -it teos
## I set all the config options in `datadir/teos.toml`

Notice the -v option to persist the container's datadir in the host FS.

One could use -v with their already populated datadir and have the tower running just fine. but will need to tweak some config settings (shut tor off).

sr-gi commented 11 months ago

I pushed in here by mistake, but how could I even do it :o

Feel free to force push on my last commit, I have it backed up.

You can allow others to add commits to your PR if they push to your repo. How did you push to her fork instead of yours, that I don't know lol

orbitalturtle commented 11 months ago

Thanks for the reviews! Those changes you made in the Dockerfile @mariocynicys seem good to me. Though I disagree with changing it to run as root - I think we should stick with the principle of least privilege, and I believe that's standard practice https://medium.com/@mccode/processes-in-containers-should-not-run-as-root-2feae3f0df3b

I'll finish making the above changes tomorrow ~

mariocynicys commented 11 months ago

Though I disagree with changing it to run as root - I think we should stick with the principle of least privilege, and I believe that's standard practice medium.com/@mccode/processes-in-containers-should-not-run-as-root-2feae3f0df3b

Very informative read. I didn't know that a root inside the container is just like a root process in the host :o. Just tried to use one of the protected ports 0-1023 for the tower and it worked just fine with it being root. This is definitely very scary.

mariocynicys commented 11 months ago

You can allow others to add commits to your PR if they push to your repo. How did you push to her fork instead of yours, that I don't know lol

I add new remotes to review then checkout the branch locally to push it to my remote instead, forgot to do the last part :joy: :joy:

tsjk commented 11 months ago

Why does this need --network=host?

mariocynicys commented 11 months ago

Why does this need --network=host?

running directly on the host network allows us to access bitcoind using localhost and allows us to listen for requests on the host port without the need to publish them.

I think we should actually go the Windows & OSX way for linux as well, to:

Unify how the image is ran over different platforms.
Limit the root access by not allowing the container to use the host network. (I am kinda convinced now that we can keep running as root user if we have no access to the host network nor the host FS).

sr-gi commented 11 months ago

Why does this need --network=host?

This is pretty old, but IIRC it was mainly for convenience. It was easier to setup. The MacOS/Windows section was added later in case someone wanted to do it for those platforms, but we expected most people to be running this in UNIX.

orbitalturtle commented 11 months ago

I finally addressed your comments @mariocynicys and @sr-gi, when you have time for another look. :)

And yes, the tutorial right now assumes you're running Teos in Docker on the same machine as bitcoind. As I eventually get further along with my own Teos Docker setup I'll try to add a guide for someone who might be running bitcoind on a separate raspberry pi, or something, which seems like the more likely real-world scenario.

orbitalturtle commented 11 months ago

@sr-gi Ahhh yup I now remember the lowercase teos naming conventions from the python teos days :p

I haven't tried connecting using the CLN client yet... But re: connecting to teosd from the host computer with teos-cli. It has been working for me without setting RPC_BIND, in linux anyways. Once teosd is running in docker, it should be exposed on localhost from the host. Then I needed to copy the needed tls cert data from the docker /.teos folder to the host's /.teos folder, so that teos-cli could connect w/o authentication issues.

What error are you getting? Just not able to connect? When using docker run, do you have the -p 9814:9814 field set?

I'll give it a try on non-linux OSs in VirtualBox some day soon to see if I can come up with anything that'll help

(Also, made those other changes that you requested above)

orbitalturtle commented 11 months ago

@sr-gi Just confirmed that loading the plugin in CLN and using lightning-cli registertowe to connect to the tower running in Docker works for me too

mariocynicys commented 11 months ago

Was able to use a non-root user with user-provided data directory: https://github.com/mariocynicys/rust-teos/commit/d2b1f31d5ec9e667a190334e1c9476e3e0192efa

Basically, this change chowns the data directory in the container runtime and not the image build time.

sr-gi commented 10 months ago

I need to re-check this on MacOS to see if I'm able to use the CLI/Client from outside

orbitalturtle commented 10 months ago

Thanks again for the reviews y'all. This is definitely looking more battle tested. I addressed those comments... I think the top portion of the README had the most changes since I reorganized it to add an option to copy over the config to Docker, upon feedback from @mariocynicys.

@sr-gi Awesome, glad to hear you got it working on MacOS. I updated the instructions with that new command, if that seems like a good enough fix.

Hoping it's close! 🤞🏻

orbitalturtle commented 9 months ago

@mariocynicys Yay! Made that last remaining change.

sr-gi commented 9 months ago

Re-ACK c3c7e99

mariocynicys commented 9 months ago

ACK c3c7e99

sr-gi commented 9 months ago

Needs rebase

orbitalturtle commented 9 months ago

@sr-gi @mariocynicys just rebased

talaia-labs / rust-teos

Dockerfile and instructions #227