search

jbogatay / docker-vpn

Dockerfile for openvpn, socks5 proxy, deluge, and privateinternetaccess port forwarding.
MIT License
7 stars 3 forks source link

Help with config #1

Closed mattfox27 closed 9 years ago

mattfox27 commented 9 years ago

Hi, Im new to docker but not new to linux, im a tad bit confused about what to do, your instructions are very clear i just want to make sure im doing this right.

So I downloaded and installed docker in ubuntu 14.04 then do I git pull the files, then I adjust the variables in the makevars file? on the host subnet that is my subnet that my computer is connected to right? On the dockerGW is that just a IP that docker generates to keep seperate than main subnet? Or do i just put that as my normal gateway/router?

Then on deluge and torrent path would I just leave that as it is or does docker actually install deluge, Or do i install deluge and openvpn seperatly and docker connects them?

thanks so much!!

mattfox27 commented 9 years ago

OK so i kinda played with it by just cloning it into vpn-container then just running sudo make, it made everything and all looked good but when i went to do make run or anything else i get this error below, showing it can't find "vpn" I know docker is a bit tricky and this is my first real go at it..

matt@XBMC:~/docker-vpn$ make run sudo "/usr/bin/docker" kill vpn || true Error response from daemon: No such container: vpn 2014/10/07 00:30:51 Error: failed to kill one or more containers sudo "/usr/bin/docker" rm -v vpn || true Error response from daemon: No such container: vpn 2014/10/07 00:30:51 Error: failed to remove one or more containers sudo "/usr/bin/docker" run -d --restart=always --name vpn --dns=8.8.8.8 --cap-add=NET_ADMIN -p 1080:1080 -p 8112:8112 -v /delugepathonhost:/config/deluge -v /torrentpathonhost:/torrents -v /etc/localtime:/etc/localtime:ro jeff/vpn flag provided but not defined: --restart

Usage: docker run [OPTIONS] IMAGE [COMMAND] [ARG...]

Run a command in a new container

-a, --attach=[] Attach to stdin, stdout or stderr. -c, --cpu-shares=0 CPU shares (relative weight) --cidfile="" Write the container ID to the file --cpuset="" CPUs in which to allow execution (0-3, 0,1) -d, --detach=false Detached mode: Run container in the background, print new container id --dns=[] Set custom dns servers --dns-search=[] Set custom dns search domains -e, --env=[] Set environment variables --entrypoint="" Overwrite the default entrypoint of the image --env-file=[] Read in a line delimited file of ENV variables --expose=[] Expose a port from the container without publishing it to your host -h, --hostname="" Container host name -i, --interactive=false Keep stdin open even if not attached --link=[] Add link to another container (name:alias) --lxc-conf=[](lxc exec-driver only) Add custom lxc options --lxc-conf="lxc.cgroup.cpuset.cpus = 0,1" -m, --memory="" Memory limit (format: , where unit = b, k, m or g) --name="" Assign a name to the container --net="bridge" Set the Network mode for the container 'bridge': creates a new network stack for the container on the docker bridge 'none': no networking for this container 'container:<name|id>': reuses another container network stack 'host': use the host network stack inside the contaner. Note: the host mode gives the container full access to local system services such as D-bus and is therefore considered insecure. -P, --publish-all=false Publish all exposed ports to the host interfaces -p, --publish=[] Publish a container's port to the host format: ip:hostPort:containerPort | ip::containerPort | hostPort:containerPort (use 'docker port' to see the actual mapping) --privileged=false Give extended privileges to this container --rm=false Automatically remove the container when it exits (incompatible with -d) --sig-proxy=true Proxify all received signal to the process (even in non-tty mode) -t, --tty=false Allocate a pseudo-tty -u, --user="" Username or UID -v, --volume=[] Bind mount a volume (e.g. from the host: -v /host:/container, from docker: -v /container) --volumes-from=[] Mount volumes from the specified container(s) -w, --workdir="" Working directory inside the container make: *\ [run] Error 2 matt@XBMC:~/docker-vpn$

jbogatay commented 9 years ago

With respect to your second post. Make sure you are running the latest version of docker. The --restart flag was added in 1.2. The one in the ubuntu repositories is old. Refer to this page: https://docs.docker.com/installation/ubuntulinux/

The docker team provides a script to install the latest in ubuntu.

curl -sSL https://get.docker.io/ubuntu/ | sudo sh
jbogatay commented 9 years ago

Keep in mind, everything in the docs that says "host" refers to the computer that RUNS docker.

mattfox27 commented 9 years ago

I get this error when compiling, does this matter? Setting up openvpn (2.3.2-7ubuntu3) ... invoke-rc.d: action cond-restart is unknown, but proceeding anyway. invoke-rc.d: policy-rc.d denied execution of cond-restart.

mattfox27 commented 9 years ago

Sorry for so many questions, I have been trying to get something like this to work for a while now and before i was using a VM for this but it was too resource intensive and kept crashing. OK so i got the docker to install and run but when i goto localhost:8112 or 1080 I get nothing. This is running on a fresh install of ubuntu 14.04 desktop x64. Is there any pre-req setup that needs to be done for docker? I got it to ver 1.2 but its running the default docker0 ip and i have not configured anything really. Is there any info I can give you to help set this up?

This is output of make run...

matt@XBMC:~/docker-vpn$ sudo make run sudo "/usr/bin/docker" kill vpn || true vpn sudo "/usr/bin/docker" rm -v vpn || true vpn sudo "/usr/bin/docker" run -d --restart=always --name vpn --dns=8.8.8.8 --cap-add=NET_ADMIN -p 1080:1080 -p 8112:8112 -v /home/matt:/config/deluge -v /home/matt:/torrents -v /etc/localtime:/etc/localtime:ro matt/vpn b066b3b4a2bfab83bf8f24a72772419cdb82dc6876500720e9c411d72303f243 matt@XBMC:~/docker-vpn$

jbogatay commented 9 years ago

The errors above are standard docker stuff, the run looks ok. I assume it's running (docker ps).

I have to assume it's the routing entries in your makefile. If the deluge web isn't properly routed -- you wont see the web ui. Port 1080 is for socks, you can't just visit it -- you would setup your browser to use that as a proxy for browsing over the vpn.

I wouldn't put both the deluge config and the torrents in the same directory. Deluge config stores settings, states. Torrent directory stores torrents/data.

What does a ifconfig say for your local network and docker interface? What are the entries in your Makefile (minus your PIA password of course)

mattfox27 commented 9 years ago

Do you need to edit the makefile too or just Makevars file? Here is the output you requested below, I was thinking it was some routing issue as well just wasn't sure where to go from here...thanks so much...

Here is docker ps output.. 74df63ea374a xbmc/vpn:latest "/sbin/my_init" 19 hours ago Up 19 hours 0.0.0.0:1080->1080/tcp, 0.0.0.0:8112->8112/tcp vpn

Ifconfig... matt@XBMC:~/docker-vpn$ ifconfig docker0 Link encap:Ethernet HWaddr 56:84:7a:fe:97:99
inet addr:172.17.42.1 Bcast:0.0.0.0 Mask:255.255.0.0 inet6 addr: fe80::5484:7aff:fefe:9799/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:11604 errors:0 dropped:0 overruns:0 frame:0 TX packets:12070 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:792133 (792.1 KB) TX bytes:1099693 (1.0 MB)

eth0 Link encap:Ethernet HWaddr 00:1e:0b:b2:bf:a5
inet addr:192.168.15.110 Bcast:192.168.15.255 Mask:255.255.255.0 inet6 addr: fe80::21e:bff:feb2:bfa5/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:247766 errors:0 dropped:0 overruns:0 frame:0 TX packets:37195 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:36010107 (36.0 MB) TX bytes:4325946 (4.3 MB) Interrupt:19 Memory:f3100000-f3120000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:65536 Metric:1 RX packets:4250 errors:0 dropped:0 overruns:0 frame:0 TX packets:4250 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:2222236 (2.2 MB) TX bytes:2222236 (2.2 MB)

vethe816 Link encap:Ethernet HWaddr ee:11:86:b9:2b:43
inet6 addr: fe80::ec11:86ff:feb9:2b43/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:10712 errors:0 dropped:0 overruns:0 frame:0 TX packets:10750 errors:0 dropped:1 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:866977 (866.9 KB) TX bytes:939338 (939.3 KB)

matt@XBMC:~/docker-vpn$

Makevars file... DOCKER := "/usr/bin/docker" IMAGE := xbmc/vpn CONTAINER := vpn DNS := 8.8.8.8 DELUGE_PORT := 8112 SOCKS_PORT := 1080 DELUGE_PATH := /docker1 TORRENT_PATH := /docker1 PIA_USER := PIA_PASSWORD := PIA_GATEWAY := nl.privateinternetaccess.com HOST_SUBNET := 192.168.15.0/24 DOCKER_GW := 172.17.42.1 TORRENT_UID := 500 TORRENT_GID := 500 #########################################

End of user configuration.

#########################################

jbogatay commented 9 years ago

That all looks good to me. Mine is slightly different because I don't run ubuntu. Ubuntu's packages are waaaay to old for my personal use ;) But I run this constantly, so much easier than split routing.

You should see the deluge webui at http://192.168.15.110:8112

Random ideas:

I use nsenter to jump into the container and see whats going on. https://github.com/phusion/baseimage-docker#login_nsenter

If you nenter into the container, you should be able to ping the host. That will verify routing is right. 

ping 192.168.15.110
mattfox27 commented 9 years ago

OK, let me give it a try...

Whats the best way to just delete the whole container and makefile and recompile it?

Just do like “docker rm vpn” Then delete directory and re download from github? I want to start fresh but every time I delete the container and try to remake it, it doesn’t seem like it starts from scratch, it doesn’t re-download deluge and go through all the compiling it did the first time.

From: jbogatay Sent: Thursday, October 9, 2014 9:34 AM To: jbogatay/docker-vpn Cc: Matt Klimmer Subject: Re: [docker-vpn] Help with config (#1)

That all looks good to me. Mine is slightly different because I don't run ubuntu. Ubuntu's packages are waaaay to old for my personal use ;) But I run this constantly, so much easier than split routing.

You should see the deluge webui at http://192.168.15.110:8112

Random ideas:

a.. run "netstat -nap --tcp" on the host and make sure it's listening on port 8112. b.. run "docker logs " to view the runtime logs of the container. c.. run "curl localhost:8112" from the host and see if there is a response from deluge. I use nsenter to jump into the container and see whats going on. https://github.com/phusion/baseimage-docker#login_nsenter

If you nenter into the container, you should be able to ping the host. That will verify routing is right.

ping 192.168.15.110 — Reply to this email directly or view it on GitHub.

jbogatay commented 9 years ago

"Make clean" will delete the final image, but if you want to start over, over -- here is a quick and dirty way to clear everything out. First, make sure no containers are running.

This will delete ALL non running containers. Make sure the container is stopped first.

docker rm $(docker ps -aq)

Then you can delete the images as well. This will delete ALL images that aren't in running containers

docker rmi $(docker images -aq)

These are non-finessed brute force type deletes. But unless you have other docker containers running -- this will get the job done easy. (It's brute force because it just attempts to delete everything, running or not. It will fail on running images, but they will be untagged)

You can leave the git clone as is, and just redo make at this point.

mattfox27 commented 9 years ago

Thank you so much for all the help I really appreciate it. In theory should this pretty much work out of the box so to speak besides the makevars file on a fresh Ubuntu install? Or is there some routing that needs to be done to? On Oct 9, 2014 10:06 AM, "jbogatay" notifications@github.com wrote:

"Make clean" will delete the final image, but if you want to start over, over -- here is a quick and dirty way to clear everything out. First, make sure no containers are running.

This will delete ALL non running containers. Make sure the container is stopped first.

docker rm $(docker ps -aq)

Then you can delete the images as well. This will delete ALL images that aren't in running containers

docker rmi $(docker images -aq)

These are non-finessed brute force type deletes. But unless you have other docker containers running -- this will get the job done easy. (It's brute force because it just attempts to delete everything, running or not. It will fail on running images, but they will be untagged)

You can leave the git clone as is, and just redo make at this point.

— Reply to this email directly or view it on GitHub https://github.com/jbogatay/docker-vpn/issues/1#issuecomment-58542576.

mattfox27 commented 9 years ago

Could it be a permission issue? Its not making and deluge config files anywhere, is that because deluge hasn't been run yet? Basically this is a fresh ubuntu 14.04 install, and I ran the script from docker to install the latest version. I tried completely removing all images/containers and reinstalling under root instead of just sudo to see if that does anything and still nothing. Its wierd because when i run docker ps it shows it listening on 8112 but i can't bring up deluge web no mater what. I installed a whole new browser, chromium. cleared all caches and still can't get it to work. Is there anything else that needs to be done to docker as far as users or permissions? I did the symlink that docker says to do in there install guide too.

thanks,

Matt

mattfox27 commented 9 years ago

OK so i got it to work but I had to go into the docker container and manully start deluge-web with nenter. Is that how its supposed to work? Or should everything start automatically when you start the docker with start_container.sh or make run?

Basically im trying to make this auto start and run on every reboot for a headless HTPC build, is this something that will work or does it pretty much reset itsself on every reboot?

jbogatay commented 9 years ago

No, everything should auto-start. I run it on a headless server myself and everything starts properly.

I was going to suggest checking the deluge log files, but on mine they are empty. Not sure why, if I have a chance tomorrow I'll take a look. I'm going to try to make the services log to stdout, that way you can inspect the logs via "docker logs" instead of nsentering into the container.

How did you start deluge-web? Is /etc/service/deluge-web/run executable?

mattfox27 commented 9 years ago

I started deluge web by using that nenter utility you suggested then ssh'd into that actual container, then I just typed deluged then deluge-web just like you would manually start it regularly. I saw the run files and the start files so I figured it should auto start. Oh man if you can help me o would so appreciate it this is the one thing I have dreamed for my htpc setup. I really appreciate the help.

Matt On Oct 11, 2014 9:44 AM, "jbogatay" notifications@github.com wrote:

No, everything should auto-start. I run it on a headless server myself and everything starts properly.

I was going to suggest checking the deluge log files, but on mine they are empty. Not sure why, if I have a chance tomorrow I'll take a look. I'm going to try to make the services log to stdout, that way you can inspect the logs via "docker logs" instead of nsentering into the container.

How did you start deluge-web? Is /etc/service/deluge-web/run executable?

— Reply to this email directly or view it on GitHub https://github.com/jbogatay/docker-vpn/issues/1#issuecomment-58755886.

jbogatay commented 9 years ago

I understand, I've been wanting to cleanly partition vpn traffic for a while, This is exactly what I wanted.

I pushed a few changes to help try and troubleshoot this.

Now after you fire the container up, you can see what is happening by running

docker logs -f <containername>

See if that helps at all.

mattfox27 commented 9 years ago

Awesome...thanks so much as soon as I get home I will fire it up and see what happens, thanks much!!! On Oct 12, 2014 8:49 AM, "jbogatay" notifications@github.com wrote:

I understand, I've been wanting to cleanly partition vpn traffic for a while, This is exactly what I wanted.

I pushed a few changes to help try and troubleshoot this.

-

I made all services (openvpn, deluged, and deluge-web) log to stdout instead of seperate logfiles and bumped the verbosity to info.

I added a line to the Dockerfile that makes double sure the services and scripts are executable.

Now after you fire the container up, you can see what is happening by running

docker logs -f

See if that helps at all.

— Reply to this email directly or view it on GitHub https://github.com/jbogatay/docker-vpn/issues/1#issuecomment-58805903.

mattfox27 commented 9 years ago

What linux are you running?

mattfox27 commented 9 years ago

IT worked!!! I deleted everything and started from scratch with your updates and after make run I went to MYip:8112 and right away deluge came up...so sweet. OK so does the PIA port script work on its own or is there any config for that? IM assuming you have to uncheck use random ports, I have used that script before. Also whats the best way to get this docker container to start on boot?

Does deluge daemon also start with this? Im trying to get couchpotato to work and its having a hard time connecting to deluge.

jbogatay commented 9 years ago

I run Archlinux. Ubuntu is great for servers but for my home server and development I don't like being so far behind.

PIA port script works automatically, you should see it doing it's thing in the log every hour (55 minutes technically)

The PIA port script will auto uncheck use random ports for you, set the correct port, and bind to the vpn interface.

Deluge daemon has to start for the webui to be... useful ;) (and it does)

The --restart=always should restart it on reboot. If not, you'll have to whip up a quick init script ubu-style.

Couchpotato -> deluge is a different thing. The only thing published outside of the container is the webui port. If you want something to actually connect to the daemon, you will have to publish that port.

Personally, I'd just use the black box style and point couchpotato to the torrent directories.

I run couchpotato in a container as well, but I use nzb's and not torrents right now so I don't have them wired together.

Glad you got it working.

mattfox27 commented 9 years ago

Ya that's what I was thinking just doing the black hole for deluge. Ya I should have looked at the settings before I asked about the port forwarding as soon as I sent that I looked and saw the port forwarding script was working. Before I had to run it manually. So if I wanted to publish the deluge port I would just add 58466:58466 to the list of ports in start_container.sh? I really really appreciate all your help my media server crashes a couple of months ago because of using a VM for VPN and torrents so this really helps me out a lot. I really appreciate all your hard work and help!!!

Thanks,

Matt On Oct 13, 2014 6:41 AM, "jbogatay" notifications@github.com wrote:

I run Archlinux. Ubuntu is great for servers but for my home server and development I don't like being so far behind.

PIA port script works automatically, you should see it doing it's thing in the log every hour (55 minutes technically)

The PIA port script will auto uncheck use random ports for you, set the correct port, and bind to the vpn interface.

Deluge daemon has to start for the webui to be... useful ;) (and it does)

The --restart=always should restart it on reboot. If not, you'll have to whip up a quick init script ubu-style.

Couchpotato -> deluge is a different thing. The only thing published outside of the container is the webui port. If you want something to actually connect to the daemon, you will have to publish that port.

Personally, I'd just use the black box style and point couchpotato to the torrent directories.

I run couchpotato in a container as well, but I use nzb's and not torrents right now so I don't have them wired together.

Glad you got it working.

— Reply to this email directly or view it on GitHub https://github.com/jbogatay/docker-vpn/issues/1#issuecomment-58893345.

jbogatay commented 9 years ago

I suspect in your earlier efforts either:

  1. openvpn wasn't connecting right (since the downstream services require it to be running) or
  2. Somehow the executable bits on the various "run" services were not set.

Not sure, either way now you can view all logs and see what's happening.

If you want to publish port 58466, edit the Dockerfile and put EXPOSE 58466, then add it to start_container.sh. ( -p 58466:58466)

Some other tips...

The phusion baseimage uses runit to manage services http://smarden.org/runit/

mattfox27 commented 9 years ago

Do you publish the port in the dockerfile or Makrvars file? Also how would you use the socks5 proxy? Does it point to the VPN gateway? Do you use it just like any proxy in deluge settings?

From: jbogatay Sent: Monday, October 13, 2014 1:04 PM To: jbogatay/docker-vpn Cc: Matt Klimmer Subject: Re: [docker-vpn] Help with config (#1)

I suspect in your earlier efforts either:

1.. openvpn wasn't connecting right (since the downstream services require it to be running) or 2.. Somehow the executable bits on the various "run" services were not set. Not sure, either way now you can view all logs and see what's happening.

If you want to publish port 58466, edit the Dockerfile and put EXPOSE 58466, then add it to start_container.sh. ( -p 58466:58466)

Some other tips...

a.. Setup another browser, make it always "private mode" and set the socks5 proxy to your server. On demand anonymous/vpnized browsing.

b.. I find if my home internet blips out, the vpn doesn't always reset cleanly. It's at the good enough point for me so I haven't spent a lot of time on it. I suspect changing the vpn config to use ip addresses instead of names would help, just haven't got around to it yet. Anyhow, if you need to restart the vpn connection you can either restart the whole container, or... nsenter into the container and restart it manually:

sv restart openvpn

The phusion baseimage uses runit to manage services http://smarden.org/runit/

— Reply to this email directly or view it on GitHub.

jbogatay commented 9 years ago

Publish the port in the Dockerfile to expose it. And in start_container.sh to map it to host. Just point your browser to servername:1080 as a socksv5 proxy and go. This is for browsing, deluge is already good to go.

mattfox27 commented 9 years ago

OK so it mapped the port properly 58846 but CP still can't connect, Not a huge issue I can use the blackhole but I love to get it working.

mattfox27 commented 9 years ago

I got it, it was the auth file i had to add a username and pass...sweet

mattfox27 commented 9 years ago

How can i change some of the openvpn startup variables? I want to try and make openvpn a little more easy to establish vpn maybe? I changed my vpn mapping to PIA's canada server and just used the IP because like you mentioned it sometimes drops, before when I used the IP it would never drop. Is there a specific reason you used NL server? Other than port forwarding? Like did it work better or something for you verses the CA server?

jbogatay commented 9 years ago

The makefile is just a convenience to set things up for the docker build.

The openvpn config is here https://github.com/jbogatay/docker-vpn/blob/master/config/openvpn/default.conf

You don't have to use the make system if you don't want to. Change whatever you want and just run

docker build -t {containername} .

I just used a makefile so I didn't have to document all the settings in all the different files.

mattfox27 commented 9 years ago

Oh ok, so you can just make your changes and do docker build instead of having to recompile the whole thing?

jbogatay commented 9 years ago

I think in your terms... build=compile. You build an image and run it.

mattfox27 commented 9 years ago

would i just do docker build -t vpn or do i have to enter all the arguments, ports ect? When i just do docker build -t vpn it doesn't go

jbogatay commented 9 years ago 
docker build -t <myname>/vpn .

The dot at the end is important, you need to pass a path to the Dockerfile.

mattfox27 commented 9 years ago

Do you ever have issues with deluge not moving completed downloads to there proper directory?

jbogatay commented 9 years ago

No, never. Make sure your permissions are correct. The gid/uid specified in the container should also exist on the server and own all the relavent directories on the host.

mattfox27 commented 9 years ago

When i try to set the GID when i run make it says that it already exists, for instance I set them both to 1000 which is my user ID but when I hit make it says that GID already exists. Is that OK or should it have its own GID or does it use an existing one?

jbogatay commented 9 years ago

They just need to be the same on both sides, however you achieve that.