search

ipfs-shipyard / cube

IPFS Cube will help people deploy and manage their own IPFS pinning services on top of existing cheap hardware, or cloud storage.
32 stars 3 forks source link

Cube fails to recognize that it has started docker instances #14

Open lanzafame opened 5 years ago

lanzafame commented 5 years ago

Cube keeps starting docker instances of ipfs and ipfs-cluster and it successfully starts the containers, but it failes to connect to them and after timing out, it starts creating another set of instances.

Creating 3 new instances
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 1
Feb 11, 2019 12:54:45 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:45 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 2
Feb 11, 2019 12:54:46 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:46 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 3
Feb 11, 2019 12:54:48 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:48 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 4
Feb 11, 2019 12:54:49 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:49 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 5
Feb 11, 2019 12:54:51 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:51 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 6
Feb 11, 2019 12:54:52 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:52 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 7
Feb 11, 2019 12:54:54 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:54 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 8
Feb 11, 2019 12:54:55 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:55 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 9
Feb 11, 2019 12:54:57 PM org.apache.http.impl.execchain.RetryExec execute
INFO: I/O exception (org.apache.http.conn.ConnectTimeoutException) caught when processing request to {}->http://172.19.0.7:9094: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out
Feb 11, 2019 12:54:57 PM org.apache.http.impl.execchain.RetryExec execute
INFO: Retrying request to {}->http://172.19.0.7:9094
Exorg.apache.http.conn.ConnectTimeoutException: Connect to 172.19.0.7:9094 [/172.19.0.7] failed: connect timed out Try: 10
#error {
 :cause "connect timed out"
 :via
 [{:type java.net.SocketTimeoutException
   :message "connect timed out"
   :at [java.net.PlainSocketImpl socketConnect "PlainSocketImpl.java" -2]}]
 :trace
 [[java.net.PlainSocketImpl socketConnect "PlainSocketImpl.java" -2]
  [java.net.AbstractPlainSocketImpl doConnect "AbstractPlainSocketImpl.java" 399]
  [java.net.AbstractPlainSocketImpl connectToAddress "AbstractPlainSocketImpl.java" 242]
  [java.net.AbstractPlainSocketImpl connect "AbstractPlainSocketImpl.java" 224]
  [java.net.SocksSocketImpl connect "SocksSocketImpl.java" 403]
  [java.net.Socket connect "Socket.java" 591]
  [org.apache.http.conn.socket.PlainConnectionSocketFactory connectSocket "PlainConnectionSocketFactory.java" 75]
  [org.apache.http.impl.conn.DefaultHttpClientConnectionOperator connect "DefaultHttpClientConnectionOperator.java" 142]
  [org.apache.http.impl.conn.BasicHttpClientConnectionManager connect "BasicHttpClientConnectionManager.java" 325]
  [org.apache.http.impl.execchain.MainClientExec establishRoute "MainClientExec.java" 381]
  [org.apache.http.impl.execchain.MainClientExec execute "MainClientExec.java" 237]
  [org.apache.http.impl.execchain.ProtocolExec execute "ProtocolExec.java" 185]
  [org.apache.http.impl.execchain.RetryExec execute "RetryExec.java" 89]
  [org.apache.http.impl.execchain.RedirectExec execute "RedirectExec.java" 111]
  [org.apache.http.impl.client.InternalHttpClient doExecute "InternalHttpClient.java" 185]
  [org.apache.http.impl.client.CloseableHttpClient execute "CloseableHttpClient.java" 83]
  [clj_http.core$request invokeStatic "core.clj" 626]
  [clj_http.core$request invoke "core.clj" 554]
  [clj_http.core$request invokeStatic "core.clj" 555]
  [clj_http.core$request invoke "core.clj" 554]
  [clojure.lang.Var invoke "Var.java" 384]
  [clj_http.client$wrap_request_timing$fn__16839 invoke "client.clj" 1085]
  [clj_http.headers$wrap_header_map$fn__15182 invoke "headers.clj" 147]
  [clj_http.client$wrap_query_params$fn__16729 invoke "client.clj" 819]
  [clj_http.client$wrap_basic_auth$fn__16735 invoke "client.clj" 842]
  [clj_http.client$wrap_oauth$fn__16740 invoke "client.clj" 859]
  [clj_http.client$wrap_user_info$fn__16749 invoke "client.clj" 879]
  [clj_http.client$wrap_url$fn__16821 invoke "client.clj" 1037]
  [clj_http.client$wrap_decompression$fn__16538 invoke "client.clj" 420]
  [clj_http.client$wrap_input_coercion$fn__16653 invoke "client.clj" 639]
  [clj_http.client$wrap_additional_header_parsing$fn__16678 invoke "client.clj" 694]
  [clj_http.client$wrap_output_coercion$fn__16640 invoke "client.clj" 583]
  [clj_http.client$wrap_exceptions$fn__16491 invoke "client.clj" 254]
  [clj_http.client$wrap_accept$fn__16693 invoke "client.clj" 737]
  [clj_http.client$wrap_accept_encoding$fn__16700 invoke "client.clj" 759]
  [clj_http.client$wrap_content_type$fn__16687 invoke "client.clj" 720]
  [clj_http.client$wrap_form_params$fn__16786 invoke "client.clj" 961]
  [clj_http.client$wrap_nested_params$fn__16807 invoke "client.clj" 995]
  [clj_http.client$wrap_flatten_nested_params$fn__16816 invoke "client.clj" 1019]
  [clj_http.client$wrap_method$fn__16754 invoke "client.clj" 895]
  [clj_http.cookies$wrap_cookies$fn__14557 invoke "cookies.clj" 131]
  [clj_http.links$wrap_links$fn__15514 invoke "links.clj" 63]
  [clj_http.client$wrap_unknown_host$fn__16824 invoke "client.clj" 1048]
  [clj_http.client$request_STAR_ invokeStatic "client.clj" 1176]
  [clj_http.client$get invokeStatic "client.clj" 1181]
  [cube.providers.docker$get_retry invokeStatic "docker.clj" 28]
  [cube.providers.docker$get_api_multiaddr invokeStatic "docker.clj" 39]
  [cube.providers.docker$create_id invokeStatic "docker.clj" 58]
  [cube.providers.docker$save_instance invokeStatic "docker.clj" 68]
  [cube.providers.docker$create invokeStatic "docker.clj" 96]
  [cube.instances$create_instances invokeStatic "instances.clj" 14]
  [cube.instances$check_instances invokeStatic "instances.clj" 34]
  [cube.instances.Instances$fn__17023 invoke "instances.clj" 53]
  [cube.scheduler$do_interval invokeStatic "scheduler.clj" 12]
  [cube.scheduler.Scheduler$fn__932 invoke "scheduler.clj" 20]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 665]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.RestFn applyTo "RestFn.java" 132]
  [clojure.core$apply invokeStatic "core.clj" 669]
  [clojure.core$bound_fn_STAR_$fn__5734 doInvoke "core.clj" 2003]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [tea_time.core.Every run "core.clj" 194]
  [tea_time.core$run_tasks_BANG_$fn__886$fn__887 invoke "core.clj" 261]
  [tea_time.core$run_tasks_BANG_$fn__886 invoke "core.clj" 260]
  [tea_time.core$run_tasks_BANG_ invokeStatic "core.clj" 250]
  [tea_time.core$start_BANG_$fn__906$fn__907 invoke "core.clj" 311]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 665]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.RestFn applyTo "RestFn.java" 132]
  [clojure.core$apply invokeStatic "core.clj" 669]
  [clojure.core$bound_fn_STAR_$fn__5734 doInvoke "core.clj" 2003]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 834]]}
lanzafame commented 5 years ago

It seems to have something to do with having a VPN running (Mullvad OpenVPN). The VPN seems to cause issues with the frontends ability to communicate to the backend as well as the backends ability to communicate with the docker daemon and then hit the APIs of the ipfs and ipfs-cluster containers.

victorb commented 5 years ago

Many thanks for the report! That it mucks with localhost connections are a bit weird (the web ui <> cube server communication) but the rest of the connections make sense to be VPNed.

Not exactly sure if/how we could fix this automatically. Exposing settings for where/how communication happens between the different pieces would solve the problem, but manually for users.