search

ipfs-shipyard / cube

IPFS Cube will help people deploy and manage their own IPFS pinning services on top of existing cheap hardware, or cloud storage.
32 stars 3 forks source link

Cube gets angry if you manually pin to the underlying IPFS Cluster #24

Open lanzafame opened 5 years ago

lanzafame commented 5 years ago 
From:
[[java.lang.Thread getStackTrace "Thread.java" 1606],
 [cube.db$validate_new_state invokeStatic "db.clj" 30],
 [cube.db$validate_new_state invoke "db.clj" 22],
 [clojure.lang.ARef validate "ARef.java" 32],
 [clojure.lang.ARef validate "ARef.java" 46],
 [clojure.lang.Atom swap "Atom.java" 66],
 [clojure.core$swap_BANG_ invokeStatic "core.clj" 2354],
 [cube.db$put invokeStatic "db.clj" 52],
 [cube.cluster$update_pins invokeStatic "cluster.clj" 53],
 [cube.cluster.Cluster$fn__34251 invoke "cluster.clj" 73],
 [cube.scheduler$do_interval invokeStatic "scheduler.clj" 12],
 [cube.scheduler.Scheduler$fn__21582 invoke "scheduler.clj" 20],
 [clojure.lang.AFn applyToHelper "AFn.java" 152],
 [clojure.lang.AFn applyTo "AFn.java" 144],
 [clojure.core$apply invokeStatic "core.clj" 665],
 [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973],
 [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973],
 [clojure.lang.RestFn invoke "RestFn.java" 425],
 [clojure.lang.AFn applyToHelper "AFn.java" 156],
 [clojure.lang.RestFn applyTo "RestFn.java" 132],
 [clojure.core$apply invokeStatic "core.clj" 669],
 [clojure.core$bound_fn_STAR_$fn__5734 doInvoke "core.clj" 2003],
 [clojure.lang.RestFn invoke "RestFn.java" 397],
 [tea_time.core.Every run "core.clj" 194],
 [tea_time.core$run_tasks_BANG_$fn__21530$fn__21531 invoke "core.clj" 261],
 [tea_time.core$run_tasks_BANG_$fn__21530 invoke "core.clj" 260],
 [tea_time.core$run_tasks_BANG_ invokeStatic "core.clj" 250],
 [tea_time.core$start_BANG_$fn__21550$fn__21551 invoke "core.clj" 311],
 [clojure.lang.AFn applyToHelper "AFn.java" 152],
 [clojure.lang.AFn applyTo "AFn.java" 144],
 [clojure.core$apply invokeStatic "core.clj" 665],
 [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973],
 [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973],
 [clojure.lang.RestFn invoke "RestFn.java" 425],
 [clojure.lang.AFn applyToHelper "AFn.java" 156],
 [clojure.lang.RestFn applyTo "RestFn.java" 132],
 [clojure.core$apply invokeStatic "core.clj" 669],
 [clojure.core$bound_fn_STAR_$fn__5734 doInvoke "core.clj" 2003],
 [clojure.lang.RestFn invoke "RestFn.java" 397],
 [clojure.lang.AFn run "AFn.java" 22],
 [java.lang.Thread run "Thread.java" 834]]
#error {
 :cause "Invalid reference state"
 :via
 [{:type java.lang.IllegalStateException
   :message "Invalid reference state"
   :at [clojure.lang.ARef validate "ARef.java" 33]}]
 :trace
 [[clojure.lang.ARef validate "ARef.java" 33]
  [clojure.lang.ARef validate "ARef.java" 46]
  [clojure.lang.Atom swap "Atom.java" 66]
  [clojure.core$swap_BANG_ invokeStatic "core.clj" 2354]
  [cube.db$put invokeStatic "db.clj" 52]
  [cube.cluster$update_pins invokeStatic "cluster.clj" 53]
  [cube.cluster.Cluster$fn__34251 invoke "cluster.clj" 73]
  [cube.scheduler$do_interval invokeStatic "scheduler.clj" 12]
  [cube.scheduler.Scheduler$fn__21582 invoke "scheduler.clj" 20]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 665]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.RestFn applyTo "RestFn.java" 132]
  [clojure.core$apply invokeStatic "core.clj" 669]
  [clojure.core$bound_fn_STAR_$fn__5734 doInvoke "core.clj" 2003]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [tea_time.core.Every run "core.clj" 194]
  [tea_time.core$run_tasks_BANG_$fn__21530$fn__21531 invoke "core.clj" 261]
  [tea_time.core$run_tasks_BANG_$fn__21530 invoke "core.clj" 260]
  [tea_time.core$run_tasks_BANG_ invokeStatic "core.clj" 250]
  [tea_time.core$start_BANG_$fn__21550$fn__21551 invoke "core.clj" 311]
  [clojure.lang.AFn applyToHelper "AFn.java" 152]
  [clojure.lang.AFn applyTo "AFn.java" 144]
  [clojure.core$apply invokeStatic "core.clj" 665]
  [clojure.core$with_bindings_STAR_ invokeStatic "core.clj" 1973]
  [clojure.core$with_bindings_STAR_ doInvoke "core.clj" 1973]
  [clojure.lang.RestFn invoke "RestFn.java" 425]
  [clojure.lang.AFn applyToHelper "AFn.java" 156]
  [clojure.lang.RestFn applyTo "RestFn.java" 132]
  [clojure.core$apply invokeStatic "core.clj" 669]
  [clojure.core$bound_fn_STAR_$fn__5734 doInvoke "core.clj" 2003]
  [clojure.lang.RestFn invoke "RestFn.java" 397]
  [clojure.lang.AFn run "AFn.java" 22]
  [java.lang.Thread run "Thread.java" 834]]}
Explanation:
:remote - failed: #{:pinned :unpin_error :pinning :pin_error} in: [:pins 0 :peer-map 1 :status] at: [:pins :peer-map :status] spec: :cube.pin.peer/status
:remote - failed: #{:pinned :unpin_error :pinning :pin_error} in: [:pins 0 :peer-map 3 :status] at: [:pins :peer-map :status] spec: :cube.pin.peer/status
:remote - failed: #{:pinned :unpin_error :pinning :pin_error} in: [:pins 0 :peer-map 4 :status] at: [:pins :peer-map :status] spec: :cube.pin.peer/status
"" - failed: (complement empty?) in: [:pins 0 :name] at: [:pins :name] spec: :cube.util/not-empty-string
Tried new state:
{:instances
 {:wanted {:docker 5},
  :running
  {:m2RK9hiHlzg
   {:type :docker,
    :metadata
    {:go-ipfs-id "0674e26c9622", :ipfs-cluster-id "3906d963cf1a"},
    :cluster-api "http://172.19.0.15:9094",
    :webui "http://172.19.0.15:9095/webui",
    :ipfs-proxy "http://172.19.0.15:9095"},
   :NUM81cWYI68
   {:type :docker,
    :metadata
    {:go-ipfs-id "bfa58118b7c1", :ipfs-cluster-id "1354af01f29d"},
    :cluster-api "http://172.19.0.17:9094",
    :webui "http://172.19.0.17:9095/webui",
    :ipfs-proxy "http://172.19.0.17:9095"},
   :1jXgasSNKKw
   {:type :docker,
    :metadata
    {:go-ipfs-id "c108d5a67eaa", :ipfs-cluster-id "62ac5c123945"},
    :cluster-api "http://172.19.0.19:9094",
    :webui "http://172.19.0.19:9095/webui",
    :ipfs-proxy "http://172.19.0.19:9095"},
   :9JFXHbssscE
   {:type :docker,
    :metadata
    {:go-ipfs-id "25f33ee7c019", :ipfs-cluster-id "b218fdbb1460"},
    :cluster-api "http://172.19.0.21:9094",
    :webui "http://172.19.0.21:9095/webui",
    :ipfs-proxy "http://172.19.0.21:9095"},
   :2P5Ij5SX4ac
   {:type :docker,
    :metadata
    {:go-ipfs-id "f900532c1c11", :ipfs-cluster-id "4de5116baf11"},
    :cluster-api "http://172.19.0.23:9094",
    :webui "http://172.19.0.23:9095/webui",
    :ipfs-proxy "http://172.19.0.23:9095"}},
  :cluster-secret
  "6d23ad4b2f4e982ee9002e7f72414e81889af9350b99467c78b24531c8b56176"},
 :monitoring
 {:pinsize {:value 0, :timestamp 1550713510},
  :freespace {:value 49999404288, :timestamp 1550713510}},
 :pins
 ({:cid "QmavdEDuwd2LqyAVUk6wjJ1shj3NzymVZ8KsQaLzMh6kZj",
   :peer-map
   [{:status :pinned,
     :timestamp "2019-02-21T01:45:12Z",
     :peer-id "QmPDU7u3UxAM3aN9geSXQDVncHCwwRo6o17Pmo5rPGN5By"}
    {:status :remote,
     :timestamp "2019-02-21T01:45:11Z",
     :peer-id "QmQdqcgTp6rrL5tT3EBFndYY3vyqnrCDX4hpvQWYNPMWXD"}
    {:status :pinned,
     :timestamp "2019-02-21T01:45:12Z",
     :peer-id "QmYfraQRGkZJQyxohpVU4C2gQqE1pEDQzJU1qpnDLx3Vd1"}
    {:status :remote,
     :timestamp "2019-02-21T01:45:11Z",
     :peer-id "QmcV7mQ5KihNDRyu4zUj7iCdAVfRFpKUN3yhKrktUS3NJv"}
    {:status :remote,
     :timestamp "2019-02-21T01:45:11Z",
     :peer-id "QmciggjH75j6yij5VphmYXBNpG5nKZdme72eZfRmYpmu9Q"}],
   :name "",
   :size 19154})}

Because cube believes it's own state of the world is the source of truth, pinning to cluster via ipfs-cluster-ctl causes it spew the above error repeatedly.

victorb commented 5 years ago

The issue is actually simpler than that. 

:remote - failed: #{:pinned :unpin_error :pinning :pin_error} in: [:pins 0 :peer-map 1 :status] at: [:pins :peer-map :status] spec: :cube.pin.peer/status

Indicates that the :status key inside the :peer-map is invalid, it should be one of #{:pinned :unpin_error :pinning :pin_error}. Basically, :remote is not defined as a valid status for a pin to be in.

Solution is simple, add :remote to https://github.com/ipfs-shipyard/cube/blob/master/src/shared/db.cljc#L52-L54 (heh, notice the "TODO" there as well :) )

Seems that's not the only issue either. A pin in Cube is defined like this: https://github.com/ipfs-shipyard/cube/blob/master/src/shared/db.cljc#L91-L94

Which says that :name has to be a string that is not empty. This is not true and a pin in ipfs-cluster could be nil. Also something that would have to be fixed but unrelated to this issue.