replikativ / datahike

A fast, immutable, distributed & compositional Datalog engine for everyone.
https://datahike.io
Eclipse Public License 1.0
1.6k stars 95 forks source link

[Bug]: "0.5.1506" cannot import dump from "0.4.1480" #552

Open awb99 opened 2 years ago

awb99 commented 2 years ago

What version of Datahike are you using?

"0.5.1506"

What version of Java are you using?

java 17.0.3

What operating system are you using?

guix

What database EDN configuration are you using?

{:store {:backend :file :path "data/datahike-db"} :keep-history? false}

Describe the bug

Trying to import a db-dump made with "0.4.1480" gets the following error: The only difference in my project is the change of the datahiek version number.

Preparing import of data/datahike-dump/eavt-dump in batches of 1000
Execution error (ExceptionInfo) at clj-cbor.error/codec-exception! (error.clj:8).
Input data ended while parsing a CBOR value.

{:clojure.main/message
 "Execution error (ExceptionInfo) at clj-cbor.error/codec-exception! (error.clj:8).\nInput data ended while parsing a CBOR value.\n",
 :clojure.main/triage
 {:clojure.error/class clojure.lang.ExceptionInfo,
  :clojure.error/line 8,
  :clojure.error/cause "Input data ended while parsing a CBOR value.",
  :clojure.error/symbol clj-cbor.error/codec-exception!,
  :clojure.error/source "error.clj",
  :clojure.error/phase :execution},
 :clojure.main/trace
 {:via
  [{:type clojure.lang.ExceptionInfo,
    :message "Input data ended while parsing a CBOR value.",
    :data {:header 91, :cbor/error :clj-cbor.codec/end-of-input},
    :at
    [clj_cbor.error$codec_exception_BANG_
     invokeStatic
     "error.clj"
     8]}],
  :trace
  [[clj_cbor.error$codec_exception_BANG_ invokeStatic "error.clj" 8]
   [clj_cbor.error$codec_exception_BANG_ invoke "error.clj" 5]
   [clj_cbor.core$try_read_value invokeStatic "core.clj" 193]
   [clj_cbor.core$try_read_value invoke "core.clj" 185]
   [clj_cbor.core$decode invokeStatic "core.clj" 219]
   [clj_cbor.core$decode invoke "core.clj" 199]
   [clj_cbor.core$decode_seq$read_data_BANG___46451
    invoke
    "core.clj"
    236]
   [clojure.core$repeatedly$fn__6479 invoke "core.clj" 5138]
   [clojure.lang.LazySeq sval "LazySeq.java" 42]
   [clojure.lang.LazySeq seq "LazySeq.java" 51]
   [clojure.lang.RT seq "RT.java" 535]
   [clojure.core$seq__5419 invokeStatic "core.clj" 139]
   [clojure.core$take_while$fn__5935 invoke "core.clj" 2908]
   [clojure.lang.LazySeq sval "LazySeq.java" 42]
   [clojure.lang.LazySeq seq "LazySeq.java" 51]
   [clojure.lang.Cons next "Cons.java" 39]
   [clojure.lang.RT next "RT.java" 713]
   [clojure.core$next__5403 invokeStatic "core.clj" 64]
   [clojure.core$dorun invokeStatic "core.clj" 3130]
   [clojure.core$doall invokeStatic "core.clj" 3136]
   [clojure.core$doall invoke "core.clj" 3136]
   [clj_cbor.core$slurp_all invokeStatic "core.clj" 279]
   [clj_cbor.core$slurp_all doInvoke "core.clj" 274]
   [clojure.lang.RestFn invoke "RestFn.java" 410]
   [datahike.migrate$import_db invokeStatic "migrate.clj" 35]
   [datahike.migrate$import_db invoke "migrate.clj" 30]
   [crb.db.datahike$db_import invokeStatic "datahike.clj" 74]
   [crb.db.datahike$db_import invoke "datahike.clj" 73]
   [crb.service.core$run_with invokeStatic "core.clj" 48]
   [crb.service.core$run_with invoke "core.clj" 46]
   [crb.run$run invokeStatic "run.clj" 44]
   [crb.run$run invoke "run.clj" 29]
   [clojure.lang.AFn applyToHelper "AFn.java" 154]
   [clojure.lang.AFn applyTo "AFn.java" 144]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.core$apply invoke "core.clj" 662]
   [clojure.run.exec$exec invokeStatic "exec.clj" 48]
   [clojure.run.exec$exec doInvoke "exec.clj" 39]
   [clojure.lang.RestFn invoke "RestFn.java" 423]
   [clojure.run.exec$_main$fn__205 invoke "exec.clj" 180]
   [clojure.run.exec$_main invokeStatic "exec.clj" 176]
   [clojure.run.exec$_main doInvoke "exec.clj" 139]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.core$apply invokeStatic "core.clj" 667]
   [clojure.main$main_opt invokeStatic "main.clj" 514]
   [clojure.main$main_opt invoke "main.clj" 510]
   [clojure.main$main invokeStatic "main.clj" 664]
   [clojure.main$main doInvoke "main.clj" 616]
   [clojure.lang.RestFn applyTo "RestFn.java" 137]
   [clojure.lang.Var applyTo "Var.java" 705]
   [clojure.main main "main.java" 40]],
  :cause "Input data ended while parsing a CBOR value.",
  :data {:header 91, :cbor/error :clj-cbor.codec/end-of-input}}}

What is the expected behaviour?

it imports the dump

How can the behaviour be reproduced?

I can share a link with my dump.

alekcz commented 1 year ago

@awb99 I had this issue the other day in prod. Basically in v0.4 the datoms were backed up as simple edn. The new version expects a binary data type as that is what it emits. Not ideal.

Seems to have been introduce in [#496]. @yflim any possibility of making import backwards compatible?