lucaong / cubdb

Elixir embedded key/value database
Apache License 2.0
548 stars 23 forks source link

Undefined Error during k/v updates, seems like a compact error #74

Open eminarcissus opened 2 months ago

eminarcissus commented 2 months ago

I'm using. the most basic setups without any configuration and just fire up a file db in the GenServer init period, afterwards I do periodically handling GenServer's timeout to maintain the database status. But after a while I started to see the following errors.

** (Protocol.UndefinedError) protocol Enumerable not implemented for %CubDB.Btree{root: {:l, [valid_c: 257041, valid_c_timestamp: 271377, valid_i: 252945, valid_i_timestamp: 256017]}, root_loc: 271396, size: 4, dirt: 107, store: %CubDB.Store.File{pid: #PID<0.243.0>, file_path: "/home/lk/p_manage/_build/dev/lib/p_manage/priv/db/0.cub"}, capacity: 32} of type CubDB.Btree (a struct). This protocol is implemented for the following type(s): Date.Range, File.Stream, Function, GenEvent.Stream, HashDict, HashSet, IO.Stream, List, Map, MapSet, Range, Stream (elixir 1.16.3) lib/enum.ex:1: Enumerable.impl_for!/1 (elixir 1.16.3) lib/enum.ex:166: Enumerable.reduce/3 (elixir 1.16.3) lib/enum.ex:4396: Enum.reduce/3 (cubdb 2.0.2) lib/cubdb/btree.ex:88: CubDB.Btree.load/3 (cubdb 2.0.2) lib/cubdb/compactor.ex:46: CubDB.Compactor.compact/2 (cubdb 2.0.2) lib/cubdb/compactor.ex:33: anonymous fn/2 in CubDB.Compactor.run/2

I'm using CubDB for some simple KV pairs for runtime configuration and data storage. while I'm keep updating on 2 keys with Poison encoded jsons. In the beginning it works without problem, but after a period of time it starts to raise the above errors. Looks more likely to be compact time error.

The structure of the stored content is [valid_c: 257041, valid_c_timestamp: 271377, valid_i: 252945, valid_i_timestamp: 256017],where timestamp are coming from :os.system_time(:millisecond) and valid_c/valid_i are Poison encoded json of a list of strings.

Is there anything I missed in configuration? Any hints would be real appreciatd.

lucaong commented 1 month ago

Hi @eminarcissus , the error looks very strange:

** (Protocol.UndefinedError) protocol Enumerable not implemented for %CubDB.Btree

The Btree module does implement the Enumerable protocol. I am not sure what is happening here.

Could you tell me more about your setup? What kind of device is CubDB running on (an embedded device or a standard computer)? Are you doing some kind of hot code reloading, or anything "unusual"?

Thank you for your report!