kocolosk
commented
2 years ago That's correct, erlfdb_tuple is this library's module for encoding data types into binaries following FoundationDB's Tuple specification. It's not required for a FoundationDB client to follow that spec, but it's a good one, and ensures that different FDB clients can interoperate on the same set of data. We could probably do with some more worked examples of creating the Erlang tuples that erlfdb_tuple expects as input.
I see at least a couple of options for handling a broader spectrum of data types:
User-Specific Type Codes
The FDB Tuple spec reserves a set of type codes for user-specific data types that do not map directly to one of the supported data types already defined in the spec. We could choose to allocate type codes within that space to support other Erlang data types. For example, we could declare that 0x40 would prefix a binary that was encoded in the Erlang External Term Format, i.e. the output of the term_to_binary function, which would allow us to serialize arbitrary Erlang terms in FoundationDB and automatically decode them on the way out. I would caution against using that capability for the Keys of records in FoundationDB, as I don't really think the External Term Format has been designed with stable sorting in mind, but it could prove useful for intermixing arbitrary Erlang terms and other data types that are natively supported in FoundationDB as part of the values. This would require a small patch to erlfdb
Ignore Tuple Layer for Values
Alternatively, your application could decide that every Value your app stores with FoundationDB is encoded with term_to_binary and ignore the Tuple layer altogether. You'd lose the ability for other FDB clients to parse the Values, but you'd have an easier native Erlang experience, e.g.
Eshell V12.1.4 (abort with ^G)
1> DB = erlfdb:open().
{erlfdb_database,#Ref<0.289576361.3161587713.93177>}
2> Key = erlfdb_tuple:pack({<<"foo">>, 123}).
<<1,102,111,111,0,21,123>>
3> erlfdb:set(DB, Key, term_to_binary(hello)).
ok
4> erlfdb:get(DB, Key).
<<131,100,0,5,104,101,108,108,111>>
5> binary_to_term(erlfdb:get(DB, Key)).
helloYou'll see I still used the Tuple encoding for Keys, as it's carefully designed for proper sorting that's essential to most data modeling techniques in FoundationDB.
Hi, I'm coming again. The problem is that I would like to store tuples into FoundationDB. But it seems it won't work fine if I want to store tuples with atom() in it to FoundaitonDB.
#{context => {invalid_tuple_term,message},exception => error,stacktrace => [{erlfdb_tuple,encode,2,[{file,"er│me has already been fetched lfdb_tuple.erl"},{line,258}]}Furthermore, If I want to store types other than binary(), the only way seems to be call
erlfdb_tuple:pack()? Any advice if I want store arbitrary type in foundationdb? That will help a lot.