replikativ / datahike

A fast, immutable, distributed & compositional Datalog engine for everyone.
https://datahike.io
Eclipse Public License 1.0
1.62k stars 95 forks source link

Ability to disable `ensure-stored-config-consistency` #654

Closed alekcz closed 9 months ago

alekcz commented 10 months ago

Describe the feature you would like to request

The ensure-stored-config-consistency function that validates connection creates issues when a valid connection may different names.

e.g. on JDBC these two stores are equivalent but not be deemed so. They appear different as because in one case the DB and datahike are on the same network and the in the other they are not.

(def pg-cfg  {:dbtype  "postgresql"
              :jdbcUrl "postgresql://user:password@localhost/konserve"})
(def pg-cfg  {:dbtype  "postgresql"
                    :jdbcUrl "postgresql://user:password@myremoteaddress.com/konserve"})

A tangentially related question: with the ensure-stored-config-consistency how would one go about updating the cache size?

Describe the solution you would like

Describe alternatives you've considered

None really. I'm not familiar enough with Datahike to work on the internal code.

alekcz commented 10 months ago

@whilo @TimoKramer thoughts?

whilo commented 10 months ago

I think disabling might not be the best idea. The problem is once you connect to a store with a potentially incompatible configuration you might corrupt the database. I agree that the jdbcUrl should be allowed to be different for the hostname and port. Maybe we could provide a multimethod that is implemented by every store and only returns the part of the config that should be matched.

whilo commented 10 months ago

store-identity is supposed to do that, I need to check again how it works for JDBC. Alternatively we could still use the random names and add them to every store and just rely on that.

alekcz commented 10 months ago

Lemme work on making store-identity more robust.

alekcz commented 10 months ago

@whilo I've almost completed the store-identity work on datahike-jdbc. The missing part in on the datahike side. The comparison doesn't use the store-identity: https://github.com/replikativ/datahike/blob/166cdbbb7031fa03e619091fb7a99c20cd550ec3/src/datahike/connector.cljc#L126C18-L126C18

whilo commented 10 months ago

Right, I think it should break out the store config there and run store-identity on it and then equality check that.

alekcz commented 9 months ago

Resolved by #655