search

vaticle / typedb

TypeDB: the polymorphic database powered by types
https://typedb.com
Mozilla Public License 2.0
3.72k stars 337 forks source link

Check data folder structure before assuming it is a database #6690

Open jamesreprise opened 1 year ago

jamesreprise commented 1 year ago

Description

When a new folder is created by another program in server/data/, TypeDB assumes that it is a database without checking the file layout first. It also creates a schema folder, even if there is no data folder in place. It then halts and produces the following error:

08:45:34.484 [main] ERROR com.vaticle.typedb.core.server.TypeDBServer - [DBS01] Invalid Database Operation: Database 'newFolder' (located at: /Users/james/typedb-server/server/data/newFolder) has incompatible data version '0' - this server supports version '2'. Please reload or migrate your data.

This new folder has no data folder at all, yet the error reports an "incompatible data version '0'".

Environment

  1. OS (where TypeDB server runs): macOS 12.6.1
  2. TypeDB version (and platform): 2.11.1

Reproducible Steps

  1. Create a new folder in typedb-server/server/data
  2. Run TypeDB

Expected Output

A warning (or error) that a given database folder is not valid.

Actual Output

08:45:34.484 [main] ERROR com.vaticle.typedb.core.server.TypeDBServer - [DBS01] Invalid Database Operation: Database 'newFolder' (located at: /Users/james/typedb-server/server/data/newFolder) has incompatible data version '0' - this server supports version '2'. Please reload or migrate your data.

and a valid schema directory is created in the folder, which is particularly strange behaviour.

flyingsilverfin commented 1 year ago

do you think just checking two directories exist is sufficient behaviour @jamesreprise ? https://github.com/vaticle/typedb/pull/6689

jamesreprise commented 1 year ago

do you think just checking two directories exist is sufficient behaviour @jamesreprise ? #6689

Based on discussions we've had, if we don't decide or control what happens in either of the directories, then we can't require anything to exist (or say that because something does or doesn't exist that the file is valid or invalid). A temporary measure could be to include a file with some magic bytes named TYPEDB_DATABASE so we can validate that this is a real database created by TypeDB. However, this isn't much added value over requiring a data and schema folder exist.

flyingsilverfin commented 1 year ago

Let's keep this in mind for TypeDB 3.0

mathieuisabel commented 1 year ago

Just FYI, I did face that situation on my side as I had a volume mounted that contained other directories (i.e. to import data).