openml / server-api

Python-based server
https://openml.github.io/server-api/
BSD 3-Clause "New" or "Revised" License
1 stars 1 forks source link

Add tests based on `SQLite` database #105

Open PGijsbers opened 11 months ago

PGijsbers commented 11 months ago

Having tests that are not dependent on the MySQL database would be a nice addition because:

The disadvantage is that a lot of tests will be "hard" to set up with a SQLite database, requiring many different tables with populated data that cover many edge cases. This likely will make it harder to interpret what is going on again, so perhaps using it only for tests which require at most a few tables with a few rows is best.

An example of a test which seems suited to run on SQLite is validating the subquery in data/list:

SELECT ds1.`did`, ds1.`status`
FROM dataset_status as ds1
WHERE ds1.`status_date`=(
    SELECT MAX(ds2.`status_date`)
    FROM dataset_status as ds2
    WHERE ds1.`did`=ds2.`did`
)

because it relies only on a single table and a few rows to test all expected permutations.

PGijsbers commented 10 months ago

Jos advised against using SQLite if it is (easily) avoidable, since you will have to account for "minor" implementation differences (such as handling errors). Differences in behavior might also be functional (e.g., keys may be case-sensitive in one, but not the other).