kuzudb / kuzu

Embeddable property graph database management system built for query speed and scalability. Implements Cypher.
https://kuzudb.com/
MIT License
1.41k stars 99 forks source link

Bug: Testing framework doesn't correctly report failure tests #4510

Open acquamarin opened 4 days ago

acquamarin commented 4 days ago

Kùzu version

master

What operating system are you using?

MAC M1

What happened?

I have a test case in doc-examples.test:

-CASE CopyFromTest
-STATEMENT LOAD EXTENSION "${KUZU_ROOT_DIRECTORY}/extension/json/build/libjson.kuzu_extension";
---- ok
-STATEMENT CREATE NODE TABLE Person (id SERIAL, name STRING, info STRUCT(height DOUBLE, age INT64, registryDate DATE previousUsernames STRING[]), PRIMARY KEY(id));
---- ok
-STATEMENT COPY Person FROM '${KUZU_ROOT_DIRECTORY}/dataset/doc-examples-json/people.json';
---- ok
-STATEMENT MATCH (p:Person) return p.* ORDER BY p.id DESC;
---- 3
2│Gregory││
1│Bob││{height: 1.810000, age: 71, previousUsernames: [theBuilder,theMinion]}
0│Alice│2024-07-31│{height: 1.680000, age: 45, previousUsernames: [obviouslyAlice,definitelyNotAlice]}
-STATEMENT DROP TABLE Person;

There is a grammar mistake in the frist create table statement and should fail, however the ctest still marks it as correct. If i run the test using e2e_test binary, it fails the test.

Are there known steps to reproduce?

No response

SterlingT3485 commented 2 days ago

seems our test framework has problem of not report error with character - in test file name.

now investigating into that

SterlingT3485 commented 2 days ago

when running single test with ./e2e_test and gtest_filter, we got:

Note: Google Test filter = json~test~doc-exsamples.JsonTypeDoc
[==========] Running 0 tests from 0 test suites.
[==========] 0 tests from 0 test suites ran. (0 ms total)
[  PASSED  ] 0 tests.

seems the it didn't wen through the google test, however it didn't be caught by "no such file"