search

pboettch / json-schema-validator

JSON schema validator for JSON for Modern C++
Other
466 stars 134 forks source link

Ctest fails on 51 tests after successful installation #141

Closed grbba closed 3 years ago

grbba commented 3 years ago

Hello,

I followed the installation using what i got from the readme file

git clone https://github.com/pboettch/json-schema-validator.git
cd json-schema-validator
mkdir build
cd build
cmake [..]
make
make install # if needed
ctest # run unit, non-regression and test-suite tests

everything completed successfully except the ctest once run reports 51 failed tests cf the attached file ctest-fail.txt

I have a JSON Schema which seems to validate properly in VSC and the json files based upon this schema as well. I also have a program using the nlohmann parser and everything works reading & parsing the the json files. I wanted nevertheless add the validation in the program as users do not necessarily use a validating editor like VSC.

loading/validating the schema actually works fine but when validating e json file then produces an : no subschema has succeeded, but one of them is required to validate error.

Looking in the code i saw this has something to do with anyOf/oneOf and the anyOf test has failed ( as of the error file i added ) maybe both are linked?

I am on a Mac MacOS 10.15.7 using clang 

% clang --version
Apple clang version 12.0.0 (clang-1200.0.32.28)
Target: x86_64-apple-darwin19.6.0
Thread model: posix

I have to admit that i am a total beginner in this field and my schema / json files build upon that may actually be wrong somewhere but I still have to figure out how to debug those things properly.

Any help is appreciated very much.

pboettch commented 3 years ago

You seem to have two different problems. One is that maybe your schema isn't correct so your validation fails.

The other one is that the non-regression tests fail. Let's focus first on the non-regs tests: could you run:

ctest -R JSON-Suite::type -V

and paste the output here?

grbba commented 3 years ago

Hello, thanks for your reply and yes they are two different issues. The result of the above command is :

UpdateCTestConfiguration  from :/Users/gregor/Development/json-schema-validator/build/DartConfiguration.tcl
UpdateCTestConfiguration  from :/Users/gregor/Development/json-schema-validator/build/DartConfiguration.tcl
Test project /Users/gregor/Development/json-schema-validator/build
Constructing a list of tests
Done constructing a list of tests
Updating test list for fixtures
Added 0 tests to meet fixture requirements
Checking test dependency graph...
Checking test dependency graph end
test 45
    Start 45: JSON-Suite::type

45: Test command: /Users/gregor/Development/json-schema-validator/test/test-pipe-in.sh "/Users/gregor/Development/json-schema-validator/build/test/JSON-Schema-Test-Suite/json-schema-test" "/Users/gregor/Development/json-schema-validator/test/JSON-Schema-Test-Suite/tests/draft7/type.json"
45: Test timeout computed to be: 10000000
45: /Users/gregor/Development/json-schema-validator/test/test-pipe-in.sh: line 10: arr: bad array subscript
1/1 Test #45: JSON-Suite::type .................***Failed    0.01 sec

0% tests passed, 1 tests failed out of 1

Total Test time (real) =   0.03 sec

The following tests FAILED:
     45 - JSON-Suite::type (Failed)
Errors while running CTest
pboettch commented 3 years ago

The problem is that the test-script run a script which requires the bash-shell to be installed and used. It seems that you system does not provide a bash-shell with the right-version. What is your version? Try bash --version.

grbba commented 3 years ago

bash --version gives

% bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin19)
Copyright (C) 2007 Free Software Foundation, Inc.

indeed my default shell is zsh

i just rerun ctest under bash and the result is the same.

pboettch commented 3 years ago

Yes, the array features used in the test-script are supported for bash 4.0 or later only.

pboettch commented 3 years ago

Let me see whether I can simplify the script for bash <4

grbba commented 3 years ago

Ok so that the tests actually fail doesn't mean anything because of that. Assuming this is the case that would leave me with a bad definition somewhere in the schema to pass validation.

That may be a good idea as i am running on a recent version of MacOS which has been kept up to date and that's what is available there. Maybe on BigSur they included 4.x now but maybe not as Apple wants you to use zsh ..

I should be able to get a 4.x version using brew though.

pboettch commented 3 years ago

For your schema, do not hesitate to open another issue.

I have not found another way to rewrite the simple bash script quickly. But it should be easy given some time and no other funnier project on the side on a Sunday afternoon.

grbba commented 3 years ago

ok i will do that then would be great to get some help on that one as well. I will try to install bash 4.x and try again for the first part.

Merci beaucoup

grbba commented 3 years ago

I have now bash 5.1 installed through brew. As its located in /usr/local/bin instead of /bin I changed that in test-pipe-in.sh and it passes all tests except one:

The following tests FAILED:
     69 - Build::cmake-install (Failed)

The result is the same if CTest is started from /bin (i.e. bash 3.x ) or /usr/local/bin (i.e. bash 5.1) but maybe there is another reference to /bin/bash somewhere.

I'll write up my second issue next.

pboettch commented 3 years ago

OK, if cmake-install fails, this is not a real problem for you I think.

grbba commented 3 years ago

Thanks for your help so far //