openmhealth / schemas

A repository of Open mHealth schemas.
Apache License 2.0
71 stars 45 forks source link

Add progress indicator #23

Closed emersonf closed 1 year ago

emersonf commented 1 year ago

Problem

The validator sometimes takes 10 seconds or more to complete, making it feel like it's hanging.

Solution

A progress indicator has been added to provide feedback on validation progress, e.g.

% python3 validator.py
Loaded 128 schemas.
Loaded 45 synthetic IEEE schemas.
Validating test data files:  98% (564/572)

It looks like there's a slowdown for the last few files. Basic profiling indicates that this is caused by establishing SSL connections to GitLab to retrieve IEEE schemas. If this becomes a problem, we can look into parallelising validation.

Miscelleanous

Only files in the test data directory having a .json suffix are now processed. Other files, e.g. .DS_Store, are ignored.