syntaxgym
is a Python package for easy targeted syntactic evaluation of
language models. It replicates the core behavior of the SyntaxGym website,
providing standardized and reproducible processes for targeted syntactic
evaluation.
syntaxgym
requires a working Docker installation.
Next, install and update the package using pip:
$ pip install -U syntaxgym