search

clarin-eric / ParlaMint

ParlaMint: Comparable Parliamentary Corpora
https://clarin-eric.github.io/ParlaMint/
41 stars 52 forks source link

Local validation of full data #614

Closed nemeek closed 1 year ago

nemeek commented 1 year ago

Where can I find how to proceed with local validation of full data? make validate-parlamint-EE does validation for sample only.

matyaskopp commented 1 year ago

This is the way how I do it:

cd PATH_TO_PARLAMINT_REPOSITORY
mkdir -p DataRelease/ParlaMint-EE
# copy files
rsync -a PATH_TO_DATA/ParlaMint-EE.TEI/* DataRelease/ParlaMint-EE/
rsync -a PATH_TO_DATA/ParlaMint-EE.TEI.ana/* DataRelease/ParlaMint-EE/
# run validation
nohup make validate-parlamint-EE DATADIR=DataRelease >  validate-parlamint-EE.log 2>&1 &
# then run conllu validation
nohup make UA-conllu-EE DATADIR=DataRelease >  validate-parlamint-EE.log 2>&1 &
# ...

This attitude makes sure that data in PATH_TO_DATA are not changed and you don't need to clean directory from extra conllu/vert/txt files