Open spasa47 opened 3 months ago
Here's what I do for pandoc-optex
:
tests/<CASENAME>.{md,native,html}
tests/<CASENAME>.flags
which has command line flags (e.g: --standalone
or -Mtoc
)tests/<CASENAME>.tex
run-test.sh
script that runs pandoc
on a test case and compares the results with diff -u
.e.g: run-test.sh tests/case1.md
# pseudo-code for run-test.sh
TESTNAME=$1
BASENAME=$( echo $1 | sed -e 's/\.md//g' -e 's/\.native//g' -e 's/\.html//g') # strip suffix
FLAGS=""
if [[ -f "$BASENAME.flags" ]] ; then
FLAGS=$(cat $BASENAME.flags)
fi
pandoc -t optex-writer-script.lua "$FLAGS" "$TESTNAME" -o "$BASENAME.tmp"
if [[ diff -u "$BASENAME.tex" "$BASENAME.tmp" > "$BASNAME-diff.tmp" ]] ; then
echo PASS: $TESTNAME
else
echo FAIL: $TESTNAME
cat $BASNAME-diff.tmp
fi
Big picture concept:
parse_margin()
, parse_picture_dimension()
, parse_color()
)make test
. The path to the pandoc executable can be overwritten with PANDOC=/.../pandoc-3.1 make test
so I can easily test multiple pandoc versions.make test-pdf
target that then typsets the .tex
files with optex -interaction=batchmode
and just checks if a .tex
file could be typset/"compiled" or not. If I add/update a test case I do a manual visual inspection of the .pdf
file.Technical notes:
make test-pdf
needs a working TexLive installation (which is a bit of a bigger download), so I'm not going to run it on GitLab's CI infrastructure. make test
only needs make
, bash
and pandoc
so it's easy and fast to get it running a GitLab CI job.
xargs -P8
or make -j8
). I currently have ~370 test files. Running them sequentially takes about 35 seconds, while using -P8
brings it down to 7 seconds.P.S: I put some general notes (not related to testing) here https://github.com/olsak/OpTeX/discussions/85#discussioncomment-9998029
Thanks for your input, I am definitely going to implement something along the lines of what you described. I am currently thinking of giving the bats project a try, as to not reinvent the wheel when it comes to unit testing in bash.
Concerning CI, it seems that it should be possible to get a full texlive distribution using GitHub actions, which would eliminate the need to install and setup texlive on every pipeline run. I have 0 experience with GitHub CI/CD (I've only ever used GitLab CI/CD), so there might be some surprises waiting for me, but as it looks now, it should be possible to run all tests via CI/CD.
I've also read through your general notes and will implement some of those ideas in the future, but for now I will focus on testing and unsupported elements (such as span) :pray: .
I am unsure as to how to do this yet, maybe different input files in different formats