Automate the testing of tutorials to ensure cross-platform compatibility

nomicode commented 4 years ago

see mika's comments here:

https://github.com/crate/crate-tutorials/pull/3

it's vital that our tutorials work. more so than the rest of our documentation, I would say -- because the tutorials are likely to be a user's first impression

we want the tutorials to work across the widest possible array of operating systems and setups

testing this manually each time is a hassle, and probably won't be done reliably. so, we should come up with a way of testing (i.e., executing) the tutorials automatically as part of the CI setup

not sure what the best approach is here. so creating this issue to collect thoughts, discussion, etc

nomicode commented 4 years ago

my initial thoughts:

we have tests set up on crate/crate. I am not too familiar with the setup so I should take a look at that
I would like to research plugins for sphinx that allow code blocks to be executed. I bet there are multiple ways of getting this working
this would need a running instance of cratedb on the machine doing the testing. we should be able to set that up for travis or github actions

nomicode commented 4 years ago

crate / crate-docs

Automate the testing of tutorials to ensure cross-platform compatibility #8