search

binder-examples / continuous-build

Show how to use continuous integration to test your dependency specifications
BSD 3-Clause "New" or "Revised" License
8 stars 10 forks source link

testing example requested #18

Open fomightez opened 6 years ago

fomightez commented 6 years ago

The documentation suggests," it would be useful to have further testing of the container." And that definitely sounds interesting after getting the basics working with CircleCI. But I am not quite sure how to go about that next and clearly the documentation author had some ideas about that?

Would it use nbval or was something simpler envisioned?
I don't know what the envisioned example was. If you are open to suggestions, maybe the example could be to see if an expected output is made?
We have code in a notebook that produces a file and to keep checking things are working we are curious to use continuous integration to test if the notebook (or isolated code) makes the file. Would that code be needed to be moved to postBuild and then some function check if the file is made. Is that handled in postBuild or by CircleCI?

betatim commented 6 years ago

From my side the idea was to add a demo of how to run "anything" inside the container as part of the CI setup. I made a (very minimal) start here https://github.com/binder-examples/continuous-build/pull/8

The motivation is that just because repo2docker can successfully build your repository doesn't mean it actually does something useful. For that you need to try and run "something" that exercises your container somehow. It could be "try and start python and import package X" or "run the first three cells of every notebook in the container" or ... basically what ever makes sense for that repo. The hardest part is getting going so having a demo of how to add a step to CircleCI that uses nbval/papermill/RStudio would be a good starting point.

fomightez commented 6 years ago

I took your minimal approach and adapted in the python code that test the basics of our import and image generating, and was able to get it to work to look for creation of a file. It is here.
Plus I added a badge to the README to get feedback. This is what we were hoping for. Thanks!

Eventually it would be nice to have an example with nbval/papermill, but it took me quite a bit to get this code working because of the need to make the code to do something useful using just one line of python code that gets sent in as part of that docker run command. And, thus I won't be tackling that myself too soon.