Open psychemedia opened 2 years ago
It's not really up to me, but my 2c is that this wouldn't be a good fit for nbval. I don't think the kind of comments & cell tags you specify in nbval are expressive enough to specify what you actually want. E.g. it's not at all obvious that 'check list' means 'check the list has exactly M items' rather than 'check that this produces a list'. And what counts as a list, anyway? HTML <ol>
and <ul>
? Plain text with - Bullet points
? Latex?
Of course, you could make something where you write check code in comments:
# assert len(outputs.find('ul')[0]) == 7
list_days_of_week()
That might well be useful, but I'd say it's something quite different from nbval.
I've been sketching some ideas here: https://github.com/ouseful-PR/nbval/tree/table-test
Currently, a crude attempt at a dataframe comparison, a crude attempt at checking the linecount of stdout
, a couple of CLI flag hacks to try to work around %%timeit
cells (I did wonder if hardwiring a regex template around that might be better), and a couple of extra tags to give a bit more variety in ignoring cell outputs from nbval
perspective (eg NB-VARIABLE-OUTPUT
) and more generally adding semantics (eg folium-map
).
These are all just sketches around a particular use-case (a set of educational material notebooks), and hacked to work for those / that purpose rather than being tested for general production use (which is beyond me). There are no tests either (my tests are informal against particular notebooks I have available locally....)
In some cases, it may be that cells generate variable output of a consistent form, for example:
In such cases, it would be useful to be able to test for such structural similarity (
nbval-check-linecount
,nbval-check-list
,nbval-check-table
) even if the detail is different.