Proposal: Remove iPython notebook from sql lessons

[abbycabs]

https://github.com/swcarpentry/bc/tree/master/novice/sql

The .ipynb files are difficult to diff and merge properly. Do people here teach from the notebook? I've always used the .md.

Let me know if you want to keep the .ipynb files in the repo! Otherwise, I propose ditching them for the easier to maintain and contribute to .md files.

[jiffyclub]

My understanding is that the lessons are written in the Notebooks because that's where they are runnable, so you can run the Notebooks to generate output so that it shows up in the Markdown files. If you weren't developing from the Notebooks you'd have to find another way to run the SQL commands and paste the results into the Markdown files.

[jiffyclub]

And as a data point: if I was teaching this unit I would probably do it from the Notebook. Seems much easier than the SQL command line. (Unless there is another alternative?)

[abbycabs]

@jiffyclub - thanks for the use case. I'd assume that if someone is writing a lesson they'd have a way to run SQL commands. But yes, fair point.

I've always taught from command line so students could follow along by typing commands, which is pretty similar to the shell lesson they just went through. But good to know you feel differently.

[jiffyclub]

With the lessons in Notebooks, if we do something like add rows to the sample DB, or in some other way modify it, it's a matter of re-running the notebooks to update the sample output in the lesson. Without the Notebooks I think that'd be a process with a lot of copy/paste moving back and forth between the lesson notes and a SQL command line.

[rgaiacs]

The .ipynb files are difficult to diff and merge properly. Do people here teach from the notebook? I've always used the .md.

Last month there was one long discussion about it at our maillist [1]. IPython Notebook make easy to manage the input and output of the commands but require us to keep the *.ipynb as the source of the lessons and *.md for use GitHub Pages.

[1] http://lists.software-carpentry.org/pipermail/discuss_lists.software-carpentry.org/2014-August/001951.html

[abbycabs]

That's a very good point @r-gaia-cs, which I did not think of.

Just want to tag @gvwilson to check if he still feels iPython should be removed here.

Catching up on reading the thread now

[gvwilson]

I'm conflicted about this. On the one hand:

1. Diff/merge is more difficult with .ipynb than with .md.
2. The %%sqlite line at the start of SQL cells in the Notebook is confusing for novices (they have a hard enough time with the difference between actual SQL and admin commands like .schema).
3. I've never taught SQL using the Notebook: I always use either the sqlite interpreter or the Firefox plugin.

On the other hand:

1. Notebooks are runnable, so we can check output. (Though in practice, how many of us carefully re-read to make sure the output is still in sync with the text?)
2. If we're telling people to use the Notebook for real work, we should use it for teaching.

I'm starting to think seriously about putting all our notes in Markdown, but using the Notebook for teaching. Dunno... :-(

[rgaiacs]

When review the our regex lesson I remember that it use Python examples but is write in Markdown. Maybe we could invite people to improve it as a test before "putting all our notes in Markdown"?

[iglpdc]

I've just taught the SQL lesson from the notebook and worked well. Sure, the sqlite magic was a bit confusing, but it's a fair trade compared to the command line. I went back and forth a couple of times during the lesson to the command line. I think it stresses the differences between the language, the command line interpreter, and the notebook, and helps to understand the case for Python.

[aaren]

We've already discussed this elsewhere, but you might find my tool notedown useful for generating the ipython notebooks from markdown automatically.

Some things it can do for this issue:

• automatically prepend the %%sqlite to input cells.
• strip the outputs from the markdown source (so then you end up with a notebook that you can populate with outputs)

I think it would need a little tweaking for the SWC lesson format, but with the new style of lessons this should be quite easy. You can then deterministically generate your notebooks from the markdown source.

Cons of this approach:

• Dependency on external tool
• Need to generate notebooks at start of the lesson.

Regarding the %%sqlite magic - you could also avoid this by starting ipython with an sql kernel, but one doesn't exist yet