added python profiling overview slides ipython notebook

marklee77 commented 10 years ago

This is the set of slides on profiling in python presented to Greg and others on Aug 20, 2014.

ethanwhite commented 10 years ago

Here's a link to the rendered notebook: http://nbviewer.ipython.org/github/marklee77/bc/blob/python-intermediate-profiling-lesson/intermediate/python/05-profiling-overview-slides.ipynb

jiffyclub commented 10 years ago

Thanks for this, some notes:

The %time magic output could use some explanation.
"Tight loop" is a bit of jargon that's probably unnecessary.
%timeit runs the loop a dynamic number of times, it won't always be 10,000.
Maybe include a link to the cProfile module.
The filename:lineno(function) column of the pstats output isn't explained.
Could you link to runsnakerun and explain its display with a screenshot?

I personally find that runsnakerun is horrible (in fact impossible) to install. I would not include it as a recommended way to visualize profiles. I would recommend SnakeViz, because it's pretty easy to install (but has problems on Windows). But, disclaimer, I wrote SnakeViz so I'm a bit partial.

marklee77 commented 10 years ago

Okay, I'll try to make some changes to address these issues. I agree that runsnake is a pain, but it is kind of the standard visualizer, and I think we'd be remiss to not at least mention it.

One clarification, I created this notebook to generate a slideshow for my presentation, so it may be a bit terse in comparison with one that's intended to be opened and worked through by a student, or just viewed online. I had mentioned this to Greg during my presentation and his feeling seemed to be that SC hasn't really settled on a definitive format for lecture notes at this time, so we should be trying lots of things.

I did try SnakeViz and it seems nice. One problem that I had at the time was that while I could call it from the notebook and get a pop-up, there was no easy way to include the output inline, which would have been ideal for a slideshow.

tbekolay commented 9 years ago

What do you think about showing the cell magic versions of these rather than the line magics? Once you try to profile something that isn't just a single function call, it becomes much more convenient to use them. And the syntax is pretty similar.

def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)
%timeit fib(10)

becomes

%%timeit
def fib(n):
    return n if n < 2 else fib(n - 1) + fib(n - 2)
fib(10)

%%prun also works, and automatically shows the stats in the bottom frame of the notebook.

Since these are slides, I can understand why you would write some functions in the form

def foo(): sleep(1)

since the code has to fit on a single slide, but still I think it would be ideal to use

def foo():
    sleep(1)

since we don't want to encourage what most Python devs will agree is bad style. Not sure if it's possible to compress the text some other way to fit into a slide?

Final comment! Please run make ipynb in the bc root in order to generate a Markdown file and commit that file so that the notebook will be rendered on generated website.

Thanks! I think this is a great addition to the Intermediate lesson :)

marklee77 commented 9 years ago

I originally developed this as slides for my 10-minute presentation for Greg, but I think the consensus I'm getting here is to change it a bit to show better coding style and make something that is more of a stand-alone notebook that learners could explore. Since it seems like others are actually interested I'll try to make an update soon, it's just been a busy few weeks at work.

tbekolay commented 9 years ago

No worries; if you don't have time right now, another option is to merge it as is, and then work on it more once this repo is split up into individual lessons. Thoughts?

marklee77 commented 9 years ago

I'm fine with merging as-is for now, and allowing other people to make changes. Otherwise I should be able to work on it sometime later on next week.

gvwilson commented 9 years ago

Hi Mark, Any further work on this, or should we merge as-is?

marklee77 commented 9 years ago

On 2014-11-18 14:03, Greg Wilson wrote:

Hi Mark, Any further work on this, or should we merge as-is?

Sorry, it's still on my to-do list. I would say merge as-is and I will continue to submit updates as time allows.

Reply to this email directly or view it on GitHub: https://github.com/swcarpentry/bc/pull/727#issuecomment-63474745

jiffyclub commented 9 years ago

There's still no description or links for RunSnakeRun or the other profiling tools that are mentioned. I think we'd need that before merging because it looks very incomplete.

swcarpentry / DEPRECATED-bc

added python profiling overview slides ipython notebook #727