Draft videos for simstore_and_cli parts 2, 3

dwhswenson commented 3 years ago

Drafts of the rest of this tutorial are available in temporary Dropbox links. @sroet @jvreede (and others): I'd appreciate any feedback you can give -- especially if it's ever hard to follow what part of the screen I'm talking about (which could be easily handled with some video editing). Unless there's a major problem, I probably won't re-record anything.

I'm hoping to publish part 2 on or around Tuesday 7 September, with part 3 about a week after that (y'know: leave them eager after that exciting cliffhanger at the end of part 2!)

Part 2

Watch or Download (524MB; 13m57s)

Description for YouTube:

This is part 2 in a 3-part series on our new storage subsystem, SimStore, and the OpenPathSampling Command Line Interface (CLI). This part focuses on running commands with the CLI.

Up-to-date tutorial materials at https://github.com/openpathsampling/mini-tutorials/tree/main/simstore_and_cli.

Video for part 1: https://www.youtube.com/watch?v=IAipZfZpwZ4

This video was made using OpenPathSampling 1.5.0 and OpenPathSampling CLI 0.2.1.

Part 3

Watch or Download (656MB; 14m53s)

Description for YouTube:

This is part 3 in a 3-part series on our new storage subsystem, SimStore, and the OpenPathSampling Command Line Interface (CLI). This part focuses on analyzing simulation results.

Up-to-date tutorial materials at https://github.com/openpathsampling/mini-tutorials/tree/main/simstore_and_cli.

Video for part 1: https://www.youtube.com/watch?v=IAipZfZpwZ4

This video was made using OpenPathSampling 1.5.0 and OpenPathSampling CLI 0.2.1.

sroet commented 3 years ago

First of all I think these are great! Most time-stamped comments are tiny and should not be fixed if they require any effort. The important one is tagged with IMPORTANT

general comments:

descriptions:

You might want to add a link to the "previous" and "next" lecture in this series in each description (after releasing the next one of course)
You might want to add simstore_and_cli to those tutorial links (just in case you want to make more than 1 series)

Part 2 comments:

(2:56) I think this might have worked better if the description of contents was still shown in the terminal (like you do for most other commands)
(3:54) There is a notification sound that bled into the recording (don\t know if this is fixable with post-processing)
(9:16) Typing on the lowest 2 shell lines can get hidden by the playback bar (at least on dropbox)
IMPORTANT (13:30) onwards seems like audio/video desync
(13:48) Are you saying "two-part" or "three-part"

Part 3 comments:

(10:50) any reason why this is not an ln scale? (which should allow you to refer back to the barrier height of about 5, right?)
One analysis question, is the drop-off seen in 11:23 correctly excluded from the rate matrices?

Other comments

(not relevant for this repo but some things I noticed while watching): form part 2:

the openpathsampling -h command should really show version info
the openpathsampling visit-all -h still lists output ncfile instead of output file (also openpathsampling md -h, and ... equilibrate -h)

from part 3:

4:44 the descriptions of the two repex movers are identical, these should probably be more verbose.

dwhswenson commented 3 years ago

Thanks for the feedback, @sroet! I've updated video and descriptions in response to your comments. Links above are to the most recent version (only change is in the video for part 2, to fix the audio sync issue)

general comments:

You might want to add a link to the "previous" and "next" lecture in this series in each description (after releasing the next one of course)

Agreed. Actually, I think it makes sense for a series to have "first", "previous", and "next" (when each is available and different). But with a 3-part series, that implies that everything except the first video would link to all the other videos. So for this series, I think I'll just link to all other videos in all descriptions (as part 1, part 2, and part3).

Obviously, draft descriptions above only have the link for part 1 so far.

You might want to add simstore_and_cli to those tutorial links (just in case you want to make more than 1 series)

Done. I guess YouTube already truncates links, so I don't have to worry about GitHub-specific URL components (tree/main/) causing confusion. (This is why the title image has the tutorial name on a different line from the link.)

Part 2 comments:

(2:56) I think this might have worked better if the description of contents was still shown in the terminal (like you do for most other commands)

You're probably right, and that was something I did on at least some run-throughs as I was developing this tutorial. Not going to change it now, though.

(3:54) There is a notification sound that bled into the recording (don\t know if this is fixable with post-processing)

I'm not hearing it at that timestamp. I do remember hearing what I think was a door shutting at one point, but I can't find it now. If it happens while I'm speaking, it'll be hard to edit out. (OTOH, a sound that occurs during the rare moments that I'm pausing is pretty easy to edit out.)

(9:16) Typing on the lowest 2 shell lines can get hidden by the playback bar (at least on dropbox)

I also noticed this. I don't think I'll change it in this video -- mainly, redoing the parts where I enlarge the relevant output of contents is too much hassle. But in the future, I'll try to leave some space below the terminal window to avoid this.

IMPORTANT (13:30) onwards seems like audio/video desync

Ugh, I thought I'd fixed this before! Fortunately, it was easy to fix.

(13:48) Are you saying "two-part" or "three-part"

Definitely "three," but let's see what the YouTube captions think! 😂 (BTW, they tend to be astonishingly good, even on technical material like this.)

Part 3 comments:

(10:50) any reason why this is not an ln scale? (which should allow you to refer back to the barrier height of about 5, right?)

Not sure what you mean here by ln scale. I don't change the values of the data; I just change the axis itself. Note that the values reported are not $0$, $-1$, $-2$; but $10^0$, $10^{-1}$, $10^{-2}$. I could set yticks in terms of $e$, but that would be significantly fancier matplotlib than I wanted to do in this video!

One analysis question, is the drop-off seen in 11:23 correctly excluded from the rate matrices?

Yes. Our calculation for the crossing probability $P(B|\lambda_0)$ is from $P^\text{tot}(\lambda_m|\lambda_0) P(B|\lambda_m)$, where $\lambda_m$ is the outermost interface. For details, see http://openpathsampling.org/latest/topics/tis_analysis.html.

Other comments

(not relevant for this repo but some things I noticed while watching): form part 2:

the openpathsampling -h command should really show version info

I don't think so. Version info isn't shown in python --help, conda --help, or git --help. However, all of these have a --version flag that reports version -- that should probably be added, but would only be the CLI version (which is why I haven't done that yet; I think most people typing openpathsampling --version would expect the version of the library, not the CLI tool, although the CLI version is the correct thing to report).

However, see https://github.com/openpathsampling/openpathsampling-cli/pull/47, which is intended to give version info for OPS library and CLI (and eventually other tools we integrate with.)

the openpathsampling visit-all -h still lists output ncfile instead of output file (also openpathsampling md -h, and ... equilibrate -h)

Fixed in https://github.com/openpathsampling/openpathsampling-cli/pull/51.

4:44 the descriptions of the two repex movers are identical, these should probably be more verbose.

Agreed. I believe this is just the default string representation of the mover -- one-way shooting movers get names when created, but repex (and most other) movers do not.

Another option to change that would be to modify high_level.move_scheme.MoveAcceptanceAnalysis._line_as_text. In this case, I think that line.move_name there is actually the mover itself, and it gets converted to a string in that method.

Actually, I've long wanted to add pandas output to the move acceptance (and I could imagine user selection of which columns to include -- the comparison to how frequently the scheme expected the move to run was extremely useful when we first developed OPS, but is mostly noise to users.) I think this would look a lot better in a notebook, and it would be far easier for users to make use of the output data.

sroet commented 3 years ago

I could set yticks in terms of $e$, but that would be significantly fancier matplotlib than I wanted to do in this video!

Yeah that is what I meant, and I thought it was as simple as doing plt.yscale("ln") instead of plt.yscale("log") but it isn't. Don't worry about it then.

jvreede commented 3 years ago

I just watched part 2 - really great video! I like it that you sometimes make typos, it is a nice way of getting the viewer's attention back.

jvreede commented 3 years ago

Now I just watched video 3 - again, very very nice!

While watching I noticed that you say "today". That would be a bit strange if you're watching the videos on the same day, so it seems better to say "this part" or "this video". I had to replay the video from 3.20 on to follow which results you were discussing. Perhaps you could highlight them with the type of highlight you used in part 2.

At 4.45, when you start discussing poor and good replica exchange it would help if the relevant numbers are highlighted.

When explaining the crossing probabilities I found it hard to follow what exactly you are looking at. Perhaps you could add a highlight or a pointer. In particular at 11.45, when you say something about a bump, which indicates the data should be looked at more carefully.

dwhswenson commented 3 years ago

I just watched part 2 - really great video! I like it that you sometimes make typos, it is a nice way of getting the viewer's attention back.

Glad you liked it! And yeah, typos obviously aren't intentional, but I think it's better for people to see how the code errors when it gets bad input than to try to edit that out.

While watching I noticed that you say "today". That would be a bit strange if you're watching the videos on the same day, so it seems better to say "this part" or "this video".

Fair point, although I'm kind of thinking of this in terms of "episodes," which is why I say "today." Actually, videos 2 and 3 (and the video for #3, which I haven't edited yet) were all recorded on the same day. (I changed my shirt in between! 😂)

dwhswenson commented 3 years ago

Whoops, commented too early .... continuing:

I had to replay the video from 3.20 on to follow which results you were discussing. Perhaps you could highlight them with the type of highlight you used in part 2.

At 4.45, when you start discussing poor and good replica exchange it would help if the relevant numbers are highlighted.

Thanks -- these are the exactly the sort of things where I wasn't 100% sure, but this feedback convinces me that I should put in some extra work to highlight those. I'll post a note here when I've updated the videos.

When explaining the crossing probabilities I found it hard to follow what exactly you are looking at. Perhaps you could add a highlight or a pointer. In particular at 11.45, when you say something about a bump, which indicates the data should be looked at more carefully.

I'll try a callout here, although I'm not entirely sure how well it'll work. I thought about it, but was worried it might be more distracting than informative. I'll ask for feedback after I try that.

Regarding the bump, I was saying that if there is one, it's a problem. There isn't one in this data.

openpathsampling / mini-tutorials