search

jupyter / jupyter_client

Jupyter protocol client APIs
https://jupyter-client.readthedocs.io
BSD 3-Clause "New" or "Revised" License
386 stars 285 forks source link

Documentation requests / experience report #173

Open gibiansky opened 8 years ago

gibiansky commented 8 years ago

I'm trying to use jupyter_client as a library and am running into some places where I wish the documentation was clearer; I figured I'd file an issue with a small "developer experience report" so that perhaps someone can try and address some of the confusion I went through trying to use the package.

  1. What is the "shortest path" to talking to a kernel? My first instinct was to click on the "manager" section, so that might be a good place to put a small blurb along the lines of, "to talk to a kernel, create a kernel manager, then create a kernel client, then use the kernel client methods", or something like that. A 2-3 line code sample that talks to a kernel with just one message and the BlockingKernelClient.
  2. There isn't any documentation for the BlockingKernelClient class, only KernelClient. It seems like it might be helpful to include that as well (I figure it has roughly the same interface?)
  3. How do I create a KernelManager? What are the **kwargs it takes in its __init__ constructor? It seems like if I just make a KernelManager(), something happens, since I can then use kernel_manager.client().channels_running and get True, but I have no idea what kernel (if any?) this client is connected to... My next guess is to pass kernel_name="My_kernel_name". Tried it, not sure what it does, so giving up here.
  4. I will instead try to use MultiKernelManager, because it takes a kernel_name argument and a start_kernel method! That seems to give me a valid kernel ID, and then using get_kernel gives me a KernelManager, which then I can use get_connection_info with and get valid ports! This is what I've wanted all along!
  5. It looks like maybe run_kernel is what I would want to use, but I have no idea what *args are, so I have no idea how to use it.
  6. Now that I have a valid running kernel, I want to get its kernel_info and send a kernel_info_request and get back the kernel_info_reply. Unlike a bunch of other messages, there doesn't seem to be a method for kernel_info... After googling for a bit I run across client.py which has a kernel_info method, and realize that that class is a KernelClient, not a KernelManager, and that I can get a client with KernelManager.client(). I get a client client and run client.kernel_info() (which I will not is not documented on the client page and I only know about it b/c I read some source). I get back a UUID which I assume is the UUID of the sent message, and I confirm this in the source of client.py, and now somehow I need to map this UUID to a response message.
  7. I quickly find client.get_shell_msg(). I guess technically I should check whether it has the right parent but given that I've only sent one message so far it is more or less guaranteed to be the response I expect if the kernel is behaving at all reasonably.
  8. This docstring is wrong, the input is a 4-length message, and the docstring forgets about the p_metadata piece, I think.
  9. The final code that I was looking for was: 
multimanager = MultiKernelManager()
manager = multimanager.get_kernel(multimanager.start_kernel("kernel-name"))
client = manager.client()
client.kernel_info()
print(client.get_shell_msg())

If in reality I just missed a bunch of documentation that already exists, then please point me to it and I'll feel silly; if not then it might be helpful to add bits and pieces to the jupyter_client docs to save whoever comes next an hour or two.

Carreau commented 8 years ago

Thanks @gibiansky for the detailed request, really appreciate that. We'll try to improve that and make it easier !

cc @willingc

willingc commented 8 years ago

@gibiansky Huge thanks for documenting your path and helping others avoid future headaches. I've taken your comments and edited them into chunks for someone (possibly me) to work on to improve the docs. Do you mind if we ping you for review when the doc PR(s) are completed?

Thanks!

  1. The "shortest path" to talking to a kernel
    • [ ] click on the "manager" section,
    • [ ] add blurb along the lines of, "to talk to a kernel, create a kernel manager, then create a kernel client, then use the kernel client methods", or something like that.
    • [ ] A 2-3 line code sample that talks to a kernel with just one message and the BlockingKernelClient.
  2. Document BlockingKernelClient class, only KernelClient has docs now.
    • [ ] Review KernelClient docs
    • [ ] Add BlockingKernelClient info
  3. Add KernelManager detailed doc
    • [ ] How do I create a KernelManager?
    • [ ] __init__ constructor What are the **kwargs?
    • [ ] Details needed here. It seems like if I just make a KernelManager(), something happens, since I can then use kernel_manager.client().channels_running and get True, but I have no idea what kernel (if any?) this client is connected to... My next guess is to pass kernel_name="My_kernel_name". Tried it, not sure what it does, so giving up here.
  4. Review MultiKernelManager docs, be
    • [ ] takes a kernel_name argument
    • [ ] has a start_kernel method! That seems to give me a valid kernel ID
    • [ ] then using get_kernel gives me a KernelManager,
    • [ ] which then I can use get_connection_info with and get valid ports! This is what I've wanted all along! :cake:
  5. Add run_kernel detail.
    • [ ] would want to use, but I have no idea what *args are, so I have no idea how to use it.
  6. Now that I have a valid running kernel:
    • [ ] get its kernel_info
    • [ ] send a kernel_info_request
    • [ ] get back the kernel_info_reply.
    • [ ] Unlike a bunch of other messages, there doesn't seem to be a method for kernel_info... After googling for a bit I run across client.py which has a kernel_info method, and realize that that class is a KernelClient, not a KernelManager, and that I can get a client with KernelManager.client().
    • [ ] I get a client client and run client.kernel_info() (which I will note is not documented on the client page and I only know about it b/c I read some source). I get back a UUID which I assume is the UUID of the sent message, and I confirm this in the source of client.py, and now somehow I need to map this UUID to a response message.
  7. client.get_shell_msg()
    • [ ] I guess technically I should check whether it has the right parent but given that I've only sent one message so far it is more or less guaranteed to be the response I expect if the kernel is behaving at all reasonably.
  8. Review docstring for accuracy.
    • [ ] This docstring is wrong, the input is a 4-length message, and the docstring forgets about the p_metadata piece, I think.
  9. :tada: The final code that I was looking for was: :tada:
    • [ ] Add example code block
multimanager = MultiKernelManager()
manager = multimanager.get_kernel(multimanager.start_kernel("kernel-name"))
client = manager.client()
client.kernel_info()
print(client.get_shell_msg())
gibiansky commented 8 years ago

@willingc Yes please feel free to loop me in on any PRs / issues related to things I file.

gibiansky commented 8 years ago

Please, please mention the following code bits!

client.start_channels()
client.wait_for_ready()

If I understand correctly, accessing client.iopub_channel creates the channel if its not created yet. This means that you can arbitrarily lose messages at the beginning depending on random timing variation if you don't call those two functions beforehand... because if the kernel sends a message, it may or may not arrive before the iopub channel is created. Not sure that's quite right but it just cost me another two hours... :-/

gibiansky commented 8 years ago

IMHO things like get_iopub_msg should just throw an exception if start_channels hasn't been called, not silently set up the channel... the latter is just asking for trouble like what I've been observing