Better autocompletion for our Python APIs

[gramster]

Right now the autocomplete in IPython leaves a lot to be desired, because it uses introspection on an object. So if you have a variable x, and do 'x.', IPython will introspect on x. This works okay if x is already bound from a previous cell, but not if it is bound in the current cell. Also, if you chain method calls in the current cell, it is useless:

Table('....').<tab>

gives autocompletion based off the working directory, so you may get something like:

.git/
.gitignore

I think this is something that could be fixed in IPython itself. Ideally it would use an automplete library that can do type inference like Jedi, although a simpler approach that may be enough for our purposes would be to use function annotations on our APIs. That said, we don't really want to customize IPython at this point so this is something we should do in the new UX.

[nikhilk]

This is something that is orthogonal to our new UX ... the completion support is coming from the IPython kernel itself, which we'll continue to use. We should definitely look into how to improve it ... but do so in a way that it can be upstreamed back to the ipython/jupyter project.

I agree ... it would be much nicer if it can be based on a more sophisticated type inference approach.

However a bit of a technical question - how would you infer? There is nothing in the signature of a python method that indicates what it returns, and the python doc-comments (as verbose as they are) aren't exactly guaranteed to be machine readable (unlike jsdoc comments).

[gramster]

Jedi does code analysis to do type inference.

If we were to do it ourselves at low cost, we would need annotations. There are many ways to do these, but the best is probably the standard function_annotations (for 3.x) or annotations (2.x); see http://python-future.org/func_annotations.html.

Note that Python 3.5 is expected to have optional typing and 3.x function annotations will be deprecated or removed at that point in favor of these; probably not a big deal though.

[gramster]

It looks like IPython semi-handles this by having a greedy option to autocomplete; however, its a hack - it executes the code before . in order to get something it can introspect on. No good for our purposes.

[nikhilk]

Oh, I didn't realize IPython tries to evaluate the code... seems dangerous, and error prone. Tempted to say we should try to turn it off. Wouldn't want to inadvertently do something that is costly, long running or worse has unexpected side-effects.

[VelizarVESSELINOV]

Atom has a great Python plugins (linter-flake8, linter-pep8, linter-pydocstyle, linter-pylama, linter-pylint, linter-python-pep257, linter-python-pep8, linter-python-pyflakes, python-isort, etc.) that we are missing in Datalab, Jupyter team is working on Jupyter-notebook. If you can make the Datalab experience closer to Atom it will be great.