Native JSON accessor

[techtonik]

I'd like use IPython for quick prototyping with external APIs, and it would be nice if JSON work could be made native without the need to import external libraries. Because JSON is the most common format for API and the web.

[Carreau]

I'm quite unsure what you mean. Json is in the standard library, and many libraries (like requests) have a .json() method.

[techtonik]

If I read data from .json, like:

me = """
[
  {
    "id": 43799,
    "name": "anatoly techtonik",
    "username": "techtonik",
    "state": "active",
    "avatar_url": "https://secure.gravatar.com/avatar/9d7e611f31c52f4d62bbe279d4f334de?s=80&d=identicon",
    "web_url": "https://gitlab.com/techtonik"
  }
]
"""

Then I'd like to use http://jmespath.org/tutorial.html to access data inside with less boilerplate than import json<enter>json.loadwhatever(me)[0]["id"].

myid = me.json()[0].id

[techtonik]

.j can available as a property of https://ipython.readthedocs.io/en/stable/interactive/shell.html#string-lists if there is a way to turn string into string list easily.

[takluyver]

I don't think this is something IPython should do. The focus is on the interface now, not on providing concise APIs for particular operations.

If you want a more convenient way to work with JSON, write a separate module for that. It could provide an IPython extension.

[techtonik]

If the focus is on the interface, what is the goals of the interface?

[takluyver]

• Providing the default Python kernel for Jupyter applications
• Offering a powerful interactive shell for Python to run in the terminal

A bunch of other stuff got added to IPython in the past, which probably wouldn't be added today. Some of this was general utilities from the days when Python packaging was so awkward that people preferred to make big collections of useful pieces to avoid installing more than one package. Those days are thankfully behind us - Python packaging still isn't great, but it's good enough to distribute much more focused packages.

Another big chunk of functionality from the past is code to make IPython behave more like a shell. Bits of this are useful, but a lot of it is ~unused and rusting, and the code to support it is convoluted and prone to corner cases. Xonsh is a more modern codebase which provides similar features - as it was designed to do this from the beginning, it has a much cleaner approach than IPython can use.

[techtonik]

I will take a look at xonsh.

So you're telling that the goal right now is to ship Python kernel for Jupiter and all helpers that are implemented so far is a mistake? What is the details of criteria for "powerful interactive shell"? I would benefit from autoimports like in golang in this shell, so I won't have to bail out to mess with requirements.txt.

[takluyver]

For the future, I'd say that any new features that work by allowing more code that's not valid Python would have to clear a very high bar for added value. I don't think that all the existing features doing that were mistakes, but if we were starting from scratch today, I think we'd try harder to do as much as possible with regular Python code, and to limit the extra syntax supported by IPython.

Python is mostly a well-designed language. It's not always the most concise, but the syntax is relatively simple, consistent and clear. When we add extra syntax for special cases, we make it less simple and less consistent. We also add a maintenance burden because we have to do some kind of parsing to handle the extra syntax; we can't rely on Python's own parser. We've already turned off some syntax by default ('autocall' would let you do f 1 2 3 instead of f(1, 2, 3)) because it was confusing new programmers learning Python about what was valid Python code.

[takluyver]

As for your specific proposal: I don't think it needs to be part of IPython, but something like this:

from json import loads

class attr_access(dict): 
     def __getattr__(self, name): 
         if name in self: 
             return self[name] 
         raise AttributeError(name) 

def json(s):
    return loads(s, object_hook=attr_access)

Would let you do almost what you suggested:

myid = json(me)[0].id

If an import is an annoyance, you can use a startup file to do imports every time you start IPython.

[techtonik]

Xonsh is a more modern codebase which provides similar features - as it was designed to do this from the beginning, it has a much cleaner approach than IPython can use.

I tried xonsh. The major design flaw is that separation between shell commands and Python are context dependent.

>>> ls -l
total 0
-rw-rw-r-- 1 snail snail 0 Mar  8 15:46 xonsh
>>> ls = 44
>>> l = 2
>>> ls -l
42
>>> del ls
>>> ls -l
total 0
-rw-rw-r-- 1 snail snail 0 Mar  8 15:46 xonsh

This example is official - https://xon.sh/tutorial.html#python-mode-vs-subprocess-mode

So there is no real alternative to IPython with clear separation between shell commands and Python syntax in regard to fast system automation.