python wrapper (and missing docstrings via juliacall )

[svenbuder]

Would be great to have for newcomers who don't necessarily know how to dig into the code yet.

Current docstring info when using Korg in python via juliacall (0.9.15):

Signature:   Korg.interpolate_marcs(*args, **kwargs)
Type:        AnyValue
String form: interpolate_marcs
File:        ~/opt/anaconda3/lib/python3.9/site-packages/juliacall/__init__.py
Docstring:   <no docstring>

[ajwheeler]

I think this is a fundamental limitation of juliacall, but I'm looking into it. Will leave this issue open even if there's nothing that can be done at the moment as it's not an ideal situation.

[andycasey]

That's right; it is a limitation of juliacall.

If you want to get the docstring you can use the @doc operator in Julia, but it is a bit clunky:

from juliacall import Main as jl

jl.seval("using Korg")

print(jl.seval("@doc Korg.format_A_X"))
  format_A_X(default_metals_H, default_alpha_H, abundances; kwargs... )

  Returns a 92 element vector containing abundances in A(X) (\log_{10}(X/H) +
  12) format for elements from hydrogen to uranium.

  Arguments
  ≡≡≡≡≡≡≡≡≡≡≡

  You can specify abundance with these positional arguments. All are optional,
  but if default_alpha_H is provided, default_metals_H must be as well.

    •  default_metals_H (default: 0), i.e. [metals/H] is the \log_{10}
       solar-relative abundance of elements heavier than He. It is
       overriden by default_alpha and abundances on a per-element basis.

    •  default_alpha_H (default: same as default_metals_H), i.e.
       [alpha/H] is the \log_{10} solar-relative abundance of the alpha
       elements (defined to be C, O, Ne, Mg, Si, S, Ar, Ca, and Ti). It
       is overriden by abundances on a per-element basis.

    •  abundances is a Dict mapping atomic numbers or symbols to [X/H]
       abundances. (Set solar_relative=false to use A(X) abundances
       instead.) These override default_metals_H. This is the only way to
       specify an abundance of He that is non-solar.

  Keyword arguments
  ≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡≡

    •  solar_relative (default: true): When true, interpret abundances as
       being in [X/H] (\log_{10} solar-relative) format. When false,
       interpret them as A(X) abundances, i.e. A(x) =
       \log_{10}(n_X/n_\mathrm{H}) + 12, where n_X is the number density
       of X. Note that abundances not specified default to the solar
       value still depend on the solar value, as they are set according
       to default_metals_H and default_alpha_H.

    •  solar_abundances (default: Korg.asplund_2020_solar_abundances) is
       the set of solar abundances to use, as a vector indexed by atomic
       number. Korg.asplund_2009_solar_abundances and
       Korg.grevesse_2007_solar_abundances are also provided for
       convienience.

I think the only options around this are:

1. Have a Korg Python package which is just a light wrapper around juliacall that exposes the docstrings. This could be made such that it would not have to be updated if there is a new release of Korg.jl: it would expose whatever functions and docstrings are available in Korg.
2. Submit a pull request to juliacall so that when they convert functions, it sets the Python docstring to whatever the Julia docstring is.
3. Put in some kind of documentation on how to access the docstring via juliacall with Python and tell users to do that.

2 is best for longevity but requires the juliacall folk to accept it. One advantage of #1 over #2 is that #1 would also mean tab-completion works for interactive Python users. I think #1 is probably preferable to #3.

[ajwheeler]

I think we should definitely start by trying to do option 2. I'm not sure that option 1 would be worth the ongoing maintenance.

Is programmatically setting docstrings in python easy? A quick internet search says that you can just set f.__doc__, but I'm not sure if that's officially supported or discouraged.

[andycasey]

You can set it programmatically. I tried option 2 yesterday but the JuliaCall/PythonCall interoperability made it a total maze to find where Python functions get wrapped. I started adding print statements just to understand how things were being called, and I quickly encountered segmentation faults. That made my motivation drop.

[ajwheeler]

Are you sure you can set it programmatically?

jl.sum.__doc__ = "test docstring" doesn't work for me.

(See also https://github.com/JuliaPy/PythonCall.jl/issues/468. Andy I know you will have seen this, but linking for the record.)

[andycasey]

You can in Python, but when I saw that response I realised why you can't do it through JuliaCall.

Here's a pure Python proof:

def f(x):
    return x**2

f.__doc__ = "Return x to the power 2"

Then help(f) shows:

Help on function f in module __main__:

f(x)
    Return x to the power 2

I might get some terminology here wrong, but the effect is the same: The reason why they say it can't be done through JuliaCall is because they aren't actually creating instances of functions, they are using the same class type just to point to all the Julia functions. If they were actually adding Python functions for each Julia function then you could set the docstring with each function. But instead they are just filling up the Python namespace with the functions available, not actually the functions themselves.

Addendum:

I don't see any reason why it would have to be done the way that they have done it. Or any reason why you couldn't just create Python functions on-the-fly for each Julia equivalent, but presumably there is a reason they did it the way they did.

[ajwheeler]

octofitterpy is a good example of a wrapper similar to what we are talking about. (Seen on Julia slack.)