Improve estimation table and its documentation

[janosg]

Issues with current version estimation table

• The argument sig_digits does not actually implement a rounding to significant digits, but a simple rounding to a fixed number of decimal points. No other kind of rounding is implemented.
• The argument left_decimals is confusing and actually rather a hack to achieve padding that anything related to rounding.
• sig_levels is too abbreviated
• The documentation is in matlab style, i.e. there are millions of examples for each argument instead of descriptions of arguments. There is also a rather long setup section before the first estimation table is shown.

To-Do Code

• [x] rename left_decimals to padding and incorporate the information that is currently only in the Notes into the description of that argument
• [x] rename sig_levels to significance_levels
• [x] remove sig_digits argument
• [x] Add a new argument called number_format. The following things would be valid for that argument:
  • [x] An integer (which would specify the number of digits behind the decimal point)
  • [x] A formatting string or list/tuple of format strings. If multiple, they are applied in order, and in between the number is converted back to float.
  • [x] A function that takes a float and returns a string
• [x] Add a new argument called add_trailing_zeros. If True, all numbers that are not formatted in scientific notation after applying number_format are padded with zeros to the right until all numbers have the same amount of decimal points.
• [x] Add a new argument called add_leading_zeros (analogous to trailing zeros)

Suggested default for number formatting

• add_trailing_zeros=True
• add_leading_zeros=False
• number_format=("{0:.3g}", "{0:.5f}", "{0:.5g}"), i.e.
  • Set all but the 3 most significant digits to zero
  • Set all digits after the fifth decimal point to zero
  • Switch to scientific notation for large numbers only at a magnitude of 1e5, despite only showing three significant digits

  • To-Do Documentation

The current documentation should be almost completely replaced by the following:

• [x] A simple code example with associated output. All optional arguments should be left out but we should show how easy it is to combine statsmodels results with custom models. I would use the same example model as in the ml tutorial and msm tutorial so we can refer to the code there and load and show the result DataFrames which can then be compared to a OLS estimation using statsmodels.
• [x] One example of an estimation table that uses multiple optional arguments at once and the associated output. In the end there should be a link to the API documentation that explains all optional arguments.
• [x] One example that introduces the statistics (number of observation, ....) and how they are added in estimagic models.
• [x] One example that uses return_type="python", does a modification to the DataFrame (e.g. drop a few rows and add a row called "controls" that has a checkmark in some columns and a cross in the others.
• [x] A general description of the philosophy behind the estimation table that inspires to use it creatively
  • Why return type "python" can cover many use-cases that are normally achieved with many arguments
  • How outputs from any language can be brought into one of our "model" formats
  • ...
• [x] Background information on float formatting strings in Python with links to the resources I mention above.

Moreover we should do the following:

• [x] Link to the estimation table documentation in all estimation examples
• [x] Replace the outputs in the estimation table notebook by images of actual latex tables instead of trying to render LaTeX in the notebook (it currently does not work)
• [x] Make sure the typo sisetup code snippet is fixed everywhere (e.g. here and here). One uses a _ and one a - and only one of them can be correct.

Discussion

• Should we formally deprecate the old arguments or simply do the changes? I think we can simply do the changes.
• What would be a good example for the more advanced sections? Should we stick to the example we are using currently in the estimation table documentation?
• Should we switch from a documentation notebook to an rst file with dropdown containers so we can accommodate more examples without cluttering the page?

[hmgaudecker]

here for discussion of rounding to significant digits

That link is not quite it, I think. Only looks to the right of the decimal separator, right?

One of my (and many others') pet peeves is that false sense of precision created by many tables. Like you see mean income of 38,234 Euros. Nobody cares about the 34 bit. People will start interpreting that (despite standard errors on the magnitude of hundreds), however, and miss the important parts.

Here is a function I have been using, probably got that from SO at some point, but cannot find the source.

def nsf(num, n=3):
    """n-Significant Figures"""
    numstr = ("{0:.%ie}" % (n - 1)).format(num)
    return float(numstr)

[hmgaudecker]

It is \sisetup{input-symbols=()}, see here https://github.com/OpenSourceEconomics/estimagic/issues/253#issuecomment-974103608

[hmgaudecker]

Another trivial but useful return type would be df. Should probably be the default even.

That is, return a dataframe with all the formatting of *** etc, which then takes advantage of the html rendering in a notebook. Currently, html looks ugly in notebooks.

[hmgaudecker]

Instead of return_type='python', maybe 'modifiable' or something like that? 'elements' ?

[hmgaudecker]

Thinking of it, why think of it as two functions?

1. Format results (i.e., return the python / modifiable stuff). Potentially have different options of returning strings only or numbers (already rounded)
2. Get the view in the output format you want

One may want to think of a short-hand again, but conceptually this might make sense?

[ChristianZimpelmann]

I checked out the version at the branch improve_estimation_table. A few remarks:

• number_format is currently also applied to "Observations". This statistic should, however, never be rounded.
• I am also thinking about two functions/steps, e.g., to allow me to exchange column and index names. By using tabular_tex this is already possible. But it would be great if tabular_tex had some default values (as estimation_table already has).
• @mpetrosian, you haven't opened a PR yet, right? Would be happy to help out if needed.
• Does anybody of you have a code example where estimation_table is used in a project? (-> just PM me)