pivottable.js display introduces NULL values that don't exist in the source pandas Dataframe

[nmontpetit]

I believe I'm using pivottable.js (in a Jupyter notebook) correctly, but please let me know if that's not the case.

In this example, I'm creating almost the simplest pandas dataframe possible:

test_data = DataFrame([['a', 'b'], ['b', 'a']], columns=['var1', 'var2'])

test_data

Dataframe:

    var1 var2
0     a     b
1     b     a

When I display this in pivottable.js

pivot_ui(test_data)

The table shows that var1 and var2 both have 3 NULL values, and a total count of 5 records.

https://i.stack.imgur.com/M9y4Q.jpg

Is this expected behavior for my usage shown above? More than happy to be told I'm an idiot and I'm using pivottable.js incorrectly. :)

If this is unexpected behavior, I can provide further information about my system configuration.

Thanks!

[nicolaskruchten]

This is indeed not expected and I can’t replicate it here... can you tell me which versions of Python, Jupyter and Pandas you used, and how you installed this library?

[nmontpetit]

Thanks, Nicolas! The version information is below. I did upgrade to the latest version of Pandas very recently.

I used pip install pivottablejs for the install.

In the meantime, is it possible to use a different data representation as input?

Thanks, -Nick

$ python --version
Python 3.6.3 :: Anaconda custom (64-bit)

$ conda list
# packages in environment at C:\Users\MontpeNX\AppData\Local\Continuum\anaconda3:
#
# Name                    Version                   Build  Channel
jupyter                   1.0.0                    py36_7
jupyter_client            5.1.0            py36h9902a9a_0
jupyter_console           5.2.0            py36h6d89b47_1
jupyter_core              4.3.0            py36h511e818_0
jupyterlab                0.35.3                   py36_0
jupyterlab_launcher       0.13.1                   py36_0
jupyterlab_server         0.2.0                    py36_0
pandas                    0.24.0           py36ha925a31_0
pivottablejs              0.9.0                    pypi_0    pypi

[nmontpetit]

Hi Nicolas,

I reinstalled Anaconda with Python 3.6 and then used conda instead of pip to install pivottablejs. That seemed to fix the issue.

For future reference, is it possible to pass in a different data representation other than a Pandas Dataframe? Probably not, I'm guessing.

Thanks, -Nick

[nicolaskruchten]

OK, glad that it's fixed for you now... odd that it broke!

To answer your question, it doesn't have to be a Pandas dataframe: anything that has a .csv(encoding='utf8') method will work: https://github.com/nicolaskruchten/jupyter_pivottablejs/blob/master/pivottablejs/__init__.py#L74

[dominic2017]

Hi, i used pip install and had this same null error.

What is the solution?

packages under my pip list: absl-py 0.7.1 altair 3.1.0 astor 0.8.0 attrs 19.1.0 backcall 0.1.0 bleach 3.1.0 bokeh 1.2.0 boto 2.48.0 certifi 2019.6.16 chardet 3.0.4 Click 7.0 colorama 0.4.1 cycler 0.10.0 decorator 4.4.0 defusedxml 0.6.0 entrypoints 0.3 Flask 1.1.0 gast 0.2.2 grpcio 1.21.1 h5py 2.9.0 idna 2.7 imageio 2.5.0 ipykernel 5.1.1 ipython 7.5.0 ipython-genutils 0.2.0 ipywidgets 7.4.2 itsdangerous 1.1.0 jedi 0.13.3 Jinja2 2.10.1 jsonschema 3.0.1 jupyter 1.0.0 jupyter-client 5.2.4 jupyter-console 6.0.0 jupyter-contrib-core 0.3.3 jupyter-contrib-nbextensions 0.5.1 jupyter-core 4.4.0 jupyter-highlight-selected-word 0.2.0 jupyter-latex-envs 1.4.6 jupyter-nbextensions-configurator 0.4.1 Keras-Applications 1.0.8 Keras-Preprocessing 1.1.0 kiwisolver 1.1.0 lime 0.1.1.29 lxml 4.3.3 Markdown 3.1.1 MarkupSafe 1.1.1 matplotlib 2.2.2 mistune 0.8.4 nbconvert 5.5.0 nbformat 4.4.0 networkx 2.3 notebook 5.7.8 numpy 1.14.3 packaging 19.0 pandas 0.24.1 pandocfilters 1.4.2 parso 0.4.0 pickleshare 0.7.5 Pillow 6.0.0 pip 19.1.1 pivottablejs 0.9.0 prometheus-client 0.6.0 prompt-toolkit 2.0.9 protobuf 3.8.0 Pygments 2.4.2 pynegv 0.45.5 pyodbc 4.0.24 pyparsing 2.4.0 pypiwin32 223 pyrsistent 0.15.2 python-certifi-win32 1.2 python-dateutil 2.7.3 pytz 2019.1 PyWavelets 1.0.3 pywin32 224 pywinpty 0.5.5 PyYAML 5.1 pyzmq 18.0.1 qgrid 1.1.1 qtconsole 4.5.1 requests 2.19.1 requests-negotiate-sspi 0.5.2 scikit-image 0.15.0 scikit-learn 0.20.2 scipy 1.1.0 seaborn 0.9.0 Send2Trash 1.5.0 setuptools 28.8.0 six 1.12.0 tensorboard 1.12.2 tensorflow 1.12.0 termcolor 1.1.0 terminado 0.8.2 testpath 0.4.2 toolz 0.9.0 tornado 6.0.2 traitlets 4.3.2 urllib3 1.22 vega 2.3.2 vega-datasets 0.7.0 virtualenv 16.6.0 virtualenvwrapper-win 1.2.5 wcwidth 0.1.7 webencodings 0.5.1 Werkzeug 0.15.4 wheel 0.33.4 widgetsnbextension 3.4.2 wincertstore 0.2 wrapt 1.11.2 xlrd 1.2.0 XlsxWriter 1.1.8

[cdebrier]

Hi everyone, I do have the problem too. It basically doubled the size of my table because of unwanted NULL values. Any clue would be more than welcome as this library is clearly a very useful tool. Thanks!

[karenbold]

me too

[ghost]

Hi, This issue may be caused by an EOL character. The workaround is here: https://github.com/nicolaskruchten/jupyter_pivottablejs/issues/52#issuecomment-528409060 Hope it help.

[PatJOConnell]

@s13i - your method worked well for me! thanks