KeyErrors exceptions ('dimensions', 'theme' and others) when calling cufflinks from multithreaded dash app

[valveca-ai]

I think there is a race condition in auth.py function get_config_file(*args) when called from multiple threads.

I'm running a dash application with multiple cufflink plots and it causes random exceptions like:

  File "/Users/xxxxxxxxxxx/cufflinks/plotlytools.py", line 709, in _iplot
    dimensions=auth.get_config_file()['dimensions']
KeyError: 'dimensions'

  File "/Users/xxxxxxxxxxx/cufflinks/plotlytools.py", line 713, in _iplot
    theme = auth.get_config_file()['theme']
KeyError: 'theme'

as well as with other keys.

I was able to debug this and noticed that dash is refreshing more than one cufflink chart in parallel over multiple threads causing a race condition in get_config_file(*args). A workaround is to add this to auth.py to cache previous results:

from functools import lru_cache
...
@lru_cache(maxsize=32)
def get_config_file(*args):
    ...

A better solution would be to serialize access to this method over multiple threads.

I'm running from cufflinks' github master branch, Python 3.7.3 and my project uses these packages:

appnope==0.1.0
attrs==19.1.0
backcall==0.1.0
bleach==3.1.0
certifi==2019.3.9
chardet==3.0.4
Click==7.0
colorlover==0.3.0
dash==0.42.0
dash-bootstrap-components==0.6.0
dash-core-components==0.47.0
dash-html-components==0.16.0
dash-renderer==0.23.0
dash-table==3.6.0
decorator==4.4.0
defusedxml==0.6.0
entrypoints==0.3
et-xmlfile==1.0.1
Flask==1.0.2
Flask-Caching==1.7.1
Flask-Compress==1.4.0
idna==2.8
ipykernel==5.1.0
ipython==7.5.0
ipython-genutils==0.2.0
ipywidgets==7.4.2
itsdangerous==1.1.0
jdcal==1.4.1
jedi==0.13.3
Jinja2==2.10.1
joblib==0.13.2
jsonschema==3.0.1
jupyter-client==5.2.4
jupyter-core==4.4.0
MarkupSafe==1.1.1
mistune==0.8.4
nbconvert==5.5.0
nbformat==4.4.0
notebook==5.7.8
numpy==1.16.3
openpyxl==2.6.2
pandas==0.24.2
pandocfilters==1.4.2
parso==0.4.0
pexpect==4.7.0
pickleshare==0.7.5
plotly==3.9.0
prometheus-client==0.6.0
prompt-toolkit==2.0.9
ptyprocess==0.6.0
Pygments==2.4.0
pyrsistent==0.15.2
python-dateutil==2.8.0
pytz==2019.1
pyzmq==18.0.1
requests==2.21.0
retrying==1.3.3
scikit-learn==0.21.0
scipy==1.2.1
Send2Trash==1.5.0
six==1.12.0
terminado==0.8.2
testpath==0.4.2
tornado==6.0.2
traitlets==4.3.2
urllib3==1.24.3
wcwidth==0.1.7
webencodings==0.5.1
Werkzeug==0.15.2
widgetsnbextension==3.4.2
xlrd==1.2.0

[CloudZazu]

I am also seeing this problem. Ran it on Windows 10 debug Flask hosting. Thanks for reporting!

[ignalex]

same issue, MAC Mojave. Cufflinks 0.15 and 0.16 @valveca-ai your workaround works THANKS. could it be submitted as as pull-request. modifiying modules files in different systems is not good.

[dmitra79]

I am also seeing these errors pop up randomly. Is this going to be fixed?

[dmitra79]

I am seeing this much more since I added flask Cache to my Dash app. Evironment: cufflinks-py 0.17.0 py_1 conda-forge dash 1.8.0 py_0 conda-forge dash-core-components 1.7.0 py_0 conda-forge dash-html-components 1.0.2 py_0 conda-forge dash-renderer 1.2.3 py_0 conda-forge dash-table 4.6.0 py_0 conda-forge flask 1.1.1 py_1 conda-forge flask-caching 1.7.1 py_0 conda-forge flask-compress 1.4.0 py_0 conda-forge plotly 4.5.0 py_0 conda-forge

[resposit]

This still seems to be broken, I just bumped into the same issue. Any chance to get it fixed ?

[jeffreyleifer]

Running version cufflinks 0.17.3 and plotly 4.8.0 and having the same issue. @valveca-ai workaround works. Any chance getting it integrated into a new release?

[abdeltif-b]

Same issue here. @valveca-ai workaround works. I hope to fix this issue in the future release.

[xiongyifan]

hard code the get_config_file func, and it works.

# auth.py 180
def get_config_file(*args):
    """
    Return specified args from `~/.config`. as dict.
    Returns all if no arguments are specified.

    Example:
        get_config_file('sharing')

    """
    return {'sharing': 'private', 'theme': 'pearl', 'colorscale': 'dflt', 'offline': True, 'offline_connected': True,
            'offline_url': '', 'offline_show_link': True, 'offline_link_text': 'Export to plot.ly',
            'datagen_mode': 'stocks', 'dimensions': None, 'margin': None, 'offline_config': None}
    # if _file_permissions:
    #     ensure_local_files()
    #     return load_json_dict(CONFIG_FILE, *args)
    # else:
    #     return _FILE_CONTENT[CONFIG_FILE]

in my case, I use Plotly Dash and cufflinks. When a variable change, I use cufflinks to create 3 Figures in 3 callbacks.