tmthyjames / SQLCell

SQLCell is a magic function for the Jupyter Notebook that executes raw, parallel, parameterized SQL queries with the ability to accept Python values as parameters and assign output data to Python variables while concurrently running Python code. And *much* more.
MIT License
150 stars 11 forks source link

Possible setup problem #80

Open xiaochuanyu opened 7 years ago

xiaochuanyu commented 7 years ago

Not sure if this is a setup problem but I followed the instructions from README and ran into the following error logged in jupyte whilst opening any notebook:

[IPKernelApp] WARNING | Unknown error in handling startup files:

Consequently, I get the following when trying to use %%sql:

ERROR:root:Cell magic `%%sql` not found.

Using jupyter notebook --debug did not produce additional info.

I'm using OSX with brew installed python.

~ jupyter --version                                                                                                                                                   
4.2.1
~ ipython2 --version                                                                                                                                                          
5.2.2
~ ipython3 --version                                                                                                                                                  
5.1.0
~ python3 --version                                                                                                                                                   
Python 3.5.2
~ python2 --version                                                                                                                                                           
Python 2.7.12

I have no other plugins/extensions installed.

What version of python/jupyer/ipython works with this extension?

tmthyjames commented 7 years ago

Hey! Thanks for the notice. I ran into that problem when I installed it on a Mac recently. Do you have pandas and SQLAlchemy installed?

Here's a way to get a clearer error message:

• Find this file /usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py (this is the directory mine is in, but just find the appropriate shellapp.py file. • add import traceback to the top of the file • Find this line: self.log.warning("Unknown error in handling startup files:") • add these lines under the previous line:

self.log.warning(e)
self.log.warning(traceback.format_exc()) 

This will produce clear error messages and allow you to debug more effectively.

If you need any further assistance, please drop another message here and I'll get back to you. It's unfortunate that IPython doesn't produce clearer error messages, but the above lines should get you some clarity.

The reason for the cryptic WARNING | Unknown error in handling startup files: error is because when you load a startup file in IPython, the file /usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py is supposed to log the errors that any startup files produces, but it doesn't.

xiaochuanyu commented 7 years ago

So I actually modified shellapp.py as follows:

  try:
        for fname in sorted(startup_files):
            self._exec_file(fname)
        except:
            self.log.warning("Unknown error in handling startup files:")
            raise # added this

The stacktrace I got is:

[IPKernelApp] WARNING | Unknown error in handling startup files:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/local/lib/python2.7/site-packages/ipykernel/__main__.py", line 3, in <module>
    app.launch_new_instance()
  File "/usr/local/lib/python2.7/site-packages/traitlets/config/application.py", line 657, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-120>", line 2, in initialize
  File "/usr/local/lib/python2.7/site-packages/traitlets/config/application.py", line 87, in catch_config_error
    return method(app, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/ipykernel/kernelapp.py", line 461, in initialize
    self.init_code()
  File "/usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py", line 263, in init_code
    self._run_startup_files()
  File "/usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py", line 355, in _run_startup_files
    self._exec_file(fname)
  File "/usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py", line 328, in _exec_file
    raise_exceptions=True)
  File "/usr/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2481, in safe_execfile
    self.compile if kw['shell_futures'] else None)
  File "/usr/local/lib/python2.7/site-packages/IPython/utils/py3compat.py", line 289, in execfile
    builtin_mod.execfile(filename, *where)
  File "/Users/xyu/.ipython/profile_default/startup/sqlcell_app.py", line 3, in <module>
    from SQLCell.sqlcell import sql, __KERNEL_VARS__
  File "/Users/xyu/.ipython/profile_default/startup/SQLCell/sqlcell.py", line 27, in <module>
    from .tasks.params import __SQLCell_GLOBAL_VARS__, unique_db_id, engine, application_name
  File "/Users/xyu/.ipython/profile_default/startup/SQLCell/tasks/params.py", line 14, in <module>
    engine = create_engine(driver+'://'+username+':'+password+'@'+host+':'+port+'/'+default_db+application_name)
TypeError: cannot concatenate 'str' and 'NoneType' objects

It seems one of those variables must be None but was not expected. I then added a print as follows:

print([driver, username, password, host, port, default_db, application_name]) # added this
create_engine(driver+'://'+username+':'+password+'@'+host+':'+port+'/'+default_db+application_name)

Running ipython2 produces:

['postgresql', None, None, 'localhost', '5432', 'phUsers', '?application_name=jupyter92dc4d16-3827-4da2-b569-08c47c484ec4']
... same stacktrace as above printed ...

So username and password are None. Tracing their origin to SQLCell/engines/engine_config.py, I see the following:

username = os.getenv('DB_USERNAME')
password = os.getenv('DB_PASSWORD')

So I guess I was supposed to define those environment variables?

tmthyjames commented 7 years ago

Yes, you can define those variables in your .bash_profile or you can utilize the ENGINE param as noted here on the readme page.

Note that you can also use the declare-engines flag to store your connection string info.

Let me know if that helps.