search

Dyalog / pynapl

Dyalog APL ←→ Python interface
MIT License
74 stars 10 forks source link

Import Pandas returns KeyError #1

Closed krish240574 closed 6 years ago

krish240574 commented 6 years ago

py←⎕New Py.Py

py.Import'pandas' DOMAIN ERROR: KeyError('call',) py.Import'pandas' When I try to import 'pandas' as above, I keep getting a DOMAIN ERROR. pandas is installed and working fine, I've double-checked.

Other imports work fine - np←py.Import'numpy' np <module 'numpy' from '/opt/A3/lib/python3.6/site-packages/numpy/init.py'>

plt←py.Import'matplotlib' plt <module 'matplotlib' from '/opt/A3/lib/python3.6/site-packages/matplotlib/init.py'>

jayfoad commented 6 years ago

Thanks for the report. I can reproduce this. Any ideas, @marinuso ?

jayfoad commented 6 years ago

Can you try to work around the problem by doing py.Exec'import pandas' instead?

py.Import tries to return a result. res←py.Import'foo' is more or less equivalent to:

py.Exec'import foo'
res←py.Eval'foo'

For reasons I don't understand yet, it's only the second line that is failing with pandas.

krish240574 commented 6 years ago

I did try the workaround you suggested, Jay. Still the same error. Regards, Krishna

On Tue, Feb 20, 2018 at 2:20 PM, jayfoad notifications@github.com wrote:

Can you try to work around the problem by doing py.Exec'import pandas' instead?

py.Import tries to return a result. res←py.Import'foo' is more or less equivalent to:

py.Exec'import foo' res←py.Eval'foo'

For reasons I don't understand yet, it's only the second line that is failing with pandas.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Dyalog/pynapl/issues/1#issuecomment-366908256, or mute the thread https://github.com/notifications/unsubscribe-auth/AEtZ9rhpH9NmfNvEpaP_YrKfulFWiQcHks5tWodHgaJpZM4SJm03 .

jayfoad commented 6 years ago

This works for me on Ubuntu 17.10:

Dyalog APL/S-64 Version 16.0.32246
Unicode Edition
Tue Feb 20 09:21:49 2018
      ]load Py
#.Py
      py←⎕NEW Py.Py
      py.Import'sys' ⋄ py.Eval'sys.version'
3.6.3 (default, Oct  3 2017, 21:45:48)
[GCC 7.2.0]
      py.Exec'import pandas as pd'
      py.Eval'pd.Series([1,3,5,np.nan,6,8])'
 1.0  3.0  5.0  nan  6.0  8.0

At what point do you get an error?

krish240574 commented 6 years ago

Dear Jay, Here are the outputs from a similar run on my machine - I'm running Ubuntu 14.04 Linux bernini 4.4.0-112-generic #135~14.04.1-Ubuntu SMP Tue Jan 23 20:41:48 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

Dyalog APL/S-64 Version 16.0.31812 Unicode Edition Tue Feb 20 09:30:16 2018

py.Import'sys' py.Eval'sys.version' 3.6.3 |Anaconda custom (64-bit)| (default, Oct 13 2017, 12:02:49) [GCC 7.2.0]

This succeeds : py.Exec'import pandas as pd'

However - I get an error when I run this :

py.Eval'pd.Series([1,3,5,np.nan,6,8])'

DOMAIN ERROR: Invalid character at offset 33 (⎕IO=1) deserialize[2] r←pyclass decode ⎕JSON json

(the above error is probably unrelated to the "import" issue) - What I intend to do is to get 'pandas' into APL and use its methods, like Marinuso explained in his youtube video - having to do an 'Eval' each time defeats the purpose, IMHO. So the error, when I do a py.Import, still exists.

Regards, Krishna

On Tue, Feb 20, 2018 at 2:53 PM, jayfoad notifications@github.com wrote:

This works for me on Ubuntu 17.10:

Dyalog APL/S-64 Version 16.0.32246 Unicode Edition Tue Feb 20 09:21:49 2018 ]load Py

.Py

  py←⎕NEW Py.Py
  py.Import'sys' ⋄ py.Eval'sys.version'

3.6.3 (default, Oct 3 2017, 21:45:48) [GCC 7.2.0] py.Exec'import pandas as pd' py.Eval'pd.Series([1,3,5,np.nan,6,8])' 1.0 3.0 5.0 nan 6.0 8.0

At what point do you get an error?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Dyalog/pynapl/issues/1#issuecomment-366917166, or mute the thread https://github.com/notifications/unsubscribe-auth/AEtZ9iNvlwWax6Pjrc0xmQx5FjtTJkQIks5tWo79gaJpZM4SJm03 .

jayfoad commented 6 years ago

What I intend to do is to get 'pandas' into APL and use its methods, like Marinuso explained in his youtube video - having to do an 'Eval' each time defeats the purpose, IMHO.

Understood. I'm just trying to narrow down where the problem occurs.

krish240574 commented 6 years ago

Ok, I'll be glad to supply any more logs you need, do let me know. Krishna

On Tue, Feb 20, 2018 at 3:23 PM, jayfoad notifications@github.com wrote:

What I intend to do is to get 'pandas' into APL and use its methods, like Marinuso explained in his youtube video - having to do an 'Eval' each time defeats the purpose, IMHO.

Understood. I'm just trying to narrow down where the problem occurs.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Dyalog/pynapl/issues/1#issuecomment-366924861, or mute the thread https://github.com/notifications/unsubscribe-auth/AEtZ9u2LHuW5SrTB4jEbLs45sFLFXBbQks5tWpX-gaJpZM4SJm03 .

jayfoad commented 6 years ago

I believe this is really a bug in the pandas code:

>>> import pandas
>>> getattr(pandas.options,'__call__')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3/dist-packages/pandas/core/config.py", line 199, in __getattr__
    v = object.__getattribute__(self, "d")[key]
KeyError: '__call__'

getattr calls a custom __getattr__ method which is supposed to return a value or raise AttributeError, not raise KeyError.

This commit looks relevant: https://github.com/pandas-dev/pandas/commit/63fc8af2ad6bee9c90766bcb648d27a0b2748a44

However, since this bug is out there in the wild, I'll tweak pynapl to tolerate it and do the right thing.

krish240574 commented 6 years ago

Ok, thank you very much, Jay. I sincerely appreciate the prompt fix. Sincere regards, Kumar

On Tue, Feb 20, 2018 at 5:13 PM, jayfoad notifications@github.com wrote:

Closed #1 https://github.com/Dyalog/pynapl/issues/1 via 55a9231 https://github.com/Dyalog/pynapl/commit/55a9231d40585c5cf7c8f8ca15f875947f3b4767 .

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Dyalog/pynapl/issues/1#event-1482227128, or mute the thread https://github.com/notifications/unsubscribe-auth/AEtZ9lM4SfamFjOFKR2GLsThzEsSyTHyks5tWq_2gaJpZM4SJm03 .

jayfoad commented 6 years ago

No problem. I have raised an issue with pandas here: https://github.com/pandas-dev/pandas/issues/19789

krish240574 commented 6 years ago

Awesome, thanks again! Regards, Krishna

On Tue, Feb 20, 2018 at 5:41 PM, jayfoad notifications@github.com wrote:

No problem. I have raised an issue with pandas here: pandas-dev/pandas#19789 https://github.com/pandas-dev/pandas/issues/19789

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/Dyalog/pynapl/issues/1#issuecomment-366958643, or mute the thread https://github.com/notifications/unsubscribe-auth/AEtZ9sDP7leg4pMaJmmtFpbmD_aizwp6ks5tWraNgaJpZM4SJm03 .