Python wrapper when?!!!

[cyberslam]

Hi Marty,

       Congrat for being a brilliant C++ programmer and getting Etaler to this stage.  For most users and including me, we need a Python Wrapper in order to use HTM due to other "data input" requirements.  If you could get a similar Jupyter Notebook going or making yours to work with like in this Notebook: "https://github.com/psdyer/NuPIC-Algorithm-API-Example", that will open up a lot of users.  

     Looking forward to your reply.

Samuel

[marty1885]

@cyberslam There is a big API change coming. Which will make a Python wrapper built now obsolete. So the Python wrapper will have to be developed after that.

In the mean while, you can use ROOT for an interactive shell. It will give you python-like experiences while we are doing our best to get a Python wrapper. ROOT also supports Jupyter Notebook!

Alternatively, being an open source project, we are very welcoming to contributions. You can build your own wrapper and make a PR! We will do our best to review your code.

[cyberslam]

Hi Martin,

       Thank you for your prompt reply.  Yes, I can wait for the "big API" change - not sure you were referring to the HTM.core or your code though.
       I will look into how to use ROOT for c++ and your code.
       My intention was to follow the Nupic's latest notebook code to gain some understanding on how to use their technology.  While your code is reportedly faster but I need a step-by-step tutorial to gain insight and then make some modifications to suit my use.
       Your talent in C++ is outstanding and I appreciate your contribution to the open source AI field.

Cheers,

Samuel Lam

[marty1885]

No, the API change won't give us NuPIC compatibility. The core design is so different that it is virtually impossible. But the API change will get rid of some quarks in Etaler's API (which loosely follows NuPIC's API).

For example, these disaster should disappear:

//In NuPIC:
SpatialPooler sp;
sp.setGlobalInfibition(false); //Use local inhibition
sp.setGlobalDensity(0.06); //Useless, global inhibition disabled
sp.setLocalAreaDensity(-0.1); //Negative value means using alternative local density mode
sp.setNumActiveColumnsPerInhArea(16); //Override the local density settings, **again**

I'm still working on the replacement mechanism.

[marty1885]

The frontend design of Etaler is somewhat similar NuPIC's. Well, it has less feature and is in C++. You might want to have a look at example1.cpp of a start. And fell free to create issues when you can't find something or something is not working. It is likely not implemented or a bug.

Also, yeah, I should have made more examples and tutorials when I'm free. Help wanted.

[cyberslam]

Just had a look at ROOT: it just another C++ interpreter. I am afraid to use it since when I got stuck, not many could help me nor can I google for help.

I understand Etaler is a fork of Nupic. It would be great if you could keep your API the same with Nupic. Etaler's speed advantage is moot if we couldn't use the same python code in jupyter/colab.

I want to work with you on Etaler's examples and tutorials using a simple IDE like Code-Blocks.

[dan-stromberg]

On 7/31/19 10:43 AM, cyberslam wrote:

I understand Etaler is a fork of Nupic. It would be great if you could keep your API the same with Nupic. Etaler's speed advantage is moot if we couldn't use the same python code in jupyter/colab.

Is it? I thought Etaler was coded from scratch using HTM ideas.

htm.core is a fork of nupic, but it has a somewhat different license (effectively speaking), and it isn't fully compatible.

HTH.

[marty1885]

Yes, Etalter is a completely from-scratch implementation of HTM. And the performance gains comes from it.Thus, it is not compatible with NuPIC/HTN.core

[marty1885]

@cyberslam Run root --notebook and a C++ Jupyter notebook will show up.

[marty1885]

It is way too early to write a Python wrapper. But it might be a good time to start discussing how we will implement one.

There are 2 options. pybind11 or cppyy. Both having their pros and cons

pybind11

pybind11 is the standard solution. It is popular library that binds C++ to python.

Pros:

• Large community
• Pythonic interface

Cons:

• A lot of code to maintain
  • We need to write code for every method and variable
• Pron to interface bugs
• Needs to update the binding whenever we change the API

cppyy

cppyy uses cling to parse and generate bindings at runtime.

Pros:

• Significant less code than pybind11
• Only needs update when there is a very huge change

Cons:

• We also have to ship C++ headers when distributing Python packages
• A lot heavier than pybind11
• Results in a C++ feeling interface
• May not work on Windows

[dan-stromberg]

There are others.  SWIG is the venerable one, and is perhaps a good choice if you plan to expose your C++ to more than Python alone.  Cython can generate very Pythonic wrappers, and is a Python-like language rather than just a binding generator.

Perhaps look over http://stromberg.dnsalias.org/~strombrg/speeding-python/

On 8/1/19 10:30 PM, Martin Chang wrote:

It is way too early to write a Python wrapper. But it might be a good time to start discussing how we will implement one.

There are 2 options. pybind11 https://github.com/pybind/pybind11 or cppyy https://bitbucket.org/wlav/cppyy. Both having their pros and cons

pybind11

pybind11 is the standard solution. It is popular library that binds C++ to python.

Pros:

• Large community
• Pythonic interface

Cons:

• A lot of code to maintain o We need to write code for every method and variable
• Pron to interface bugs

• Needs to update the binding whenever we change the API

  cppyy

cppyy uses cling https://github.com/root-project/cling to parse and generate bindings at runtime.

Pros:

• Significant less code than pybind11
• Only needs update when there is a very huge change

Cons:

• We also have to ship C++ headers when distributing Python packages
• A lot heavier than pybind11
• Results in a C++ feeling interface
• May not work on Windows

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/etaler/Etaler/issues/45?email_source=notifications&email_token=AMGVVMLQGIY6VXX6K5BY5F3QCPA7VA5CNFSM4IH7ACF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD3MUNXY#issuecomment-517555935, or mute the thread https://github.com/notifications/unsubscribe-auth/AMGVVMOXQTSBINSBWQFH67TQCPA7VANCNFSM4IH7ACFQ.

[marty1885]

I'll hopefully make a binding using cppyy described above in the next new moths. As I've been using Etaler in Python through ROOT (Based on cppyy). The interface works quite nicely. I'll just have to figure out how to get tantalizer lists working.

[dan-stromberg]

That's awesome.

I'd love to pitch ETaler to my company for use in our large CPython project.

Thanks!

On Fri, Oct 25, 2019 at 8:14 PM Martin Chang notifications@github.com wrote:

I'll hopefully make a binding using cppyy described above in the next new moths. As I've been using Etaler in Python through ROOT (Based on cppyy). The interface works quite nicely. I'll just have to figure out how to get tantalizer lists working.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/etaler/Etaler/issues/45?email_source=notifications&email_token=AMGVVMLYVAWTRYWEYMIN2MTQQOYYFA5CNFSM4IH7ACF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECJ6GEQ#issuecomment-546562834, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMGVVMIYTMU2P5TJ7DWYU2LQQOYYFANCNFSM4IH7ACFQ .

[marty1885]

@dan-stromberg Great to hear that! Do you have any deadline? I can try making the binding before that.

[dan-stromberg]

We're using nupic, and htm.core isn't an option for us because of its (de facto) license. So the Python 2.x deadline on January 1st is a bit ominous. We've got most of our code moved to CPython 3.x, but not all of it because of nupic getting left behind.

Thanks for taking an interest in this.

On Mon, Oct 28, 2019 at 11:51 PM Martin Chang notifications@github.com wrote:

@dan-stromberg https://github.com/dan-stromberg Great to hear that! Do you have any deadline? I can try making the binding before that.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/etaler/Etaler/issues/45?email_source=notifications&email_token=AMGVVMLBYCOJNTCFY67BMD3QQ7MPRA5CNFSM4IH7ACF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECPNZSQ#issuecomment-547282122, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMGVVMK3VLQ2NZBHU5HM25LQQ7MPRANCNFSM4IH7ACFQ .

[marty1885]

I'll update you once I have something to share. But time might be tight.

Happy to help.

[dan-stromberg]

I can't guarantee that management will go for it, but I'd love to suggest etaler.

It's possible we'll end up rewriting the nupic-utilizing code in Go, or just continuing with CPython 2.x past the 2.x EOL.

Thanks.

On Tue, Oct 29, 2019 at 8:14 PM Martin Chang notifications@github.com wrote:

I'll update you once I have something to share. But time might be tight.

Happy to help.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/etaler/Etaler/issues/45?email_source=notifications&email_token=AMGVVMPES7JQWINLPLDAK63QRD33BA5CNFSM4IH7ACF2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECSY77Y#issuecomment-547721215, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMGVVMLR4EILQILEWWSRRX3QRD33BANCNFSM4IH7ACFQ .

[marty1885]

It's a lot easier that I thought. A PoC is working already. https://github.com/etaler/PyEtaler

[marty1885]

PyEtaler is the official binding now. It is quite usable and convers 99% of Etaler's functions.

Closing.