Python 3.x support ?

[jschueller]

Optizelle does not support Python 3.x, are there any plans to support it ?

[jschueller]

saw the forum...

[josyoun]

Yeah, sorry to slow play this for so long. The intention is to deprecate Python 2.7 support and move completely to Python 3.x. The reason that it has been delayed for so long is that I wanted to move to a different infrastructure and algorithm that'd be more robust and reliable than what we have now. And, unfortunately, that has taken a massive amount of time. I do have a prototype code that I've been using on contracts, which has been helpful for working out any algorithmic bugs and it's close to being finalized. In truth, I've been a bit quiet about announcing progress because I'm hesitate to overpromise and underdeliver, but it really is coming and most likely within the year.

Thanks for checking-in and stay tuned.

[jschueller]

maybe you'd be willing to publish it on a branch here so I can play with it ?

[josyoun]

In a more equitable world, that would be the right thing to do. And, to be sure, I'd very much like to. That said, there's a bit of risk in doing so, so I'll have to wait for now.

Basically, I believe there's risk due to how intellectual property law works in the U.S. The new code uses algorithms that are new. It's my intention to release them openly and produce a code under an open source license, but I think it's important to get the ordering right, which is (1) publish the algorithms in a refereed journal and (2) release the code onto the internet. What I don't want to have happen is have someone scrape the algorithm and then do something like file for patent protection, which would then lock all of us out. Now, theoretically, this is all math, so it shouldn't be patentable, but it does happen in practice as long as it's worded correctly and this has affected me personally with some of the work that I do. Further, even if the algorithm is published, which should provide prior art to protect against a patent, it still may not be enough as what happens to this guy shows. Anyway, I don't think there's a perfect way to protect against all challenges, but I've been harmed by this in the past, so I'm a little conservative with how I work.

And, again, to be clear, I'm not particularly happy about the IP situation. At the end of the day, though, I think it'll be worth the wait if we can get something out that is open source and free of IP concerns.

[jschueller]

did you not split the work on algorithms from the python3 port on separate branches ? I'd be only interested in the latter :)

[josyoun]

Yeah, unfortunately, everything is wrapped up together. Basically, the algorithms affect the interfaces and vice versa, so I don't have a great way of separating them. It'll all come soon though ;-)

[jschueller]

Any news on this ?

[josyoun]

This should be complete in commit 6abe5d5aaf388e4230bfde40644960bdd83f9ed4. For the most part, I did a straightforward migration to Python 3.8, so it may not be in current Python style. That said, it works on all of my platforms, so if there are no immediate issues I'll generate new installers and push to master.

[josyoun]

I should say, if you want to try this, the current develop branch has all of the updates.

[jschueller]

I tried it, unfortunately it crashes on import (Linux/python3.8.2), likely in the compiled module:

#0  0x00007fffec6ebd20 in ?? ()
#1  0x00007ffff7c4a23f in _PyImport_LoadDynamicModuleWithSpec () from /usr/lib/libpython3.8.so.1.0
#2  0x00007ffff7c4a65d in ?? () from /usr/lib/libpython3.8.so.1.0
#3  0x00007ffff7b17397 in ?? () from /usr/lib/libpython3.8.so.1.0
#4  0x00007ffff7b10f80 in PyVectorcall_Call () from /usr/lib/libpython3.8.so.1.0
#5  0x00007ffff7bc9b20 in _PyEval_EvalFrameDefault () from /usr/lib/libpython3.8.so.1.0
#6  0x00007ffff7bb48f4 in _PyEval_EvalCodeWithName () from /usr/lib/libpython3.8.so.1.0
#7  0x00007ffff7bb5c7b in _PyFunction_Vectorcall () from /usr/lib/libpython3.8.so.1.0

[josyoun]

Could you try building from a fresh build directory? I was trying to debug this issue on a Linux build that had an old Python 2.7 build as well. At first, it was segfaulting as well, but as I started to compile and recompile everything to get debugging flags, the segfault disappeared. Now, I can't reproduce it with a release build and no debugging flags, release build with debugging flags, or a debug build. Basically, the segfault evaporated once I rebuild everything enough times.

[jschueller]

I tried a clean build with different debug options, it gives me a more useful error this time, it seem to come from Optizelle::Python::Messaging::python:

Program received signal SIGSEGV, Segmentation fault.
std::_Function_handler<void (std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&), Optizelle::Python::Messaging::python(Optizelle::Python::PyObjectPtr const&)::$_0>::_M_invoke(std::_Any_data const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) (__functor=..., __args=<error reading variable: Cannot access memory at address 0x2>)
    at /usr/bin/../lib/gcc/x86_64-linux-gnu/7.5.0/../../../../include/c++/7.5.0/bits/std_function.h:316
316             (*_Base::_M_get_pointer(__functor))(

[josyoun]

Is it one of the unit tests that crashes or your own code? If it's one of the unit tests, which one?

[jschueller]

no just "import Optizelle"

[josyoun]

I just pushed a new update that fixes a minor problem. That said, the code works for me on three different Linux systems, two Windows, and two macOS. If this doesn't work, what distribution are you using? Not sure if I'll need to recreate that environment.

[jschueller]

It works now

[josyoun]

Ok. Great! One of my test systems is Arch, so I was beating my head over this. Give me a bit and I'll push to master.