Examples using PYGMO fail with Boost 1.57

[astrojuanlu]

When trying to run examples 1, 3, 4 and 5 with latest PyKEP and PyGMO the "Monotonic Basin Hopping" process takes place but in the end all die with the same message:

Traceback (most recent call last):
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/multiprocessing/queues.py", line 266, in _feed
    send(obj)
RuntimeError: unregistered class - derived class not registered or exported

  File "_ex5.py", line 22, in run_example5
    archi = archipelago(algo,prob,8,20, topology=topo)
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/site-packages/PyGMO/core/__init__.py", line 342, in _generic_archi_ctor
    self.push_back(island(args[0], args[1], args[3]))
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/site-packages/PyGMO/core/__init__.py", line 55, in __get_deepcopy__
    return deepcopy(self)
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/copy.py", line 182, in deepcopy
    rv = reductor(2)
RuntimeError: unregistered class - derived class not registered or exported

Google suggests that it is a Boost problem, but I run all PyKEP and PyGMO test and all pass (I have boost 1.57.0). I will try with earlier versions of PyGMO or Boost to see what happens.

Edit: 4 does not run because of #12.

[darioizzo]

mmm ... let us know.... might want to reproduce this ..

[astrojuanlu]

I am already compiling PyKEP PaGMO 1.1.5 but I think this issue is similar to esa/pagmo#97.

Edit: typo.

[astrojuanlu]

I can confirm examples 1 and 3 run with PaGMO/PyGMO 1.1.5 and PyKEP a6dd41711, but example 5 still fails in the same way:

$ python -c "import _ex5; _ex5.run_example5()"
Problem name: <class 'PyKEP.trajopt._mga_1dsm.mga_1dsm'>
    Global dimension:           10
    Integer dimension:          0
    Fitness dimension:          1
    Constraints dimension:          0
    Inequality constraints dimension:   0
    Lower bounds: [5844, 0, 0, 0, 1.0000000000000001e-05, 255.67499999999998, -6.2831853071795862, 1.1000000000000001, 1.0000000000000001e-05, 1.0000000000000001e-05]
    Upper bounds: [6209, 1, 1, 2500, 0.99999000000000005, 1095.75, 6.2831853071795862, 30, 0.99999000000000005, 0.99999000000000005]
    Constraints tolerance: []

     Sequence: ['earth', 'venus', 'earth']
     Add launcher vinf to the objective?: False
     Add final vinf to the objective?: True
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "_ex5.py", line 22, in run_example5
    archi = archipelago(algo,prob,8,20, topology=topo)
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/site-packages/PyGMO/core/__init__.py", line 342, in _generic_archi_ctor
    self.push_back(island(args[0], args[1], args[3]))
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/site-packages/PyGMO/core/__init__.py", line 55, in __get_deepcopy__
    return deepcopy(self)
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/copy.py", line 182, in deepcopy
    rv = reductor(2)
RuntimeError: unregistered class - derived class not registered or exported

[astrojuanlu]

And finally, example 5 from PyKEP 1.1.3 (after conveniently replacing interplanetary with traj) fails again with the same error as above.

There are no more tagged releases of either PyKEP nor PaGMO in GitHub so these are all the combinations I could imagine :) If you provide me some hints on how to debug this I can try and see if I am able to isolate the issue.

[astrojuanlu]

Hello @darioizzo, have you been able to reproduce this issues? Is there anything else I can try from my side?

[darioizzo]

In my systems (OSX and linux) the examples run fine. Here an example of output for ex3:

Running Monotonic Basin Hopping ....

1. Local solution: [-1173.3464571941849] Global best: [-447.447846038335] i New solution accepted. Constraints vector: [-8.8034593500568289e-06, 2.7178569160536636e-05, 5.8233945095400916e-07, -3.9695514942277416e-05, -2.8295608351448263e-05, ... ]
2. Local solution: [-639.08663218338938] Global best: [-1173.3464571941849] i
3. Local solution: [-1179.4029635149034] Global best: [-1173.3464571941849] i
4. Local solution: [-1112.1147147515787] Global best: [-1173.3464571941849] i
5. Local solution: [-1252.0185483207781] Global best: [-1173.3464571941849] i
6. Local solution: [-1287.0394555555224] Global best: [-1173.3464571941849] i Is the solution found a feasible trajectory? False

So I would need to have informations on your system. Can you post the output of cmake?

[astrojuanlu]

Example 3 runs fine for me now, the problem seems to be example 5 (see https://github.com/esa/pykep/issues/13#issuecomment-72312055). This is the output of my cmake command:

+ cmake .. -DCMAKE_INSTALL_PREFIX=/home/juanlu/.miniconda3/envs/_build -DBUILD_PYKEP:BOOL=ON -DPYTHON_EXECUTABLE=/home/juanlu/.miniconda3/envs/_build/bin/python -DPYTHON_LIBRARY=/home/juanlu/.miniconda3/envs/_build/lib/libpython2.7.so -DPYTHON_INCLUDE_DIR=/home/juanlu/.miniconda3/envs/_build/include/python2.7 -DPYTHON_MODULES_PATH=/home/juanlu/.miniconda3/envs/_build/lib/python2.7/site-packages -DBOOST_ROOT=/home/juanlu/.miniconda3/envs/_build -DBoost_NO_SYSTEM_PATHS:BOOL=ON
-- The C compiler identification is GNU 4.9.2
-- The CXX compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Required Boost libraries: serialization;date_time;python
-- Boost version: 1.57.0
-- Found the following Boost libraries:
--   serialization
--   date_time
--   python
-- Detected Boost version: 105700
-- Boost include dirs: /home/juanlu/.miniconda3/envs/_build/include
-- Boost libraries: /home/juanlu/.miniconda3/envs/_build/lib/libboost_serialization.so;/home/juanlu/.miniconda3/envs/_build/lib/libboost_date_time.so;/home/juanlu/.miniconda3/envs/_build/lib/libboost_python.so
-- Found PythonLibs: /home/juanlu/.miniconda3/envs/_build/lib/libpython2.7.so (found version "2.7.9") 
-- Found PythonInterp: /home/juanlu/.miniconda3/envs/_build/bin/python (found version "2.7.9") 
-- Python libraries: /home/juanlu/.miniconda3/envs/_build/lib/libpython2.7.so
-- Python library: /home/juanlu/.miniconda3/envs/_build/lib/libpython2.7.so
-- Python interpreter is: /home/juanlu/.miniconda3/envs/_build/bin/python
-- Python packages dir is: site-packages
-- Python library version: 27
-- Python modules install path: lib/python2.7/site-packages
-- Configuring done
-- Generating done
-- Build files have been written to: /home/juanlu/.miniconda3/conda-bld/work/build

[darioizzo]

Example 5 also runs fine in our systems .... You must have something going on with the installation.

1) Use the current master branch in esa/pykep 2) run a simple evolution instead of using the archipealgo

 # archi = archipelago(algo,prob,8,20, topology=topo)
 pop = population(prob,20)
 pop = algo.evolve(pop)

2a) If all goes well try now an island:

 # archi = archipelago(algo,prob,8,20, topology=topo)
 #pop = population(prob,20)
 #pop = algo.evolve(pop)
 isl = island(algo,prob,20)
 isl.evolve(1)

What line creates the problem?

Let us know,

Dario

[astrojuanlu]

Using current master and PyGMO 1.1.5:

2) When using a simple evolution these are my results:

$ python -c "import _ex5; _ex5.run_example5()"
Problem name: <class 'PyKEP.trajopt._mga_1dsm.mga_1dsm'>
    Global dimension:           10
    Integer dimension:          0
    Fitness dimension:          1
    Constraints dimension:          0
    Inequality constraints dimension:   0
    Lower bounds: [5844, 0, 0, 0, 1.0000000000000001e-05, 255.67499999999998, -6.2831853071795862, 1.1000000000000001, 1.0000000000000001e-05, 1.0000000000000001e-05]
    Upper bounds: [6209, 1, 1, 2500, 0.99999000000000005, 1095.75, 6.2831853071795862, 30, 0.99999000000000005, 0.99999000000000005]
    Constraints tolerance: []

     Sequence: ['earth', 'venus', 'earth']
     Add launcher vinf to the objective?: False
     Add final vinf to the objective?: True
Running a Self-Adaptive Differential Evolution Algorithm .... on 8 parallel islands
Done!! Best solution found is: 7.79813899235 km / sec
First Leg: earth to venus
Departure: 2016-May-26 19:53:27.148396 (5990.82878644 mjd2000) 
Duration: 313.887216054days
VINF: 2.20832240182 km/sec
DSM after 116.616764793 days
DSM magnitude: 3113.75494467m/s

leg no. 2: venus to earth
Duration: 446.589772133days
Fly-by epoch: 2017-Apr-05 17:11:02.615495 (6304.71600249 mjd2000) 
Fly-by radius: 14.9535697425 planetary radii
DSM after 120.597369 days
DSM magnitude: 2957.69171647m/s

Arrival at earth
Arrival epoch: 2018-Jun-26 07:20:18.927749 (6751.30577463 mjd2000) 
Arrival Vinf: 1726.69233121m/s
Total mission time: 2.0820725207 years

2a) When using an island though the example fails:

$ python -c "import _ex5; _ex5.run_example5()"
Problem name: <class 'PyKEP.trajopt._mga_1dsm.mga_1dsm'>
    Global dimension:           10
    Integer dimension:          0
    Fitness dimension:          1
    Constraints dimension:          0
    Inequality constraints dimension:   0
    Lower bounds: [5844, 0, 0, 0, 1.0000000000000001e-05, 255.67499999999998, -6.2831853071795862, 1.1000000000000001, 1.0000000000000001e-05, 1.0000000000000001e-05]
    Upper bounds: [6209, 1, 1, 2500, 0.99999000000000005, 1095.75, 6.2831853071795862, 30, 0.99999000000000005, 0.99999000000000005]
    Constraints tolerance: []

     Sequence: ['earth', 'venus', 'earth']
     Add launcher vinf to the objective?: False
     Add final vinf to the objective?: True
Running a Self-Adaptive Differential Evolution Algorithm .... on 8 parallel islands
Traceback (most recent call last):
  File "/home/juanlu/.miniconda3/envs/pykep27/lib/python2.7/multiprocessing/queues.py", line 266, in _feed
    send(obj)
RuntimeError: unregistered class - derived class not registered or exported

[darioizzo]

It seems that in your system boost serialization has some kind of problem ... my guess is that it is related to this http://stackoverflow.com/questions/4432019/boost-serialization-exception-unregistered-class-serializing-polymorphic-base.

The simplest way of tackling this (unfortunately I cannot help as I cannot reproduce this problem in our set ups) is to

1) Understand if the problem is in pykep or in pygmo. Try using simpler problems in pygmo (e.g. problem.ackley(50)) and launch island/archipelago evolutions on it. 2) Try to see if you can reproduce the error in a simple c++ file 3) Locate what class has the serialization problem. 4) try different options as suggested in the stackoverflow thread.

Let me know....

[astrojuanlu]

Thanks a lot for the pointers. If I have to get into the C++ it will take me some time but I will try. Just for curiosity, what version of boost are you using? I don't know any other hints on what can be the difference between our setups. I assume you are building in Linux, am I right? On Feb 8, 2015 12:43 PM, "Dario Izzo" notifications@github.com wrote:

It seems that in your system boost serialization has some kind of problem ... my guess is that it is related to this http://stackoverflow.com/questions/4432019/boost-serialization-exception-unregistered-class-serializing-polymorphic-base.

The simplest way of tackling this (unfortunately I cannot help as I cannot reproduce this problem in our set ups) is to

1) Understand if the problem is in pykep or in pygmo. Try using simpler problems in pygmo (e.g. problem.ackley(50)) and launch island/archipelago evolutions on it. 2) Try to see if you can reproduce the error in a simple c++ file 3) Locate what class has the serialization problem. 4) try different options as suggested in the stackoverflow thread.

Let me know....

Reply to this email directly or view it on GitHub https://github.com/esa/pykep/issues/13#issuecomment-73407411.

[astrojuanlu]

I changed the title of the issue because I found the root cause. I tried with Boost 1.48 and 1.55 and all the examples run fine, but the problems come with Boost 1.57. This is more a PyGMO issue anyway, but at least now I have more hints to debug it.

[darioizzo]

Strange though, on my mac I have boost 1.57 and examples work fine ..... keep digging and let us know ...

[spichardo]

Hi,

I'm running in a similar issue with my own problem to optimize

I have to port my original PyGMO project (which works with no issue) to PagMO since I need to use MPI (very, very slow evaluation of solution vector),

I also have Boost 1.57, Pagmo/Pygmo 1.1.5, GCC 4.4,7 here I'm copying a barebone code , I used the code of cec2009 as a model to code my problem

the example in https://esa.github.io/pagmo/mpi_support.html runs with no issue

Following you can see a barebone code that can be used to show the error

#include "pagmo.h"
#include "problem/base.h"
using namespace pagmo;

namespace pagmo { namespace problem {
    class __PAGMO_VISIBLE UltrasoundPropertiesProblem:public base
    {
    public:
        UltrasoundPropertiesProblem();
        base_ptr clone() const;

    protected:
        void objfun_impl(fitness_vector &, const decision_vector &) const;
        void compute_constraints_impl(constraint_vector &, const decision_vector &) const;
};
}}

BOOST_CLASS_EXPORT_KEY(pagmo::problem::UltrasoundPropertiesProblem)

namespace pagmo { namespace problem {
    UltrasoundPropertiesProblem::UltrasoundPropertiesProblem():base(12,0,3,12,12,1e-12)
    {
        const double lb[] = {1470.,1470.,1470.,750.,750.,750.,1.,1.,1.,1.,1.,1.};
        const double ub[] = {1500.,3000.,3000.,2250.,2250.,2250.,20.,20.,20.,40.,40.,40.};
        set_bounds(lb,ub);

    }

    base_ptr UltrasoundPropertiesProblem::clone() const
    {
        return base_ptr(new UltrasoundPropertiesProblem(*this));
    }

    void UltrasoundPropertiesProblem::objfun_impl(fitness_vector &fv, const decision_vector &xv) const
    {
        int NRank = mpi_environment::get_rank();
         // THIS is just dummy code for the moment
        const double * x=&xv[0];
        double * f=&fv[0];
        f[0]=x[0]+x[1]+x[2]+x[3];
        f[1]=x[4]+x[5]+x[6]+x[7];
        f[2]=x[8]+x[9]+x[10]+x[11];
    }

    void UltrasoundPropertiesProblem::compute_constraints_impl(constraint_vector &cv, const decision_vector & xv) const
    {

        int NRank = mpi_environment::get_rank();

        const double * x=&xv[0];
        double * c=&cv[0];

         // THIS is just dummy code for the moment

        c[0]=-1.;
        c[1]=-1.;
        c[2]=-1.;
        c[3]=-1.;
        c[4]=-1.;
        c[5]=-1.;
        c[6]=-1.;
        c[7]=-1.;
        c[8]=-1.;
        c[9]=-1.;
        c[10]=-1.;
        c[11]=-1.;

    }

}}

int main()
{
        // Initialise the MPI environment.
        mpi_environment env;

        // Create a problem and an algorithm.
        pagmo::problem::base_ptr prob=problem::UltrasoundPropertiesProblem().clone();

        pagmo::problem::base_ptr dprob = problem::death_penalty(*prob,pagmo::problem::death_penalty::SIMPLE).clone();

        algorithm::base_ptr algo = algorithm::nsga2().clone();
        // Create an archipelago of 10 MPI islands.
        archipelago a;
        a.set_topology(topology::ring());
        for (int i = 0; i < 10; ++i) {
                a.push_back(mpi_island(*algo,*dprob,8));
        }
        // Evolve the archipelago 10 times.
        a.evolve(1);
        a.join();
        return 0;
}

I compile it with: mpic++ -std=c++0x -pthread -DNDEBUG -I/usr/local/include/pagmo/src/ -DPAGMO_ENABLE_MPI -L/usr/local/lib/ -lpagmo main.cpp

when trying to run it, I get

mpiexec --hostfile hostfile -n 10 -N 1 a.out
Error during island evolution using Nondominated Sorting Genetic Algorithm II (NSGA-II): unregistered class - derived class not registered or exported
...

good to know that maybe I can fix this by just downgrading boost to 1.55, I'll give it shot..

[astrojuanlu]

This problem is still present using PyKEP 1.2.0 and latest PyGMO. What I find striking is that if I downgrade Boost I don't even need a rebuild of the libraries... this is definitely worth investigating.

[darioizzo]

I am closing this issue as the examples (except number 4) are being updated. In case they do not work in the new code PyKEP 1.2.2, open a new ticket.

[astrojuanlu]

I can confirm all the examples run now, even with latest Boost (probably thanks to #29). Thanks! :smile: