pickle error in windows 10

[aliirmak]

i am using conda. i am getting the following pickle error: AttributeError: Can't pickle local object 'Space.init..'

The full stack is:

`Traceback (most recent call last):

File "C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin2.py", line 97, in optimizer.optimize("example_1d_branin_scenario.json", branin_function_1d)

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\optimizer.py", line 125, in optimize config, black_box_function=black_box_function, profiling=profiling

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\bo.py", line 447, in main profiling,

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\random_scalarizations.py", line 541, in random_scalarizations profiling=profiling,

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\local_search.py", line 470, in local_search process.start()

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\process.py", line 112, in start self._popen = self._Popen(self)

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 223, in _Popen return _default_context.get_context().Process._Popen(process_obj)

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 322, in _Popen return Popen(process_obj)

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\popen_spawn_win32.py", line 89, in init reduction.dump(process_obj, to_child)

File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj)

AttributeError: Can't pickle local object 'Space.init..'`

According to a pythorch forum entry I found, multiprocessing is not implemented in Windows (but then this was back in 2018). According to here, a possible solution is adding --num_workers=0 to command line. But it may be specific to pytorch.

What would be a possible solution to run this in Windows?

[arturluis]

Hey @aliirmak ,

We have just pushed a new version with a potential fix to this issue. Can you install the latest version (e.g. pip install hypermapper==2.2.5) and see if the error persists?

We are having some trouble replicating this issue, if it persists, can you try adding

"number_of_cpus": 1,

to the json and see if you still get the multiprocessing pickle error?

Thanks,

[aliirmak]

2.2.5 didn't fix the problem. modifying the json file by adding _"number_ofcpus": 1 made an optimization run provided in one of the example codes possible.

[arturluis]

Hi @aliirmak , do you get a new error with v2.2.5 or is it still the same issue as before?

[aliirmak]

It is still the same issue. Below is the full traceback:

  The 1d Branin function has one global optimum at x1 = 
  (x, y) at minimum is: (3.141592653589793,0.39788735772973816)
  Design of experiment phase, number of new doe samples = 3 .......
  x1,value,Timestamp
  3.250674453049861,0.46193520917883646,2
  8.359378914475585,5.655850297112645,2
  -3.245808676595992,105.5506003442002,2

  End of doe/resume phase, the number of evaluated configurations is: 3

  Starting optimization iteration 1
   C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin2.py:67: UserWarning:Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  Traceback (most recent call last):

    File "C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin2.py", line 97, in <module>
      optimizer.optimize("example_1d_branin_scenario.json", branin_function_1d)

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\optimizer.py", line 125, in optimize
      config, black_box_function=black_box_function, profiling=profiling

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\bo.py", line 447, in main
      profiling,

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\random_scalarizations.py", line 565, in random_scalarizations
      profiling=profiling,

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\local_search.py", line 539, in local_search
      process.start()

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\process.py", line 112, in start
      self._popen = self._Popen(self)

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 223, in _Popen
      return _default_context.get_context().Process._Popen(process_obj)

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 322, in _Popen
      return Popen(process_obj)

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
      reduction.dump(process_obj, to_child)

    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\reduction.py", line 60, in dump
      ForkingPickler(file, protocol).dump(obj)

  AttributeError: Can't pickle local object 'Space.__init__.<locals>.<lambda>'

   C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin2.py:67: UserWarning:Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 105, in spawn_main
      exitcode = _main(fd)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 114, in _main
      prepare(preparation_data)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 225, in prepare
      _fixup_main_from_path(data['init_main_from_path'])
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
      run_name="__mp_main__")
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\runpy.py", line 263, in run_path
      pkg_name=pkg_name, script_name=fname)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\runpy.py", line 96, in _run_module_code
      mod_name, mod_spec, pkg_name, script_name)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\runpy.py", line 85, in _run_code
      exec(code, run_globals)
    File "C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin2.py", line 97, in <module>
      optimizer.optimize("example_1d_branin_scenario.json", branin_function_1d)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\optimizer.py", line 125, in optimize
      config, black_box_function=black_box_function, profiling=profiling
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\bo.py", line 447, in main
      profiling,
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\random_scalarizations.py", line 565, in random_scalarizations
      profiling=profiling,
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\local_search.py", line 539, in local_search
      process.start()
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\process.py", line 112, in start
      self._popen = self._Popen(self)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 223, in _Popen
      return _default_context.get_context().Process._Popen(process_obj)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 322, in _Popen
      return Popen(process_obj)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
      prep_data = spawn.get_preparation_data(process_obj._name)
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
      _check_not_importing_main()
    File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
      is not going to be frozen to produce an executable.''')
  RuntimeError: 
          An attempt has been made to start a new process before the
          current process has finished its bootstrapping phase.

          This probably means that you are not using fork to start your
          child processes and you have forgotten to use the proper idiom
          in the main module:

              if __name__ == '__main__':
                  freeze_support()
                  ...

          The "freeze_support()" line can be omitted if the program
          is not going to be frozen to produce an executable.

[arturluis]

Hi @aliirmak ,

I was able to reproduce the "can't pickle X" error on a Windows machine and then was able to make a fix for it. The latest version of HyperMapper fixes this issue for me, can you pull (or install with pip) the version 2.2.7 and see if the error is fixed now?

Cheers,

[aliirmak]

Hi @arturluis,

Thanks for getting back!

The new pip didn't solve the problem for me. I am getting another error (not the same) when I run on spyder:

    The 1d Branin function has one global optimum at x1 = π
    (x, y) at minimum is: (3.141592653589793,0.39788735772973816)
    {
        "application_name": "1d_branin",
        "optimization_objectives": [
            "value"
        ],
        "design_of_experiment": {
            "number_of_samples": 3
        },
        "optimization_iterations": 10,
        "models": {
            "model": "gaussian_process"
        },
        "input_parameters": {
            "x1": {
                "parameter_type": "real",
                "values": [
                    -5,
                    10
                ]
            }
        }
    }
    Design of experiment phase, number of new doe samples = 3 .......
    x1,value,Timestamp
    8.802105477911676,2.2756929565792747,2
    3.3727935998446625,0.6834484269083685,2
    -2.284307410463647,68.27158439204081,3

    End of doe/resume phase, the number of evaluated configurations is: 3

    Starting optimization iteration 1
     C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin_trial_myversion.py:63: UserWarning:Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
    Traceback (most recent call last):

      File "C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin_trial_myversion.py", line 99, in <module>
        optimizer.optimize("example_1d_branin_scenario_myversion.json", branin_function_1d)

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\optimizer.py", line 125, in optimize
        config, black_box_function=black_box_function, profiling=profiling

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\bo.py", line 459, in main
        acquisition_function_optimizer,

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\random_scalarizations.py", line 571, in random_scalarizations
        profiling=profiling,

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\local_search.py", line 539, in local_search
        process.start()

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\process.py", line 112, in start
        self._popen = self._Popen(self)

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 322, in _Popen
        return Popen(process_obj)

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\popen_spawn_win32.py", line 89, in __init__
        reduction.dump(process_obj, to_child)

      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\reduction.py", line 60, in dump
        ForkingPickler(file, protocol).dump(obj)

    BrokenPipeError: [Errno 32] Broken pipe

     C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin_trial_myversion.py:63: UserWarning:Matplotlib is currently using agg, which is a non-GUI backend, so cannot show the figure.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 105, in spawn_main
        exitcode = _main(fd)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 114, in _main
        prepare(preparation_data)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 225, in prepare
        _fixup_main_from_path(data['init_main_from_path'])
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 277, in _fixup_main_from_path
        run_name="__mp_main__")
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\runpy.py", line 263, in run_path
        pkg_name=pkg_name, script_name=fname)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\runpy.py", line 96, in _run_module_code
        mod_name, mod_spec, pkg_name, script_name)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\runpy.py", line 85, in _run_code
        exec(code, run_globals)
      File "C:\Users\Ali Ozdagli\OneDrive - Vanderbilt\Research\surrogate modeling\hypermapper_trial\branin_trial_myversion.py", line 99, in <module>
        optimizer.optimize("example_1d_branin_scenario_myversion.json", branin_function_1d)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\optimizer.py", line 125, in optimize
        config, black_box_function=black_box_function, profiling=profiling
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\bo.py", line 459, in main
        acquisition_function_optimizer,
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\random_scalarizations.py", line 571, in random_scalarizations
        profiling=profiling,
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\site-packages\hypermapper\local_search.py", line 539, in local_search
        process.start()
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\process.py", line 112, in start
        self._popen = self._Popen(self)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 223, in _Popen
        return _default_context.get_context().Process._Popen(process_obj)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\context.py", line 322, in _Popen
        return Popen(process_obj)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\popen_spawn_win32.py", line 46, in __init__
        prep_data = spawn.get_preparation_data(process_obj._name)
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 143, in get_preparation_data
        _check_not_importing_main()
      File "C:\ProgramData\Anaconda3\envs\hypermapper\lib\multiprocessing\spawn.py", line 136, in _check_not_importing_main
        is not going to be frozen to produce an executable.''')
    RuntimeError: 
            An attempt has been made to start a new process before the
            current process has finished its bootstrapping phase.

            This probably means that you are not using fork to start your
            child processes and you have forgotten to use the proper idiom
            in the main module:

                if __name__ == '__main__':
                    freeze_support()
                    ...

            The "freeze_support()" line can be omitted if the program
            is not going to be frozen to produce an executable.`

In addition, when I run using jupyter, I don't get any error. But, compared to the case where I use "number_of_cpus": 1, code runs extremely slow.

I attached my minimal example branin_trial_myversion.zip.

[aliirmak]

as a follow-up, wrapping python script with if __name__=="__main__": makes it running. On the other hand, it suffers from extremely slow processing. "number_of_cpus": 1 flag still runs fastest.

[arturluis]

Hey @aliirmak ,

Sorry for the delay. I'm glad you're not getting the pickling issues anymore, although now there is this slowness issue. We are investigating it here and it is caused by some known issues between Windows and the python multiprocessing library that we use. We will work on a fix for it in the future, but it might take a little time before it is available, in the meantime, running on a single cpu (i.e. setting number_of_cpus to 1) will probably be better for you.

Best,

[aliirmak]

@arturluis

thanks for the update. i think multiprocessing lib for windows is working poorly and many people on the web are addressing this issue. at this point, the potential speed gain is possible only in linux.

let me know if you need additional input/testing later

[arturluis]

Hey @aliirmak

Exactly. This seems to be a well known issue with Windows, but there seem to be possible fixes/workarounds. We will investigate them and definitely contact you once we have potential fixes to do some testing!

In the meantime, yes, the parallelization speedup will work for linux (and probably macOS) only.

Best!

[arturluis]

Hey,

I'm closing this as the original issue has been addressed. We'll reopen this issue in the future if/when we work on the parallelization issue for Windows.

Best,