pickle related error

[rgokulsm]

I'm trying to run the demo examples (through anaconda / jupyter) but am hitting some pickle related issues **AttributeError: Can't pickle local object 'local_search.<locals>.parallel_optimization_function'** I am not familiar with the issue - I tried googling about this but could not find a satisfactory answer. Could you please tell me if this is an error specific to my system / setup or if this is a known issue?

Thank you!

[arturluis]

Hi @rgokulsm ,

I can't say I have encountered that issue before, but I'd like to investigate it a bit further. Can you provide me the information below so I can try to replicate the issue?

• Which cell of the notebook triggered that error?
• Did HM generate a log file of the execution (usually called hypermapper_logfile.log)? If so, please share that file with us
• What is your machine setup (E.g. OS and python version)?

Any other details you have that might be of note, let me know as it helps us understand what caused this issue.

Cheers, Artur Souza

[rgokulsm]

Thank you Artur!!

Actually I run into two different errors I'm showcasing them in separate comments:

Error 1 - The code is pasted below:

import json

scenario = {}
scenario["application_name"] = "1d_branin"
scenario["optimization_objectives"] = ["value"]

number_of_RS = 2
scenario["design_of_experiment"] = {}
scenario["design_of_experiment"]["number_of_samples"] = number_of_RS
scenario["optimization_iterations"] = 8

scenario["models"] = {}
scenario["models"]["model"] = "gaussian_process"

scenario["input_parameters"] = {}
x1 = {}
x1["parameter_type"] = "ordinal"
x1["values"] = [-5.0, -4.5, -4.0, -3.5, -3.0, -1.5, -1.0, -0.5, 0.0, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 7.0, 7.5,
                8.0, 8.5, 9.0, 10.0]

scenario["input_parameters"]["x1"] = x1

with open("example_ordinal_1d_branin_scenario.json", "w") as scenario_file:
    json.dump(scenario, scenario_file, indent=4)

f = open("example_ordinal_1d_branin_scenario.json", "r")
text = f.read()
print(text, flush=True)
f.close()

import math
def branin_function_1d(X):
    # The function must receive a dictionary
    x1 = X['x1']

    # Branin function computation
    a = 1.0
    b = 5.1 / (4.0 * math.pi * math.pi)
    c = 5.0 / math.pi
    r = 6.0
    s = 10.0
    t = 1.0 / (8.0 * math.pi)
    x2 = 2.275
    value = a * (x2 - b * x1 * x1 + c * x1 - r) ** 2 + s * (1 - t) * math.cos(x1) + s

    # The function must return the objective value (a number)
    return value

import sys
import os
from hypermapper import optimizer
stdout = sys.stdout # Jupyter uses a special stdout and HyperMapper logging overwrites it. Save stdout to restore later

# Call HyperMapper to optimize the 1d Branin function
optimizer.optimize("example_ordinal_1d_branin_scenario.json", branin_function_1d)
sys.stdout = stdout

The error:

Design of experiment phase, number of new doe samples = 2 .......
x1,value,Timestamp
2.5,2.6137237843308414,18
-3.5,119.33746909170875,20

End of doe/resume phase, the number of evaluated configurations is: 2

Starting optimization iteration 1
---------------------------------------------------------------------------
ZeroDivisionError                         Traceback (most recent call last)
<ipython-input-6-357412dd26bf> in <module>
      5 
      6 # Call HyperMapper to optimize the 1d Branin function
----> 7 optimizer.optimize("example_ordinal_1d_branin_scenario.json", branin_function_1d)
      8 sys.stdout = stdout

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/optimizer.py in optimize(parameters_file, black_box_function)
    122         or (optimization_method == "prior_guided_optimization")
    123     ):
--> 124         data_array = bo.main(
    125             config, black_box_function=black_box_function, profiling=profiling
    126         )

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/bo.py in main(config, black_box_function, profiling)
    434         local_search_t0 = datetime.datetime.now()
    435         if epsilon > epsilon_greedy_threshold:
--> 436             best_configuration = bo_method(
    437                 config,
    438                 data_array,

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/random_scalarizations.py in random_scalarizations(config, data_array, param_space, fast_addressing_of_data_array, regression_models, iteration_number, objective_weights, objective_limits, classification_model, profiling)
    528     optimization_function_parameters["number_of_cpus"] = config["number_of_cpus"]
    529 
--> 530     _, best_configuration = local_search(
    531         local_search_starting_points,
    532         local_search_random_points,

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/local_search.py in local_search(local_search_starting_points, local_search_random_points, param_space, fast_addressing_of_data_array, enable_feasible_predictor, optimization_function, optimization_function_parameters, scalarization_key, number_of_cpus, previous_points, profiling, noise)
    398 
    399         # the number of splits of the list of input points that each process is expected to handle
--> 400         uniform_partition_fraction = len(uniform_configurations) / (
    401             partitions_per_cpu * number_of_cpus
    402         )

ZeroDivisionError: division by zero

Log File

{'application_name': '1d_branin', 'optimization_objectives': ['value'], 'design_of_experiment': {'number_of_samples': 2, 'doe_type': 'random sampling'}, 'optimization_iterations': 8, 'models': {'model': 'gaussian_process', 'number_of_trees': 10, 'max_features': 0.5, 'bootstrap': False, 'min_samples_split': 5}, 'input_parameters': {'x1': {'parameter_type': 'ordinal', 'values': [-5.0, -4.5, -4.0, -3.5, -3.0, -1.5, -1.0, -0.5, 0.0, 2.0, 2.5, 3.0, 3.5, 4.0, 4.5, 5.0, 5.5, 7.0, 7.5, 8.0, 8.5, 9.0, 10.0], 'prior': 'uniform'}}, 'log_file': 'hypermapper_logfile.log', 'profiling': False, 'noise': True, 'profiling_file': 'profiles/profile.csv', 'append_profiles': False, 'number_of_cpus': 0, 'max_number_of_predictions': 1000000, 'time_budget': -1, 'number_of_repetitions': 1, 'hypermapper_mode': {'mode': 'default'}, 'output_image': {'output_image_pdf_file': 'output_pareto.pdf', 'image_xlog': False, 'image_ylog': False}, 'feasible_output': {'name': 'Valid', 'true_value': 'True', 'false_value': 'False', 'enable_feasible_predictor': False, 'enable_feasible_predictor_grid_search_on_recall_and_precision': False, 'feasible_predictor_grid_search_validation_file': '/home/lnardi/spatial-lang/results/apps_classification_test_set/BlackScholes.csv'}, 'timestamp': 'Timestamp', 'evaluations_per_optimization_iteration': 1, 'run_directory': '.', 'output_data_file': 'output_samples.csv', 'output_pareto_file': 'output_pareto.csv', 'acquisition_function': 'EI', 'scalarization_method': 'tchebyshev', 'weight_sampling': 'flat', 'bounding_box_limits': [0, 1], 'optimization_method': 'bayesian_optimization', 'local_search_starting_points': 10, 'local_search_random_points': 10000, 'local_search_evaluation_limit': -1, 'scalarization_key': 'scalarization', 'local_search_scalarization_weights': [1], 'print_parameter_importance': False, 'normalize_inputs': False, 'epsilon_greedy_threshold': 0.1, 'model_posterior_weight': 10, 'model_good_quantile': 0.05, 'prior_estimation_file': 'samples.csv', 'prior_estimation_quantile': 0.1, 'estimate_multivariate_priors': False, 'resume_optimization': False, 'resume_optimization_data': 'output_samples.csv', 'bandwidth_parameter': 0, 'bandwidth_n_factor': 100, 'prior_limit_estimation_points': 10000, 'posterior_computation_lower_limit': 1e-08, 'custom_gaussian_prior_means': [0], 'custom_gaussian_prior_stds': [-1], 'acquisition_function_optimizer': 'local_search', 'evolution_population_size': 50, 'evolution_generations': 150, 'mutation_rate': 1, 'evolution_crossover': False, 'regularize_evolution': False, 'batch_size': 2, 'print_best': 'auto', 'print_posterior_best': False}
Design of experiment phase, number of new doe samples = 2 .......
x1,value,Timestamp
-4.5,190.30580997466993,7
0.0,33.47773764227026,10

End of doe/resume phase, the number of evaluated configurations is: 2

End of DoE - Time     0.0289 sec
Starting optimization iteration 1
End of training - Time       0.19 sec
Model fitting time     0.1862 sec
Total RS time     0.0004 sec

[rgokulsm]

Error 2 here -

Code:

import math
def branin_function_1d(X):
    # The function must receive a dictionary
    x1 = X['x1']

    # Branin function computation
    a = 1.0
    b = 5.1 / (4.0 * math.pi * math.pi)
    c = 5.0 / math.pi
    r = 6.0
    s = 10.0
    t = 1.0 / (8.0 * math.pi)
    x2 = 2.275
    value = a * (x2 - b * x1 * x1 + c * x1 - r) ** 2 + s * (1 - t) * math.cos(x1) + s

    # The function must return the objective value (a number)
    return value

import matplotlib
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
%matplotlib inline
plt.rcParams['figure.figsize'] = [12, 6]
plt.rcParams['font.size'] = 18
point_size = matplotlib.rcParams['lines.markersize']**2.8
point_size_optimum = matplotlib.rcParams['lines.markersize']**2

optimum = math.pi
value_at_optimum=branin_function_1d({'x1': optimum})

# Sample 1000 (x,y) pairs from the function to plot its curve
branin_line_xs = np.linspace(-5, 10, 1000)
branin_line_ys = []
for x in branin_line_xs:
    y = branin_function_1d({'x1': x})
    branin_line_ys.append(y)
plt.plot(branin_line_xs, branin_line_ys, label="1D Branin Function")

# Mark the known optimum on the curve
plt.scatter(optimum, value_at_optimum, s=point_size_optimum, marker='o', color="black", label="Minimum")

plt.legend()
plt.xlabel("x1")
plt.ylabel("value")
plt.show()
print("The 1d Branin function has one global optimum at x1 = \u03C0", flush=True)
print("(x, y) at minimum is: ("+str(optimum)+","+str(value_at_optimum)+")", flush=True)

import json
scenario = {}
scenario["application_name"] = "1d_branin"
scenario["optimization_objectives"] = ["value"]

number_of_RS = 3
scenario["design_of_experiment"] = {}
scenario["design_of_experiment"]["number_of_samples"] = number_of_RS

scenario["optimization_iterations"] = 10

scenario["models"] = {}
scenario["models"]["model"] = "gaussian_process"

scenario["input_parameters"] = {}
x1 = {}
x1["parameter_type"] = "real"
x1["values"] = [-5, 10]

scenario["input_parameters"]["x1"] = x1

with open("example_1d_branin_scenario.json", "w") as scenario_file:
    json.dump(scenario, scenario_file, indent=4)

f = open("example_1d_branin_scenario.json", "r")
text = f.read()
print(text, flush=True)
f.close()

import sys
import os
from hypermapper import optimizer
stdout = sys.stdout # Jupyter uses a special stdout and HyperMapper logging overwrites it. Save stdout to restore later

# Call HyperMapper to optimize the 1d Branin function
optimizer.optimize("example_1d_branin_scenario.json", branin_function_1d)
sys.stdout = stdout

Error message:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-11-0f7f56e76637> in <module>
      5 
      6 # Call HyperMapper to optimize the 1d Branin function
----> 7 optimizer.optimize("example_1d_branin_scenario.json", branin_function_1d)
      8 sys.stdout = stdout

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/optimizer.py in optimize(parameters_file, black_box_function)
    122         or (optimization_method == "prior_guided_optimization")
    123     ):
--> 124         data_array = bo.main(
    125             config, black_box_function=black_box_function, profiling=profiling
    126         )

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/bo.py in main(config, black_box_function, profiling)
    434         local_search_t0 = datetime.datetime.now()
    435         if epsilon > epsilon_greedy_threshold:
--> 436             best_configuration = bo_method(
    437                 config,
    438                 data_array,

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/random_scalarizations.py in random_scalarizations(config, data_array, param_space, fast_addressing_of_data_array, regression_models, iteration_number, objective_weights, objective_limits, classification_model, profiling)
    528     optimization_function_parameters["number_of_cpus"] = config["number_of_cpus"]
    529 
--> 530     _, best_configuration = local_search(
    531         local_search_starting_points,
    532         local_search_random_points,

/opt/anaconda3/envs/env_6_2021/lib/python3.9/site-packages/hypermapper/local_search.py in local_search(local_search_starting_points, local_search_random_points, param_space, fast_addressing_of_data_array, enable_feasible_predictor, optimization_function, optimization_function_parameters, scalarization_key, number_of_cpus, previous_points, profiling, noise)
    454 
    455             for process in processes:
--> 456                 process.start()
    457                 input_queue.put(None)
    458             input_queue.join()

/opt/anaconda3/envs/env_6_2021/lib/python3.9/multiprocessing/process.py in start(self)
    119                'daemonic processes are not allowed to have children'
    120         _cleanup()
--> 121         self._popen = self._Popen(self)
    122         self._sentinel = self._popen.sentinel
    123         # Avoid a refcycle if the target function holds an indirect

/opt/anaconda3/envs/env_6_2021/lib/python3.9/multiprocessing/context.py in _Popen(process_obj)
    222     @staticmethod
    223     def _Popen(process_obj):
--> 224         return _default_context.get_context().Process._Popen(process_obj)
    225 
    226 class DefaultContext(BaseContext):

/opt/anaconda3/envs/env_6_2021/lib/python3.9/multiprocessing/context.py in _Popen(process_obj)
    282         def _Popen(process_obj):
    283             from .popen_spawn_posix import Popen
--> 284             return Popen(process_obj)
    285 
    286     class ForkServerProcess(process.BaseProcess):

/opt/anaconda3/envs/env_6_2021/lib/python3.9/multiprocessing/popen_spawn_posix.py in __init__(self, process_obj)
     30     def __init__(self, process_obj):
     31         self._fds = []
---> 32         super().__init__(process_obj)
     33 
     34     def duplicate_for_child(self, fd):

/opt/anaconda3/envs/env_6_2021/lib/python3.9/multiprocessing/popen_fork.py in __init__(self, process_obj)
     17         self.returncode = None
     18         self.finalizer = None
---> 19         self._launch(process_obj)
     20 
     21     def duplicate_for_child(self, fd):

/opt/anaconda3/envs/env_6_2021/lib/python3.9/multiprocessing/popen_spawn_posix.py in _launch(self, process_obj)
     45         try:
     46             reduction.dump(prep_data, fp)
---> 47             reduction.dump(process_obj, fp)
     48         finally:
     49             set_spawning_popen(None)

/opt/anaconda3/envs/env_6_2021/lib/python3.9/multiprocessing/reduction.py in dump(obj, file, protocol)
     58 def dump(obj, file, protocol=None):
     59     '''Replacement for pickle.dump() using ForkingPickler.'''
---> 60     ForkingPickler(file, protocol).dump(obj)
     61 
     62 #

AttributeError: Can't pickle local object 'local_search.<locals>.parallel_optimization_function'

Log file:

{'application_name': '1d_branin', 'optimization_objectives': ['value'], 'design_of_experiment': {'number_of_samples': 3, 'doe_type': 'random sampling'}, 'optimization_iterations': 10, 'models': {'model': 'gaussian_process', 'number_of_trees': 10, 'max_features': 0.5, 'bootstrap': False, 'min_samples_split': 5}, 'input_parameters': {'x1': {'parameter_type': 'real', 'values': [-5, 10], 'prior': 'uniform'}}, 'log_file': 'hypermapper_logfile.log', 'profiling': False, 'noise': True, 'profiling_file': 'profiles/profile.csv', 'append_profiles': False, 'number_of_cpus': 0, 'max_number_of_predictions': 1000000, 'time_budget': -1, 'number_of_repetitions': 1, 'hypermapper_mode': {'mode': 'default'}, 'output_image': {'output_image_pdf_file': 'output_pareto.pdf', 'image_xlog': False, 'image_ylog': False}, 'feasible_output': {'name': 'Valid', 'true_value': 'True', 'false_value': 'False', 'enable_feasible_predictor': False, 'enable_feasible_predictor_grid_search_on_recall_and_precision': False, 'feasible_predictor_grid_search_validation_file': '/home/lnardi/spatial-lang/results/apps_classification_test_set/BlackScholes.csv'}, 'timestamp': 'Timestamp', 'evaluations_per_optimization_iteration': 1, 'run_directory': '.', 'output_data_file': 'output_samples.csv', 'output_pareto_file': 'output_pareto.csv', 'acquisition_function': 'EI', 'scalarization_method': 'tchebyshev', 'weight_sampling': 'flat', 'bounding_box_limits': [0, 1], 'optimization_method': 'bayesian_optimization', 'local_search_starting_points': 10, 'local_search_random_points': 10000, 'local_search_evaluation_limit': -1, 'scalarization_key': 'scalarization', 'local_search_scalarization_weights': [1], 'print_parameter_importance': False, 'normalize_inputs': False, 'epsilon_greedy_threshold': 0.1, 'model_posterior_weight': 10, 'model_good_quantile': 0.05, 'prior_estimation_file': 'samples.csv', 'prior_estimation_quantile': 0.1, 'estimate_multivariate_priors': False, 'resume_optimization': False, 'resume_optimization_data': 'output_samples.csv', 'bandwidth_parameter': 0, 'bandwidth_n_factor': 100, 'prior_limit_estimation_points': 10000, 'posterior_computation_lower_limit': 1e-08, 'custom_gaussian_prior_means': [0], 'custom_gaussian_prior_stds': [-1], 'acquisition_function_optimizer': 'local_search', 'evolution_population_size': 50, 'evolution_generations': 150, 'mutation_rate': 1, 'evolution_crossover': False, 'regularize_evolution': False, 'batch_size': 2, 'print_best': 'auto', 'print_posterior_best': False}
Design of experiment phase, number of new doe samples = 3 .......
x1,value,Timestamp
3.9349659618323702,3.553449980702834,8
3.3219640321734634,0.5722828316766897,10
0.007318332712835662,33.390893769290294,12

End of doe/resume phase, the number of evaluated configurations is: 3

End of DoE - Time     0.0364 sec
Starting optimization iteration 1
End of training - Time       0.26 sec
Model fitting time     0.2562 sec
Total RS time     0.4186 sec

[rgokulsm]

I'm using MacOS. I am running on Anaconda / Jupyter. And I'm using python 3.9. I do have the log file. Let me try out some clean runs for both these issues and share the log file asap. Thank you!

[rgokulsm]

Added log files to the earlier comments. Thank you!

[arturluis]

Thanks @rgokulsm ,

I'll have a look and get back to you on this!

[arturluis]

Hi @rgokulsm ,

We've just made a new release of HM that fixes some bugs in HM. I added fixes to the two bugs you mentioned but had trouble replicating the pickle issue.

Can you update HyperMapper to the latest release and try again?

[rgokulsm]

Hi! Thank you!

I just gave it a shot. The old errors are gone but I am hitting a new pickle issue:

AttributeError: Can't pickle local object 'Space.__init__.<locals>.<lambda>'

I get this for 3 different examples (one for single real parameter, one for multi real params and one for single ordinal param)... Two of these examples are the same as the ones I'd pasted earlier.

Could your code fix have changed something about "parallel_optimization_function" which makes it work now and something similar can now be used for the new error?

I am using an environment with other software installed (since I need to run these tools together). If it might help the debug I can try to run in a new environment..

Thank you !!

[arturluis]

Hi @rgokulsm ,

The same fix will not easily work for this case I'm afraid. I am having difficulty replicating the issue, can you share some more details of your setup?

• Are you always running on a jupyter notebook?
• Can you share the output of conda list or pip freeze so I can see which package versions you have?

Also, if you are able to try on a different environment as you mentioned to see if the issue persists it would be helpful for us.

Cheers,

[rgokulsm]

Thank you @arturluis ! I've shared my packages below in case there is any obvious thing you can catch.

Yes I am always running out o a jupyter notebook atm. I will try this out soon in a different environment and get back to you.

conda list
# packages in environment at /opt/anaconda3/envs/env_6_2021:
#
# Name                    Version                   Build  Channel
anyio                     2.2.0            py39hecd8cb5_1  
appnope                   0.1.2           py39hecd8cb5_1001  
argon2-cffi               20.1.0           py39h9ed2024_1  
async_generator           1.10               pyhd3eb1b0_0  
attrs                     21.2.0             pyhd3eb1b0_0  
babel                     2.9.1              pyhd3eb1b0_0  
backcall                  0.2.0              pyhd3eb1b0_0  
bayesian-optimization     1.2.0                    pypi_0    pypi
bleach                    3.3.0              pyhd3eb1b0_0  
boto                      2.49.0                   pypi_0    pypi
brotlipy                  0.7.0           py39h9ed2024_1003  
ca-certificates           2020.10.14                    0    anaconda
cctools                   927.0.2              h5ba7a2e_4  
certifi                   2021.5.30        py39hecd8cb5_0  
cffi                      1.14.5           py39h2125817_0  
chardet                   4.0.0           py39hecd8cb5_1003  
clang                     10.0.0          default_hf57f61e_0  
clang_osx-64              10.0.0               h05bbb7f_0  
clangxx                   10.0.0          default_hf57f61e_0  
clangxx_osx-64            10.0.0               h05bbb7f_1  
click                     8.0.1                    pypi_0    pypi
cloudpickle               1.6.0                    pypi_0    pypi
compiler-rt               10.0.0               h47ead80_0  
compiler-rt_osx-64        10.0.0               hbcc88fd_0  
configspace               0.4.19                   pypi_0    pypi
cryptography              3.4.7            py39h2fd3fbb_0  
curl                      7.71.1               hb0a8c7a_1    anaconda
cycler                    0.10.0                   pypi_0    pypi
cython                    0.29.24                  pypi_0    pypi
dask                      2021.7.2                 pypi_0    pypi
decorator                 4.4.2                    pypi_0    pypi
defusedxml                0.7.1              pyhd3eb1b0_0  
dill                      0.3.3                    pypi_0    pypi
distributed               2021.7.2                 pypi_0    pypi
dlx                       1.0.4                    pypi_0    pypi
docplex                   2.21.207                 pypi_0    pypi
entrypoints               0.3                      pypi_0    pypi
fastdtw                   0.3.4                    pypi_0    pypi
fastjsonschema            2.15.1                   pypi_0    pypi
fsspec                    2021.7.0                 pypi_0    pypi
gpy                       1.10.0                   pypi_0    pypi
h5py                      3.1.0                    pypi_0    pypi
heapdict                  1.0.1                    pypi_0    pypi
hypermapper               2.2.4                    pypi_0    pypi
idna                      2.10               pyhd3eb1b0_0  
importlib-metadata        3.10.0           py39hecd8cb5_0  
importlib_metadata        3.10.0               hd3eb1b0_0  
inflection                0.5.1                    pypi_0    pypi
ipykernel                 5.5.5                    pypi_0    pypi
ipython                   7.24.1                   pypi_0    pypi
ipython_genutils          0.2.0              pyhd3eb1b0_1  
ipywidgets                7.6.3                    pypi_0    pypi
jedi                      0.18.0                   pypi_0    pypi
jinja2                    3.0.1                    pypi_0    pypi
joblib                    1.0.1                    pypi_0    pypi
json5                     0.9.5                      py_0  
jsonschema                3.2.0                      py_2  
jupyter-packaging         0.7.12             pyhd3eb1b0_0  
jupyter_client            6.1.12             pyhd3eb1b0_0  
jupyter_core              4.7.1            py39hecd8cb5_0  
jupyter_server            1.4.1            py39hecd8cb5_0  
jupyterlab                3.0.14             pyhd3eb1b0_1  
jupyterlab-widgets        1.0.0                    pypi_0    pypi
jupyterlab_pygments       0.1.2                      py_0  
jupyterlab_server         2.4.0              pyhd3eb1b0_0  
kiwisolver                1.3.1                    pypi_0    pypi
krb5                      1.18.2               h75d18d8_0    anaconda
ld64                      450.3                h3c32e8a_4  
libcurl                   7.71.1               h8a08a2b_1    anaconda
libcxx                    10.0.0                        1  
libedit                   3.1.20191231         h1de35cc_1    anaconda
libffi                    3.3                  hb1e8313_2  
libllvm10                 10.0.1               h76017ad_5  
libsodium                 1.0.18               h1de35cc_0  
libssh2                   1.9.0                ha12b0ac_1    anaconda
llvm-openmp               10.0.0               h28b9765_0  
locket                    0.2.1                    pypi_0    pypi
lxml                      4.6.3                    pypi_0    pypi
markupsafe                2.0.1            py39h9ed2024_0  
matplotlib                3.4.2                    pypi_0    pypi
matplotlib-inline         0.1.2                    pypi_0    pypi
mistune                   0.8.4           py39h9ed2024_1000  
more-itertools            8.8.0                    pypi_0    pypi
mpmath                    1.2.1                    pypi_0    pypi
msgpack                   1.0.2                    pypi_0    pypi
multitasking              0.0.9                    pypi_0    pypi
nbclassic                 0.2.6              pyhd3eb1b0_0  
nbclient                  0.5.3              pyhd3eb1b0_0  
nbconvert                 6.0.7            py39hecd8cb5_0  
nbformat                  5.1.3              pyhd3eb1b0_0  
ncurses                   6.2                  h0a44026_1  
nest-asyncio              1.5.1              pyhd3eb1b0_0  
networkx                  2.5.1                    pypi_0    pypi
notebook                  6.4.0            py39hecd8cb5_0  
ntlm-auth                 1.5.0                    pypi_0    pypi
numpy                     1.20.1                   pypi_0    pypi
openssl                   1.1.1k               h9ed2024_0  
packaging                 20.9               pyhd3eb1b0_0  
pandas                    1.2.3                    pypi_0    pypi
pandoc                    2.12                 hecd8cb5_0  
pandocfilters             1.4.3            py39hecd8cb5_1  
paramz                    0.9.5                    pypi_0    pypi
parso                     0.8.2                    pypi_0    pypi
partd                     1.2.0                    pypi_0    pypi
patsy                     0.5.1                    pypi_0    pypi
pcre                      8.45                 h23ab428_0  
pexpect                   4.8.0              pyhd3eb1b0_3  
pickleshare               0.7.5           pyhd3eb1b0_1003  
pillow                    8.2.0                    pypi_0    pypi
pip                       21.1.2           py39hecd8cb5_0  
ply                       3.11                     pypi_0    pypi
prometheus_client         0.11.0             pyhd3eb1b0_0  
prompt-toolkit            3.0.18                   pypi_0    pypi
psutil                    5.8.0                    pypi_0    pypi
ptyprocess                0.7.0              pyhd3eb1b0_2  
py-bobyqa                 1.3                      pypi_0    pypi
pybind11                  2.6.2                    pypi_0    pypi
pycparser                 2.20                       py_2  
pydoe                     0.3.8                    pypi_0    pypi
pydot                     1.4.2                    pypi_0    pypi
pygments                  2.9.0              pyhd3eb1b0_0  
pylatexenc                2.10                     pypi_0    pypi
pynisher                  0.6.4                    pypi_0    pypi
pyopenssl                 20.0.1             pyhd3eb1b0_1  
pyparsing                 2.4.7              pyhd3eb1b0_0  
pyrfr                     0.8.2                    pypi_0    pypi
pyrsistent                0.17.3           py39h9ed2024_0  
pyscf                     1.7.6                    pypi_0    pypi
pysocks                   1.7.1            py39hecd8cb5_0  
python                    3.9.5                h88f2d9e_3  
python-constraint         1.4.0                    pypi_0    pypi
python-dateutil           2.8.1              pyhd3eb1b0_0  
pytz                      2021.1             pyhd3eb1b0_0  
pyyaml                    5.4.1                    pypi_0    pypi
pyzmq                     22.1.0                   pypi_0    pypi
qiskit                    0.29.0                   pypi_0    pypi
qiskit-aer                0.8.2                    pypi_0    pypi
qiskit-aqua               0.9.4                    pypi_0    pypi
qiskit-ibmq-provider      0.16.0                   pypi_0    pypi
qiskit-ignis              0.6.0                    pypi_0    pypi
qiskit-nature             0.1.4                    pypi_0    pypi
qiskit-terra              0.18.1                   pypi_0    pypi
quandl                    3.6.0                    pypi_0    pypi
readline                  8.1                  h9ed2024_0  
requests                  2.25.1             pyhd3eb1b0_0  
requests-ntlm             1.1.0                    pypi_0    pypi
retworkx                  0.9.0                    pypi_0    pypi
scikit-learn              0.24.1                   pypi_0    pypi
scikit-quant              0.8.2                    pypi_0    pypi
scipy                     1.7.1                    pypi_0    pypi
seaborn                   0.11.1                   pypi_0    pypi
send2trash                1.5.0              pyhd3eb1b0_1  
setuptools                52.0.0           py39hecd8cb5_0  
six                       1.16.0                   pypi_0    pypi
smac                      1.0.1                    pypi_0    pypi
sniffio                   1.2.0            py39hecd8cb5_1  
sortedcontainers          2.4.0                    pypi_0    pypi
sqcommon                  0.3.2                    pypi_0    pypi
sqimfil                   0.3.7                    pypi_0    pypi
sqlite                    3.35.4               hce871da_0  
sqsnobfit                 0.4.5                    pypi_0    pypi
statsmodels               0.12.2                   pypi_0    pypi
swig                      4.0.2                h23ab428_3  
symengine                 0.7.2                    pypi_0    pypi
sympy                     1.7.1                    pypi_0    pypi
tapi                      1000.10.8            ha1b3eb9_0  
tblib                     1.7.0                    pypi_0    pypi
terminado                 0.10.1                   pypi_0    pypi
testpath                  0.5.0                    pypi_0    pypi
threadpoolctl             2.1.0                    pypi_0    pypi
tk                        8.6.10               hb0a8c7a_0  
toolz                     0.11.1                   pypi_0    pypi
tornado                   6.1              py39h9ed2024_0  
traitlets                 5.0.5              pyhd3eb1b0_0  
tweedledum                1.1.0                    pypi_0    pypi
tzdata                    2020f                h52ac0ba_0  
urllib3                   1.26.5                   pypi_0    pypi
wcwidth                   0.2.5                      py_0  
webencodings              0.5.1                    pypi_0    pypi
websocket-client          1.1.0                    pypi_0    pypi
websockets                9.1                      pypi_0    pypi
wheel                     0.36.2             pyhd3eb1b0_0  
widgetsnbextension        3.5.1                    pypi_0    pypi
xz                        5.2.5                h1de35cc_0  
yfinance                  0.1.55                   pypi_0    pypi
z3                        0.2.0                    pypi_0    pypi
z3-solver                 4.8.12.0                 pypi_0    pypi
zeromq                    4.3.4                h23ab428_0  
zict                      2.0.0                    pypi_0    pypi
zipp                      3.4.1              pyhd3eb1b0_0  
zlib                      1.2.11               h1de35cc_3  

pip freeze
anyio @ file:///opt/concourse/worker/volumes/live/96440bbe-d2f1-4a9e-5edf-600248ff38bd/volume/anyio_1617783321037/work/dist
appnope @ file:///opt/concourse/worker/volumes/live/6ca6f098-d773-4461-5c91-a24a17435bda/volume/appnope_1606859448531/work
argon2-cffi @ file:///opt/concourse/worker/volumes/live/38e8fb2b-1295-4bdf-4adf-b20acbe4d91b/volume/argon2-cffi_1607022498041/work
async-generator @ file:///home/ktietz/src/ci/async_generator_1611927993394/work
attrs @ file:///tmp/build/80754af9/attrs_1620827162558/work
Babel @ file:///tmp/build/80754af9/babel_1620871417480/work
backcall @ file:///home/ktietz/src/ci/backcall_1611930011877/work
bayesian-optimization==1.2.0
bleach @ file:///tmp/build/80754af9/bleach_1612211392645/work
boto==2.49.0
brotlipy==0.7.0
certifi==2021.5.30
cffi @ file:///opt/concourse/worker/volumes/live/a70943eb-ad97-40c6-78c9-98764c1bee07/volume/cffi_1613246936716/work
chardet @ file:///opt/concourse/worker/volumes/live/7e1102c4-8702-40f2-63d6-f260ce5f85e4/volume/chardet_1607706831384/work
click==8.0.1
cloudpickle==1.6.0
ConfigSpace==0.4.19
cryptography @ file:///opt/concourse/worker/volumes/live/d5dda287-c0b3-4861-7262-fab05baa64dc/volume/cryptography_1616769284011/work
cycler==0.10.0
Cython==0.29.24
dask==2021.7.2
decorator==4.4.2
defusedxml @ file:///tmp/build/80754af9/defusedxml_1615228127516/work
dill==0.3.3
distributed==2021.7.2
dlx==1.0.4
docplex==2.21.207
entrypoints==0.3
fastdtw==0.3.4
fastjsonschema==2.15.1
fsspec==2021.7.0
GPy==1.10.0
h5py==3.1.0
HeapDict==1.0.1
hypermapper==2.2.4
idna @ file:///home/linux1/recipes/ci/idna_1610986105248/work
importlib-metadata @ file:///opt/concourse/worker/volumes/live/bdafb81c-7edc-4122-77fe-a69839ec3c71/volume/importlib-metadata_1617877361684/work
inflection==0.5.1
ipykernel==5.5.5
ipython==7.24.1
ipython-genutils @ file:///tmp/build/80754af9/ipython_genutils_1606773439826/work
ipywidgets==7.6.3
jedi==0.18.0
Jinja2==3.0.1
joblib==1.0.1
json5==0.9.5
jsonschema @ file:///tmp/build/80754af9/jsonschema_1602607155483/work
jupyter-client @ file:///tmp/build/80754af9/jupyter_client_1616770841739/work
jupyter-core @ file:///opt/concourse/worker/volumes/live/d225f67f-0726-47b9-4510-c6aff5625ca4/volume/jupyter_core_1612213300495/work
jupyter-packaging @ file:///tmp/build/80754af9/jupyter-packaging_1613502826984/work
jupyter-server @ file:///opt/concourse/worker/volumes/live/c0b6c5cd-8b5f-482c-6789-42a64b3d2acc/volume/jupyter_server_1616084049292/work
jupyterlab @ file:///tmp/build/80754af9/jupyterlab_1619133235951/work
jupyterlab-pygments @ file:///tmp/build/80754af9/jupyterlab_pygments_1601490720602/work
jupyterlab-server @ file:///tmp/build/80754af9/jupyterlab_server_1617134334258/work
jupyterlab-widgets==1.0.0
kiwisolver==1.3.1
locket==0.2.1
lxml==4.6.3
MarkupSafe @ file:///opt/concourse/worker/volumes/live/1bcf0940-5bf8-4171-61e3-2133f5885e8b/volume/markupsafe_1621528148241/work
matplotlib==3.4.2
matplotlib-inline==0.1.2
mistune @ file:///opt/concourse/worker/volumes/live/4217afd5-dad1-438d-6f79-e4992ccda0e5/volume/mistune_1607364880245/work
more-itertools==8.8.0
mpmath==1.2.1
msgpack==1.0.2
multitasking==0.0.9
nbclassic @ file:///tmp/build/80754af9/nbclassic_1616085367084/work
nbclient @ file:///tmp/build/80754af9/nbclient_1614364831625/work
nbconvert @ file:///opt/concourse/worker/volumes/live/41c54e2d-699e-4b21-6d46-bf9dde99a677/volume/nbconvert_1607370470905/work
nbformat @ file:///tmp/build/80754af9/nbformat_1617383369282/work
nest-asyncio @ file:///tmp/build/80754af9/nest-asyncio_1613680548246/work
networkx==2.5.1
notebook @ file:///opt/concourse/worker/volumes/live/8b811b26-0985-4d66-61d2-eae9abb84ef2/volume/notebook_1621625613100/work
ntlm-auth==1.5.0
numpy==1.20.1
packaging @ file:///tmp/build/80754af9/packaging_1611952188834/work
pandas==1.2.3
pandocfilters @ file:///opt/concourse/worker/volumes/live/d8ef4635-066d-4ffe-5341-12ebf01bd094/volume/pandocfilters_1605120459573/work
paramz==0.9.5
parso==0.8.2
partd==1.2.0
patsy==0.5.1
pexpect @ file:///tmp/build/80754af9/pexpect_1605563209008/work
pickleshare @ file:///tmp/build/80754af9/pickleshare_1606932040724/work
Pillow==8.2.0
ply==3.11
prometheus-client @ file:///tmp/build/80754af9/prometheus_client_1623189609245/work
prompt-toolkit==3.0.18
psutil==5.8.0
ptyprocess @ file:///tmp/build/80754af9/ptyprocess_1609355006118/work/dist/ptyprocess-0.7.0-py2.py3-none-any.whl
Py-BOBYQA==1.3
pybind11==2.6.2
pycparser @ file:///tmp/build/80754af9/pycparser_1594388511720/work
pyDOE==0.3.8
pydot==1.4.2
Pygments @ file:///tmp/build/80754af9/pygments_1621606182707/work
pylatexenc==2.10
pynisher==0.6.4
pyOpenSSL @ file:///tmp/build/80754af9/pyopenssl_1608057966937/work
pyparsing @ file:///home/linux1/recipes/ci/pyparsing_1610983426697/work
pyrfr==0.8.2
pyrsistent @ file:///opt/concourse/worker/volumes/live/f518b3a6-f049-4498-73a8-3d98fed23e04/volume/pyrsistent_1607365207647/work
pyscf==1.7.6
PySocks @ file:///opt/concourse/worker/volumes/live/112288ac-9cb0-4e73-768b-13baf4ca6419/volume/pysocks_1605305820043/work
python-constraint==1.4.0
python-dateutil @ file:///home/ktietz/src/ci/python-dateutil_1611928101742/work
pytz @ file:///tmp/build/80754af9/pytz_1612215392582/work
PyYAML==5.4.1
pyzmq==22.1.0
qiskit==0.29.0
qiskit-aer==0.8.2
qiskit-aqua==0.9.4
qiskit-ibmq-provider==0.16.0
qiskit-ignis==0.6.0
qiskit-nature==0.1.4
qiskit-terra==0.18.1
Quandl==3.6.0
requests @ file:///tmp/build/80754af9/requests_1608241421344/work
requests-ntlm==1.1.0
retworkx==0.9.0
scikit-learn==0.24.1
scikit-quant==0.8.2
scipy==1.7.1
seaborn==0.11.1
Send2Trash @ file:///tmp/build/80754af9/send2trash_1607525499227/work
six==1.16.0
smac==1.0.1
sniffio @ file:///opt/concourse/worker/volumes/live/38ca9e9e-09d1-4d43-5a0f-b546422e7807/volume/sniffio_1614030472707/work
sortedcontainers==2.4.0
SQCommon==0.3.2
SQImFil==0.3.7
SQSnobFit==0.4.5
statsmodels==0.12.2
symengine==0.7.2
sympy==1.7.1
tblib==1.7.0
terminado==0.10.1
testpath==0.5.0
threadpoolctl==2.1.0
toolz==0.11.1
tornado @ file:///opt/concourse/worker/volumes/live/2c1a63a2-006b-48ee-56b9-0cfe8b4927f9/volume/tornado_1606942321278/work
traitlets @ file:///home/ktietz/src/ci/traitlets_1611929699868/work
tweedledum==1.1.0
urllib3==1.26.5
wcwidth @ file:///tmp/build/80754af9/wcwidth_1593447189090/work
webencodings==0.5.1
websocket-client==1.1.0
websockets==9.1
widgetsnbextension==3.5.1
yfinance==0.1.55
z3==0.2.0
z3-solver==4.8.12.0
zict==2.0.0
zipp @ file:///tmp/build/80754af9/zipp_1615904174917/work

[arturluis]

Hi @rgokulsm ,

We have pushed a new version of hypermapper (2.2.5) with a potential fix for this issue, can you install it (e.g. pip3 install hypermapper==2.2.5) and see if the error still happens?

Thanks,

[peterhchen]

I used "conda install hypermmpper==2.2.5" and still got the same error: AttributeError: Can't pickle local object "Space.inti..'

[rgokulsm]

Apologies for my delayed response. I agree with @peterhchen . Still get the same error unfortunately. I must say I still have not tried it in a clean new environment (as was suggested earlier) but considering that others are seeing the error as well, maybe its not an environment issue? Thank you!

AttributeError: Can't pickle local object 'Space.__init__.<locals>.<lambda>'

[peterhchen]

Is this issues from Window 10? If I change to Linux, do I have this issue? This problem come from optimizer.optimze ("example_d1_brain_scenario.json", breain_funciton_1d)

When hypermapper start optimization iteration 4. The AttributeError ... I cannot copy the error message since this is the company computer cannot copy out (Computer can only copy from web into computer) the message.

What is the impact of this error message? Can I ignore this error message?

[peterhchen]

I run the "Chakong-Hames" example in the demo script. I have error: optimizer.optimize ("example_chkong_haimes_scenario.json", chakong_haimes) NameError: name 'chakong_haimes' is not defined.

[peterhchen]

It seems the pickle error does not affect the result. The results write into the _output_samples.csv file.

[rgokulsm]

I think it fails after the initialization phase. Only the random samples are created and thats what you see in the csv..

Here's what I get with RS=10

Design of experiment phase, number of new doe samples = 10 .......
x1,value,Timestamp
3.0,0.506752310227002,5
5.5,18.060674124988978,6
2.0,7.124828102039894,7
2.5,2.6137237843308414,7
-3.5,119.33746909170875,7
0.0,33.47773764227026,7
4.5,8.649933213308866,8
5.0,13.730034933787278,8
4.0,4.053398307697645,8
-3.0,93.8541933295621,8

End of doe/resume phase, the number of evaluated configurations is: 10

Starting optimization iteration 1
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)

[arturluis]

Hey @rgokulsm and @peterhchen ,

I am having trouble replicating this issue, it would be great if we could come up with a minimum working example that reproduces this issue so that I can fix it better. I'm using a clean Ubuntu 20.04 machine and installing HyperMapper in a clean conda environment:

conda create hypermapper
conda activate hypermapper
conda install pip
pip install hypermapper

and then I run this example @rgokulsm provided:

import math
def branin_function_1d(X):
    # The function must receive a dictionary
    x1 = X['x1']

    # Branin function computation
    a = 1.0
    b = 5.1 / (4.0 * math.pi * math.pi)
    c = 5.0 / math.pi
    r = 6.0
    s = 10.0
    t = 1.0 / (8.0 * math.pi)
    x2 = 2.275
    value = a * (x2 - b * x1 * x1 + c * x1 - r) ** 2 + s * (1 - t) * math.cos(x1) + s

    # The function must return the objective value (a number)
    return value

import matplotlib
import matplotlib.pyplot as plt
import pandas as pd
import numpy as np
%matplotlib inline
plt.rcParams['figure.figsize'] = [12, 6]
plt.rcParams['font.size'] = 18
point_size = matplotlib.rcParams['lines.markersize']**2.8
point_size_optimum = matplotlib.rcParams['lines.markersize']**2

optimum = math.pi
value_at_optimum=branin_function_1d({'x1': optimum})

# Sample 1000 (x,y) pairs from the function to plot its curve
branin_line_xs = np.linspace(-5, 10, 1000)
branin_line_ys = []
for x in branin_line_xs:
    y = branin_function_1d({'x1': x})
    branin_line_ys.append(y)
plt.plot(branin_line_xs, branin_line_ys, label="1D Branin Function")

# Mark the known optimum on the curve
plt.scatter(optimum, value_at_optimum, s=point_size_optimum, marker='o', color="black", label="Minimum")

plt.legend()
plt.xlabel("x1")
plt.ylabel("value")
plt.show()
print("The 1d Branin function has one global optimum at x1 = \u03C0", flush=True)
print("(x, y) at minimum is: ("+str(optimum)+","+str(value_at_optimum)+")", flush=True)

import json
scenario = {}
scenario["application_name"] = "1d_branin"
scenario["optimization_objectives"] = ["value"]

number_of_RS = 3
scenario["design_of_experiment"] = {}
scenario["design_of_experiment"]["number_of_samples"] = number_of_RS

scenario["optimization_iterations"] = 10

scenario["models"] = {}
scenario["models"]["model"] = "gaussian_process"

scenario["input_parameters"] = {}
x1 = {}
x1["parameter_type"] = "real"
x1["values"] = [-5, 10]

scenario["input_parameters"]["x1"] = x1

with open("example_1d_branin_scenario.json", "w") as scenario_file:
    json.dump(scenario, scenario_file, indent=4)

f = open("example_1d_branin_scenario.json", "r")
text = f.read()
print(text, flush=True)
f.close()

import sys
import os
from hypermapper import optimizer
stdout = sys.stdout # Jupyter uses a special stdout and HyperMapper logging overwrites it. Save stdout to restore later

# Call HyperMapper to optimize the 1d Branin function
optimizer.optimize("example_1d_branin_scenario.json", branin_function_1d)
sys.stdout = stdout

but I can't seem to replicate the issue. Do you note anything different between your setup and mine? If not, I'll try to replicate the issue on a Windows or MacOS machine and see if it some OS-specific issue.

[peterhchen]

Thanks Our target machine is Ubuntu. Right now, I tried hypermmaper on windows 10 with “conda install hypermmpaer” All demo cases are failed.

On Tue, Sep 21, 2021 at 4:20 PM arturluis @.***> wrote:

Hey @rgokulsm https://github.com/rgokulsm and @peterhchen https://github.com/peterhchen ,

I am having trouble replicating this issue, it would be great if we could come up with a minimum working example that reproduces this issue so that I can fix it better. I'm using a clean Ubuntu 20.04 machine and installing HyperMapper in a clean conda environment:

conda create hypermapper conda activate hypermapper conda install pip pip install hypermapper

and then I run this example @rgokulsm https://github.com/rgokulsm provided:

import math def branin_function_1d(X):

The function must receive a dictionary

x1 = X['x1']

# Branin function computation
a = 1.0
b = 5.1 / (4.0 * math.pi * math.pi)
c = 5.0 / math.pi
r = 6.0
s = 10.0
t = 1.0 / (8.0 * math.pi)
x2 = 2.275
value = a * (x2 - b * x1 * x1 + c * x1 - r) ** 2 + s * (1 - t) * math.cos(x1) + s

# The function must return the objective value (a number)
return value

import matplotlib import matplotlib.pyplot as plt import pandas as pd import numpy as np %matplotlib inline plt.rcParams['figure.figsize'] = [12, 6] plt.rcParams['font.size'] = 18 point_size = matplotlib.rcParams['lines.markersize']2.8 point_size_optimum = matplotlib.rcParams['lines.markersize']2

optimum = math.pi value_at_optimum=branin_function_1d({'x1': optimum})

Sample 1000 (x,y) pairs from the function to plot its curve

branin_line_xs = np.linspace(-5, 10, 1000) branin_line_ys = [] for x in branin_line_xs: y = branin_function_1d({'x1': x}) branin_line_ys.append(y) plt.plot(branin_line_xs, branin_line_ys, label="1D Branin Function")

Mark the known optimum on the curve

plt.scatter(optimum, value_at_optimum, s=point_size_optimum, marker='o', color="black", label="Minimum")

plt.legend() plt.xlabel("x1") plt.ylabel("value") plt.show() print("The 1d Branin function has one global optimum at x1 = \u03C0", flush=True) print("(x, y) at minimum is: ("+str(optimum)+","+str(value_at_optimum)+")", flush=True)

import json scenario = {} scenario["application_name"] = "1d_branin" scenario["optimization_objectives"] = ["value"]

number_of_RS = 3 scenario["design_of_experiment"] = {} scenario["design_of_experiment"]["number_of_samples"] = number_of_RS

scenario["optimization_iterations"] = 10

scenario["models"] = {} scenario["models"]["model"] = "gaussian_process"

scenario["input_parameters"] = {} x1 = {} x1["parameter_type"] = "real" x1["values"] = [-5, 10]

scenario["input_parameters"]["x1"] = x1

with open("example_1d_branin_scenario.json", "w") as scenario_file: json.dump(scenario, scenario_file, indent=4)

f = open("example_1d_branin_scenario.json", "r") text = f.read() print(text, flush=True) f.close()

import sys import os from hypermapper import optimizer stdout = sys.stdout # Jupyter uses a special stdout and HyperMapper logging overwrites it. Save stdout to restore later

Call HyperMapper to optimize the 1d Branin function

optimizer.optimize("example_1d_branin_scenario.json", branin_function_1d) sys.stdout = stdout

but I can't seem to replicate the issue. Do you note anything different between your setup and mine? If not, I'll try to replicate the issue on a Windows or MacOS machine and see if it some OS-specific issue.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-924458201, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBDZ4EWMVKAN4CRTSHPTUDEHNBANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[rgokulsm]

@arturluis Thanks! I tried a fresh environment in MacOS and I still hit the error (with the above example). As far as I can see it is a fatal error. Can you also post the correct output you get? I want to confirm that the output I get is only partial and only up to the point of the error (i.e. the error is actually fatal).

[peterhchen]

Thanks for the very important message

Peter

On Tue, Sep 21, 2021 at 5:12 PM Gokul Subramanian Ravi < @.***> wrote:

@arturluis https://github.com/arturluis Thanks! I tried a fresh environment in MacOS and I still hit the error. As far as I can see it is a fatal error. Can you also post the correct output you get? I want to confirm that the output I get is only partial and only up to the point of the error (i.e. the error is actually fatal).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-924478978, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBDYXQVJIKPKKSTFV2UTUDENQDANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[peterhchen]

Thanks

Anaconda/Jupiter notebook under Windows 10 does not work well I will try Ubuntu as they suggested

Peter

On Tue, Sep 21, 2021 at 3:12 PM Gokul Subramanian Ravi < @.***> wrote:

I think it fails after the initialization phase. Only the random samples are created and thats what you see in the csv..

Here's what I get with RS=10

Design of experiment phase, number of new doe samples = 10 ....... x1,value,Timestamp 3.0,0.506752310227002,5 5.5,18.060674124988978,6 2.0,7.124828102039894,7 2.5,2.6137237843308414,7 -3.5,119.33746909170875,7 0.0,33.47773764227026,7 4.5,8.649933213308866,8 5.0,13.730034933787278,8 4.0,4.053398307697645,8 -3.0,93.8541933295621,8

End of doe/resume phase, the number of evaluated configurations is: 10

Starting optimization iteration 1

AttributeError Traceback (most recent call last)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-924426054, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD5F6VQ7AM5X4N3BBWDUDD7NHANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[arturluis]

Hi @peterhchen and @rgokulsm ,

I was finally able to reproduce the pickle issue on a Windows machine here. I then made some fixes that solved the problem from my side. Can you pull or install the latest (2.2.7) version of HyperMapper and see if the issue is fixed for you as well?

Cheers,

[peterhchen]

Thanks

Too good to be true. I move from Ubuntu virtual machine into normal windows-10 for testing.

I will let my company know. Our final delivery is docker containers on red zone (security) Linux.

Peter H Chen

On Wed, Sep 29, 2021 at 2:53 PM arturluis @.***> wrote:

Hi @peterhchen https://github.com/peterhchen and @rgokulsm https://github.com/rgokulsm ,

I was finally able to reproduce the pickle issue on a Windows machine here. I then made some fixes that solved the problem from my side. Can you pull or install the latest (2.2.7) version of HyperMapper and see if the issue is fixed for you as well?

Cheers,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-930572395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD7NXE5A7P3OILM4ZKDUEODF7ANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[peterhchen]

Hi,

In the single objective pdf generation, when no .csv in the output folder. The error message is very strange. Should report something related to "no csv existed..." file.

[image: image.png]

Thanks.

On Wed, Sep 29, 2021 at 2:53 PM arturluis @.***> wrote:

Hi @peterhchen https://github.com/peterhchen and @rgokulsm https://github.com/rgokulsm ,

I was finally able to reproduce the pickle issue on a Windows machine here. I then made some fixes that solved the problem from my side. Can you pull or install the latest (2.2.7) version of HyperMapper and see if the issue is fixed for you as well?

Cheers,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-930572395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD7NXE5A7P3OILM4ZKDUEODF7ANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[peterhchen]

I created an example for 3 objectives (f1_value, f2_value, f3_value) for pareto plot based on example 5: chakong haimes. I can only see f1_value and f2_value.

1. I cannot see f3_value on the plot. Is there a way to plot f1_value, f2_value, and f3_value?
2. Is all the data in the hypermapper_logfile.log?

in chakong_haimes.py ... f3_value = f1_value * f2_value ... output_matrics["f3_value"] = f3_value

In chakong_hamles_scenario.json: "optimization_objectives": ["f1_value", "f2_value", "f3_value"]

python chaong_hamies.py running OK hm-compute-pareto chakong_hamies_scenario.josn hm-plot-pareto chakong_haimes_scenario.json

Show pdf plot

xdg-open chaking_hames_output_pareto.pdf

On Wed, Sep 29, 2021 at 2:53 PM arturluis @.***> wrote:

Hi @peterhchen https://github.com/peterhchen and @rgokulsm https://github.com/rgokulsm ,

I was finally able to reproduce the pickle issue on a Windows machine here. I then made some fixes that solved the problem from my side. Can you pull or install the latest (2.2.7) version of HyperMapper and see if the issue is fixed for you as well?

Cheers,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-930572395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD7NXE5A7P3OILM4ZKDUEODF7ANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[peterhchen]

Thanks for the excellent fix.

I verified the windows/anaconda/Jupyter Notebook code. I do not need the Ubuntu Virtual Machine which cannot attached file due to company security reason.

Peter H. Chen

On Wed, Sep 29, 2021 at 2:53 PM arturluis @.***> wrote:

Hi @peterhchen https://github.com/peterhchen and @rgokulsm https://github.com/rgokulsm ,

I was finally able to reproduce the pickle issue on a Windows machine here. I then made some fixes that solved the problem from my side. Can you pull or install the latest (2.2.7) version of HyperMapper and see if the issue is fixed for you as well?

Cheers,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-930572395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD7NXE5A7P3OILM4ZKDUEODF7ANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[rgokulsm]

@arturluis The original pickle error is fixed. So I will close this issue and open a new one if any other questions / issues arise. Thank you so much for spending time on this fix!

[peterhchen]

Thanks for wonderful fix

On Sat, Oct 2, 2021 at 10:16 AM Gokul Subramanian Ravi < @.***> wrote:

@arturluis https://github.com/arturluis The original pickle error is fixed. So I will close this issue and open a new one if any other questions / issues arise. Thank you so much for spending time on this fix!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-932786815, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD4ZMXY7GAAUIOGFEX3UE45AVANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[peterhchen]

Hi Luinardi/Hypermapper and teams: Below attached case I follow your "resume optimization" created for branin dataset. It is running fine in Ubuntu but crash under windows-10. Please help. Thanks

Windows-10/Anaconda3: Error message:

[image: image.png]

Ubuntu running fine:

[image: image.png] Peter H. Chen

On Wed, Sep 29, 2021 at 2:53 PM arturluis @.***> wrote:

Hi @peterhchen https://github.com/peterhchen and @rgokulsm https://github.com/rgokulsm ,

I was finally able to reproduce the pickle issue on a Windows machine here. I then made some fixes that solved the problem from my side. Can you pull or install the latest (2.2.7) version of HyperMapper and see if the issue is fixed for you as well?

Cheers,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-930572395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD7NXE5A7P3OILM4ZKDUEODF7ANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

[peterhchen]

Attached is detailed document.

On Tue, Oct 5, 2021 at 2:08 PM Peter Chen @.***> wrote:

Hi Luinardi/Hypermapper and teams: Below attached case I follow your "resume optimization" created for branin dataset. It is running fine in Ubuntu but crash under windows-10. Please help. Thanks

Windows-10/Anaconda3: Error message:

[image: image.png]

Ubuntu running fine:

[image: image.png] Peter H. Chen

On Wed, Sep 29, 2021 at 2:53 PM arturluis @.***> wrote:

Hi @peterhchen https://github.com/peterhchen and @rgokulsm https://github.com/rgokulsm ,

I was finally able to reproduce the pickle issue on a Windows machine here. I then made some fixes that solved the problem from my side. Can you pull or install the latest (2.2.7) version of HyperMapper and see if the issue is fixed for you as well?

Cheers,

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/luinardi/hypermapper/issues/48#issuecomment-930572395, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADBSBD7NXE5A7P3OILM4ZKDUEODF7ANCNFSM5CZ7J2PQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

-- Sincerely Yours, Peter H. Chen

-- Sincerely Yours, Peter H. Chen