Problems in Mac OS X

[pnovoa]

I have run the example code (sudo python3 my_experiment.py) on an iMac19.1 with Intel Core i5 3GHz CPU macOS Big Sur and I get the following error:

loading region bounding boxes for computing carbon emissions region, this may take a moment... 454/454... rate=829.41 Hz, eta=0:00:00, total=0:00:00, wall=18:11 -044 Done! experiment_impact_tracker.compute_tracker.ImpactTracker - WARNING - Gathering system info for reproducibility... experiment_impact_tracker.compute_tracker.ImpactTracker - WARNING - Done initial setup and information gathering... experiment_impact_tracker.compute_tracker.ImpactTracker - ERROR - Encountered exception when launching power monitor thread. --- Logging error --- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/compute_tracker.py", line 202, in launch_impact_monitor self.p, self.queue = launch_power_monitor(self.logdir, self.initial_info, self.logger) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/utils.py", line 83, in wrapper p.start() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'processify..process_func'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 1079, in emit msg = self.format(record) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 923, in format return fmt.format(record) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 659, in format record.message = record.getMessage() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 363, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/Users/theuser/PycharmProjects/test_green/experiment-impact-tracker/examples/my_experiment.py", line 70, in my_experiment() File "/Users/theuser/PycharmProjects/test_green/experiment-impact-tracker/examples/my_experiment.py", line 55, in my_experiment tracker.launch_impact_monitor() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/compute_tracker.py", line 210, in launch_impact_monitor self.logger.error(ex_type, ex_value, Message: <class 'AttributeError'> Arguments: (AttributeError("Can't pickle local object 'processify..process_func'"), ' File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/compute_tracker.py", line 202, in launch_impact_monitor\n self.p, self.queue = launch_power_monitor(self.logdir, self.initial_info, self.logger)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/utils.py", line 83, in wrapper\n p.start()\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start\n self._popen = self._Popen(self)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen\n return _default_context.get_context().Process._Popen(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen\n return Popen(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in init\n super().init(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in init\n self._launch(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch\n reduction.dump(process_obj, fp)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump\n ForkingPickler(file, protocol).dump(obj)\n') --- Logging error --- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/compute_tracker.py", line 202, in launch_impact_monitor self.p, self.queue = launch_power_monitor(self.logdir, self.initial_info, self.logger) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/utils.py", line 83, in wrapper p.start() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'processify..process_func'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 1079, in emit msg = self.format(record) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 923, in format return fmt.format(record) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 659, in format record.message = record.getMessage() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/logging/init.py", line 363, in getMessage msg = msg % self.args TypeError: not all arguments converted during string formatting Call stack: File "/Users/theuser/PycharmProjects/test_green/experiment-impact-tracker/examples/my_experiment.py", line 70, in my_experiment() File "/Users/theuser/PycharmProjects/test_green/experiment-impact-tracker/examples/my_experiment.py", line 55, in my_experiment tracker.launch_impact_monitor() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/compute_tracker.py", line 210, in launch_impact_monitor self.logger.error(ex_type, ex_value, Message: <class 'AttributeError'> Arguments: (AttributeError("Can't pickle local object 'processify..process_func'"), ' File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/compute_tracker.py", line 202, in launch_impact_monitor\n self.p, self.queue = launch_power_monitor(self.logdir, self.initial_info, self.logger)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/utils.py", line 83, in wrapper\n p.start()\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start\n self._popen = self._Popen(self)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen\n return _default_context.get_context().Process._Popen(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen\n return Popen(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in init\n super().init(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in init\n self._launch(process_obj)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch\n reduction.dump(process_obj, fp)\n File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump\n ForkingPickler(file, protocol).dump(obj)\n') Traceback (most recent call last): File "/Users/theuser/PycharmProjects/test_green/experiment-impact-tracker/examples/my_experiment.py", line 70, in my_experiment() File "/Users/theuser/PycharmProjects/test_green/experiment-impact-tracker/examples/my_experiment.py", line 55, in my_experiment tracker.launch_impact_monitor() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/compute_tracker.py", line 202, in launch_impact_monitor self.p, self.queue = launch_power_monitor(self.logdir, self.initial_info, self.logger) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/experiment_impact_tracker/utils.py", line 83, in wrapper p.start() File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start self._popen = self._Popen(self) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen return _default_context.get_context().Process._Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen return Popen(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in init super().init(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in init self._launch(process_obj) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch reduction.dump(process_obj, fp) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump ForkingPickler(file, protocol).dump(obj) AttributeError: Can't pickle local object 'processify..process_func'

Any idea what may be causing this error? I have googled this bug but found almost nothing about it.

[Breakend]

Hi, I believe this is fixed in the main branch and is due to a difference in multiprocessing in python 3.8. See https://github.com/Breakend/experiment-impact-tracker/issues/40 . Let us know if you still see the issue on the main branch codebase

[pnovoa]

Hi, I believe this is fixed in the main branch and is due to a difference in multiprocessing in python 3.8. See #40 . Let us know if you still see the issue on the main branch codebase

Thank you! It works 100%...however, when I run generate-carbon-impact-statement /tmp/tmphgefknol/ "USA" I get:

Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/bin/generate-carbon-impact-statement", line 120, in sys.exit(main(sys.argv[1:])) File "/Library/Frameworks/Python.framework/Versions/3.9/bin/generate-carbon-impact-statement", line 58, in main ssc = pd.read_csv("https://raw.githubusercontent.com/country-level-scc/cscc-database-2018/master/cscc_db_v2.csv") File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/parsers.py", line 605, in read_csv return _read(filepath_or_buffer, kwds) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/parsers.py", line 457, in _read parser = TextFileReader(filepath_or_buffer, kwds) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/parsers.py", line 814, in init self._engine = self._make_engine(self.engine) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/parsers.py", line 1045, in _make_engine return mapping[engine](self.f, self.options) # type: ignore[call-arg] File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/parsers.py", line 1862, in init self._open_handles(src, kwds) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/parsers.py", line 1357, in _open_handles self.handles = get_handle( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/common.py", line 558, in get_handle ioargs = _get_filepath_or_buffer( File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/common.py", line 288, in _get_filepath_or_buffer req = urlopen(filepath_or_buffer) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/common.py", line 194, in urlopen return urllib.request.urlopen(*args, *kwargs) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open result = self._call_chain(self.handle_open, protocol, protocol + File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain result = func(args) File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1385, in https_open return self.do_open(http.client.HTTPSConnection, req, File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1345, in do_open raise URLError(err) urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1123)

Do you have any idea what causes that error? Thank you in advance...

[pnovoa]

I already solved the SSL: CERTIFICATE_VERIFY_FAILED problem (https://stackoverflow.com/questions/50236117/scraping-ssl-certificate-verify-failed-error-for-http-en-wikipedia-org), but I still have 0 impact in the carbon statement. :(

[Breakend]

Can you provide logs for this issue as well to help us solve it? Do you have PowerGadget installed on your Mac?

What did you do to solve the SSL issue? If there are any changes on our side, we'd like to make sure they're addressed.

Thanks!

[pnovoa]

Hi! I solved the SSL problem using the recommendations in the StackOverFlow link. It wasn't a problem from experiment-impact-tracker. So, I think it was a good idea to have raised an issue to avoid downloading the country data. About that, I have a question. How can I interpret a negative carbon social cost of a certain execution?

[Breakend]

Hi, we've fixed it so it doesn't download anything in the main branch now. A negative social cost of carbon can be explained like we do in this paper: https://arxiv.org/pdf/2007.02786.pdf in appendix A1.

"Note the Canada-specific social cost of carbon is negative in this case as explained by Ricke et al. (2018): “The CSCC captures the amount of marginal damage (or, if negative, the benefit) expected to occur in an individual country as a consequence of additional CO2 emission... Northern Europe, Canada and the Former Soviet Union have negative CSCC values because their current temperatures are below the economic optimum.”

We are still considering whether we may want to include a non-country-specific cost of carbon for our tracker, but this is the explanation we would attach for now.

[pnovoa]

Thank you!

[Breakend]

It seems like this issue is resolved, but please re-open if this is still a problem. Thanks!