Closed forresti closed 4 years ago
Hi,
Sorry if this wasn't more clear! We're working on clearing up some issues that might be confusing as people bring them up.
The data.json contains logs about power/energy readings during training. The pickle file is meant for one time information recorded at the start/end of training (e.g., the cpu you're using, etc.).
Since we're using a polling method, you will only see logs showing up in the data.json with energy readings once every few seconds since we're intending this for long-running jobs (and it takes a few seconds to get accurate CPU/GPU readings of the workload).
It looks like you might have an exceptionally fast system so it didn't get a chance to log anything. We'll make the example a bit longer so that it's guaranteed to output something and will try to make this clearer. Tagging #4 to track this. You can also make the loop longer in the example to make sure that this really fixes the issue and logs the correct information. E.g., change this to 1000 or something https://github.com/Breakend/experiment-impact-tracker/blob/master/examples/my_experiment.py#L59
Hope that helps!
@Breakend Thanks for the very fast reply!
Per your suggestion, I reran the experiment with 1000 iterations instead of 100 iterations. After doing that, what I see is:
data.json
now has 8 data points in itimpact_tracker_log.log
is still empty.Is this normal?
Yup, that's normal. The data.json file contains the energy readings while the .log file is meant for exceptions and that sort of thing so we can debug better.
Great. Thanks again for the help.
I also just wanted to share that I was able to use the provided generate-carbon-impact-statement
script to produce a summary of the results.
Specifically, when I do this:
#!/bin/bash
input_dir=/tmp/tmpg_btwq_9
python scripts/generate-carbon-impact-statement $input_dir "USA"
...it prints:
This work contributed 0.002 kg of $\text{CO}_{2eq}$ to the atmosphere and used 0.008 kWh of electricity, having a USA-specific social cost of carbon of \$0.00 (\$0.00, \$0.00). Carbon accounting information can be found here: \url{<TODO: Insert URL of generated HTML appendix with all info here>}. The social cost of carbon uses models from \citep{ricke2018country} and this statement and carbon emissions information was generated using \emph{experiment-impact-tracker}\citep{henderson2019climate}.
I am just getting started using this repo, so my apologies for the naive question. When I run
python examples/my_experiment.py
, it prints:OK, so what's in
/tmp/tmpw3feackg
?Hmm, isn't it odd that 2 of the 3 files are empty?
But, let's see what's in the
info.pkl
file.Here's a pseudocode summary of what's in the pickle file:
Questions
data.json
andimpact_tracker_log.log
files?data.json
andimpact_tracker_log.log
files that are empty in my case?