2 misc problems in create-compute-appendix

[mvesin]

Hi all, it seems like there is 2 misc problems in current version of create-compute-appendix :

• a pdb.set_trace() invocation not commented out
• a test in parsing GPU infos

Here is a proposed diff :

diff --git a/scripts/create-compute-appendix b/scripts/create-compute-appendix
--- a/scripts/create-compute-appendix
+++ b/scripts/create-compute-appendix
@@ -211,7 +211,7 @@ def _aggregated_data_for_filterset(output_dir,

                 # {k: [v] for k, v in info["gpu_info"].items()})
                 if "gpu_info" in info:
-                    import pdb;pdb.set_trace()
+                    #import pdb;pdb.set_trace()
                     gpu_data_frame = pd.DataFrame.from_dict(info["gpu_info"])
                     gpu_infos_all[experiment_set_names[exp_set]].append(
                         gpu_data_frame)
@@ -283,7 +283,8 @@ def _create_leaf_page(output_directory, all_infos, exp_set_name, description, ex
             data_zip_paths_all[exp_set_name][i], html_output_dir)

         template_args = {}
-        if gpu_infos_all and gpu_infos_all[exp_set_name] and gpu_infos_all[exp_set_name][i]:
+        #if gpu_infos_all and gpu_infos_all[exp_set_name] and gpu_infos_all[exp_set_name][i]:
+        if gpu_infos_all and gpu_infos_all and pd.DataFrame(gpu_infos_all[exp_set_name][i]).empty:
             template_args["gpu_info"] = gpu_infos_all[exp_set_name][i].T

         template_args["exp_set_names_titles"] = [(_format_setname(experiment_set_names[exp_set]), experiment_set_names[exp_set])

[pnovoa]

I have also experienced some issues when executing the following command:

create-compute-appendix --site_spec specifications.json --output_dir site data

loading region bounding boxes for computing carbon emissions region, this may take a moment... 454/454... rate=796.77 Hz, eta=0:00:00, total=0:00:00, wall=16:22 -044 Done! /Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix:30: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width. pd.set_option('display.max_colwidth', -1) Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 431, in sys.exit(main(sys.argv[1:])) File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 427, in main _recursive_create(all_log_dirs, args.output_dir, site_spec) File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 364, in _recursive_create _recursive_create(filtered_dirs, new_output_dir, File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 342, in _recursive_create df = _gather_executive_summary( File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 49, in _gather_executive_summary for j in range(len(aggregated_info[exp_name][executive_summary_variables[0]])): KeyError: 'AverageReturnPerkWh'

Any idea?

[jieru-hu]

I have also experienced some issues when executing the following command:

create-compute-appendix --site_spec specifications.json --output_dir site data

loading region bounding boxes for computing carbon emissions region, this may take a moment... 454/454... rate=796.77 Hz, eta=0:00:00, total=0:00:00, wall=16:22 -044 Done! /Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix:30: FutureWarning: Passing a negative integer is deprecated in version 1.0 and will not be supported in future version. Instead, use None to not limit the column width. pd.set_option('display.max_colwidth', -1) Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Filtered dirs: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 431, in sys.exit(main(sys.argv[1:])) File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 427, in main _recursive_create(all_log_dirs, args.output_dir, site_spec) File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 364, in _recursive_create _recursive_create(filtered_dirs, new_output_dir, File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 342, in _recursive_create df = _gather_executive_summary( File "/Library/Frameworks/Python.framework/Versions/3.9/bin/create-compute-appendix", line 49, in _gather_executive_summary for j in range(len(aggregated_info[exp_name][executive_summary_variables[0]])): KeyError: 'AverageReturnPerkWh'

Any idea?

Could you provide a repro? Maybe upload the data/log?

[jieru-hu]

@mvesin Thanks for reporting. This should be fixed in master now. :)

@pnovoa feel free to open a new issue with the repro mentioned above. Thank you!