Automating NECB 2011 QAQC run

[padmassun]

Padmassun:

1. Need to correct output folder structure complete July-06
2. Add comments complete July-07
3. Modify btap_results_extractor to include the is_baseline key for the qaqc and simulations json file. Eliminate the use of btap_results_extractor July-10
4. Also change the folder structure of btap_results_extracter to be consistent with the necb-analysis-spreadsheet Eliminate the use of btap_results_extractor July-10

Source: https://github.com/canmet-energy/necb-analysis-spreadsheet

[padmassun]

Modify the cli-spreadsheet.rb's simulations.json file generation algorithm to add measures' arguments information

EDIT: July-07 completed

[padmassun]

Add Feature: Output a csv file instead of a .log file for error logging

EDIT: DONE July-06-2017

[padmassun]

From now on the simulations.json and qaqc.json file will have a is_baseline key (Default value set to false)

I will be adding a new options switch to specify whether the current run is a baseline.

• This switch will be -b or --baseline
• if the switch i invoked, this will set the key named is_baseline to true for qaqc.json and simulations.json file.

[padmassun]

TODO: modify btap_results_extractor to include the is_baseline key for the qaqc and simulations json file.

EDIT: Also change the folder structure of btap_results_extracter to be consistent with the necb-analysis-spreadsheet

[padmassun]

Added baseline switch and changed folder structure https://github.com/canmet-energy/necb-analysis-spreadsheet/commit/e2720f77ad0b31f0e5aaf8847a749189e1ee407

[padmassun]

Added ECM data to qaqc and simulations.json file https://github.com/canmet-energy/necb-analysis-spreadsheet/commit/d89a5a4bb11ca7ea140ba20e856cde948b4fb549

[padmassun]

There seems to be an issue with long file names, and windows not recognizing the filepath.

[padmassun]

I will try to run this from a linux system and see If I encounter this error.

[padmassun]

The linux system also had this issue.

The main cause of this issue was due to threading process from the parallel gem. After the threads has been changed to processes this problem was not encountered again.

Parallel.map(comp_datapoint_list, in_threads: 20) { |dp|

was changed to

processess = Parallel::processor_count
Parallel.map(comp_datapoint_list, in_processes: processess) { |dp|

[padmassun]

There was no more errors encountered. The simulations runs fine.

[padmassun]

Will be adding an index file which

1. maps the index of the simulation file to uuid.
2. map uuid to building_type and cities.
3. Contains a list of building_type and cities

This file will be used to optimize the parallel coordinates plot #256

Currently the cities and the building types are generated using the config file. Additionaly, each time the user regenerates the plot, the whole simulations.json file is iterated to find the information based on uuid, building_type and cities. This slows down the page...

[padmassun]

Completed