IN-CORE / pyincore

pyIncore is a component of IN-CORE. It is a python package consisting of two primary components: 1) a set of service classes to interact with the IN-CORE web services, and 2) IN-CORE analyses . The pyIncore allows users to apply various hazards to infrastructure in selected areas, propagating the effect of physical infrastructure damage and loss of functionality to social and economic impacts.
Mozilla Public License 2.0
25 stars 7 forks source link

Some improvements and bug fixes when handling error codes and unmapped inventory in pyincore analyses #10

Open diegoac2 opened 3 years ago

diegoac2 commented 3 years ago

Issue1: When there are inventory items not mapped to a DFR3, the csv result of the analysis generates an empty row with just "guid" field. Provide feedback to user why there is empty row - may be a debug message?

Issues2:  if the first row of the output has an error or is unmapped to a fragility, the CSV generation fails. - This can be fixed by using pandas instead of "csv" package. Or continue using "csv", iterate through entire list to find the longest dict and use it to generate the headers (little expensive than pandas option) example, this works:

{{"guid","LS_0","LS_1","LS_2","LS_3","DS_0","DS_1","DS_2","DS_3","DS_4","haz_expose" "00919ec2-5c9d-4269-8ddb-e8d2a0d960d4","0.8608868844","0.6585604091","0.5295291166","0.3289412247","0.1391131156","0.2023264753","0.1290312925","0.2005878919","0.3289412247","yes" "efd6e76c-cf38-4265-be16-b5c6b53843b2","","","","","","","","","","error" "5f1a56e1-07ec-4e4d-a6a4-ee5146f4da56","0.7273581592","0.5920946565","0.471725649","0.2573799957","0.2726418408","0.1352635027","0.1203690075","0.2143456533","0.2573799957","yes"}}

baseanalysis fails to generate an output like below when the first row doesn't have LS/DS values:

{{"guid","LS_0","LS_1","LS_2","LS_3","DS_0","DS_1","DS_2","DS_3","DS_4", "haz_expose" "00919ec2-5c9d-4269-8ddb-e8d2a0d960d4","","","","","","","","","","error" "efd6e76c-cf38-4265-be16-b5c6b53843b2","0.7273581592","0.5920946565","0.471725649","0.2573799957","0.2726418408","0.1352635027","0.1203690075","0.2143456533","0.2573799957", "yes" "5f1a56e1-07ec-4e4d-a6a4-ee5146f4da56","0.7273581592","0.5920946565","0.471725649","0.2573799957","0.2726418408","0.1352635027","0.1203690075","0.2143456533","0.2573799957", "yes"}}  

Discussion item:

if there were errors calculating damage for a few inventory items in the analysis, the result will contain some rows with empty values for LS_x & DS_x. Even though the analysis itself works fine, when an output like this chanined to MCS or mean damage it will fail because it expects to find LS/DS values. We should discuss how to handle chained analyses in this case.

longshuicy commented 1 year ago

Duplicated to #195 #131