UTAustin_edm has forecasts whose values do not sum to 1.0

matthewcornell commented 6 years ago

I'm importing all component models into the reichlab 'Zoltar' forecast repository, and the validation code identified this problem with the following files in the directory https://github.com/FluSightNetwork/cdc-flusight-ensemble/tree/master/model-forecasts/component-models/UTAustin_edm . The cause is that they have rows whose values are 'NaN':

20101114-UTAustin_edm-20101129.cdc.csv Bin did not sum to 1.0. csv_filename=20101114-UTAustin_edm-20101129.cdc.csv, template_location=HHS Region 5, template_target=Season peak week, forecast_bin_sum=0
20110403-UTAustin_edm-20110418.cdc.csv Bin did not sum to 1.0. csv_filename=20110403-UTAustin_edm-20110418.cdc.csv, template_location=HHS Region 5, template_target=Season peak week, forecast_bin_sum=0 s 20111016-UTAustin_edm-20111031.cdc.csv
20111023-UTAustin_edm-20111107.cdc.csv Bin did not sum to 1.0. csv_filename=20111023-UTAustin_edm-20111107.cdc.csv, template_location=US National, template_target=Season peak week, forecast_bin_sum=0
20120101-UTAustin_edm-20120116.cdc.csv Bin did not sum to 1.0. csv_filename=20120101-UTAustin_edm-20120116.cdc.csv, template_location=HHS Region 2, template_target=Season peak week, forecast_bin_sum=0
20120108-UTAustin_edm-20120123.cdc.csv Bin did not sum to 1.0. csv_filename=20120108-UTAustin_edm-20120123.cdc.csv, template_location=HHS Region 6, template_target=Season peak week, forecast_bin_sum=0
20120115-UTAustin_edm-20120130.cdc.csv Bin did not sum to 1.0. csv_filename=20120115-UTAustin_edm-20120130.cdc.csv, template_location=HHS Region 2, template_target=Season peak week, forecast_bin_sum=0
20120401-UTAustin_edm-20120416.cdc.csv Bin did not sum to 1.0. csv_filename=20120401-UTAustin_edm-20120416.cdc.csv, template_location=HHS Region 10, template_target=Season peak week, forecast_bin_sum=0
20120408-UTAustin_edm-20120423.cdc.csv Bin did not sum to 1.0. csv_filename=20120408-UTAustin_edm-20120423.cdc.csv, template_location=HHS Region 1, template_target=Season peak week, forecast_bin_sum=0
20120415-UTAustin_edm-20120430.cdc.csv Bin did not sum to 1.0. csv_filename=20120415-UTAustin_edm-20120430.cdc.csv, template_location=HHS Region 1, template_target=Season peak week, forecast_bin_sum=0
20120422-UTAustin_edm-20120507.cdc.csv Bin did not sum to 1.0. csv_filename=20120422-UTAustin_edm-20120507.cdc.csv, template_location=HHS Region 2, template_target=Season peak week, forecast_bin_sum=0
20120429-UTAustin_edm-20120514.cdc.csv Bin did not sum to 1.0. csv_filename=20120429-UTAustin_edm-20120514.cdc.csv, template_location=HHS Region 5, template_target=Season peak week, forecast_bin_sum=0
20140330-UTAustin_edm-20140414.cdc.csv Bin did not sum to 1.0. csv_filename=20140330-UTAustin_edm-20140414.cdc.csv, template_location=US National, template_target=Season peak week, forecast_bin_sum=0
20141123-UTAustin_edm-20141208.cdc.csv Bin did not sum to 1.0. csv_filename=20141123-UTAustin_edm-20141208.cdc.csv, template_location=HHS Region 9, template_target=Season peak week, forecast_bin_sum=0
20151213-UTAustin_edm-20151228.cdc.csv Bin did not sum to 1.0. csv_filename=20151213-UTAustin_edm-20151228.cdc.csv, template_location=HHS Region 10, template_target=Season peak week, forecast_bin_sum=0
20160417-UTAustin_edm-20160502.cdc.csv Bin did not sum to 1.0. csv_filename=20160417-UTAustin_edm-20160502.cdc.csv, template_location=US National, template_target=Season peak week, forecast_bin_sum=0
20161023-UTAustin_edm-20161107.cdc.csv Bin did not sum to 1.0. csv_filename=20161023-UTAustin_edm-20161107.cdc.csv, template_location=HHS Region 1, template_target=Season peak week, forecast_bin_sum=0
20161030-UTAustin_edm-20161114.cdc.csv Bin did not sum to 1.0. csv_filename=20161030-UTAustin_edm-20161114.cdc.csv, template_location=HHS Region 10, template_target=Season peak week, forecast_bin_sum=0
20161120-UTAustin_edm-20161205.cdc.csv Bin did not sum to 1.0. csv_filename=20161120-UTAustin_edm-20161205.cdc.csv, template_location=HHS Region 3, template_target=Season peak week, forecast_bin_sum=0
20170423-UTAustin_edm-20170508.cdc.csv Bin did not sum to 1.0. csv_filename=20170423-UTAustin_edm-20170508.cdc.csv, template_location=HHS Region 8, template_target=Season peak week, forecast_bin_sum=0

nickreich commented 6 years ago

@sjfox this might help you track down errors...

sjfox commented 6 years ago

Thank you, this is extremely helpful! Hopefully can get things fixed this week.

FluSightNetwork / cdc-flusight-ensemble

UTAustin_edm has forecasts whose values do not sum to 1.0 #106