Script to marshal csv SmartFire data to bluesky input structure

Susan will provide csv file structure and examples.

It will probably be something like the following:

Fires CSV

id,event_id,latitude,longitude,type,area,date_time,slope,state,country,fips,scc,fuel_1hr,fuel_10hr,fuel_100hr,fuel_1khr,fuel_10khr,fuel_gt10khr,shrub,grass,rot,duff,litter,moisture_1hr,moisture_10hr,moisture_100hr,moisture_1khr,moisture_live,moisture_duff,consumption_flaming,consumption_smoldering,consumption_residual,consumption_duff,min_wind,max_wind,min_wind_aloft,max_wind_aloft,min_humid,max_humid,min_temp,max_temp,min_temp_hour,max_temp_hour,sunrise_hour,sunset_hour,snow_month,rain_days,heat,pm25,pm10,co,co2,ch4,nox,nh3,so2,voc,VEG,canopy,event_url,fccs_number,sf_event_guid,sf_server,sf_stream_name,timezone
SF11C112212114042848520,SF11E698479,26.313,-77.123,RX,99.9999997516,201604120000-05:00,10.0,Unknown,Unknown,-9999,2810015000,,,,,,,,,,,,10.0,12.0,12.0,22.0,130.0,150.0,,,,,6.0,6.0,6.0,6.0,40.0,80.0,13.0,30.0,4,14,6,19,5,8,,,,,,,,,,,,,http://128.208.123.111/smartfire/events/ff7c6ee9-5f54-4fd3-bd22-ebff53b429f9,,ff7c6ee9-5f54-4fd3-bd22-ebff53b429f9,128.208.123.111,realtime,-5.0

Events CSV

id,event_name,total_area,total_heat,total_pm25,total_pm10,total_pm,total_co,total_co2,total_ch4,total_nmhc,total_nox,total_nh3,total_so2,total_voc,total_bc,total_h2,total_nmoc,total_no,total_no2,total_oc,total_tpc,total_tpm
SF11E698479,"Unnamed fire in Palm Beach County, Florida",264.7058816955166,0.0,0.0,0.0,0,0.0,0.0,0.0,0,0.0,0.0,0.0,0.0,0,0,0,0,0,0,0,0

Resulting Bluesky input data

[
    "fires": {
        'id': 'SF11C112212114042848520',
        'type': 'RX',
        "event_of": {
            'id': 'SF11E698479',
            "name": "Unnamed fire in Palm Beach County, Florida",
            'url': 'http://128.208.123.111/smartfire/events/ff7c6ee9-5f54-4fd3-bd22-ebff53b429f9',

        },
        "activity": [
            {
                "active_areas": [
                    {
                        'country': 'Unknown',
                        'start': '2016-04-12T00:00:00",
                        "utc_offset": "-05:00',
                        'max_humid': 80.0,
                        'max_temp': 30.0,
                        'max_temp_hour': 14,
                        'max_wind': 6.0,
                        'max_wind_aloft': 6.0,
                        'min_humid': 40.0,
                        'min_temp': 13.0,
                        'min_temp_hour': 4,
                        'min_wind': 6.0,
                        'min_wind_aloft': 6.0,
                        'moisture_100hr': 12.0,
                        'moisture_10hr': 12.0,
                        'moisture_1hr': 10.0,
                        'moisture_1khr': 22.0,
                        'moisture_duff': 150.0,
                        'moisture_live': 130.0,
                        'fuel_100hr': '',
                        'fuel_10hr': '',
                        'fuel_10khr': '',
                        'fuel_1hr': '',
                        'fuel_1khr': '',
                        'fuel_gt10khr': '',
                        'rain_days': 8,
                        'slope': 10.0,
                        'snow_month': 5,
                        'state': 'Unknown',
                        'sunrise_hour': 6,
                        'sunset_hour': 19,
                        "specified_points": [
                            {
                                'lat': 26.313,
                                'lng': -77.123,
                                "name": "HMW-32434",
                                "area": 99.9999997516
                            }
                        ]                        
                    }
                ]
            }
        ]       
    }
]

pnwairfire / bluesky

Script to marshal csv SmartFire data to bluesky input structure #109