US-EPA-CAMD / easey-ui

Project Management repo for EPA Clean Air Markets Division (CAMD) Business Suite of applications
MIT License
0 stars 0 forks source link

Emissions import errors from historical #5960

Open ntknguyen opened 7 months ago

ntknguyen commented 7 months ago

Import Big Bend BB06 2023 Q3 emissions from historical. Private Zenhub Image

ergjustin commented 5 months ago

@esaber76 It would be helpful if you provide more information about this ticket so that I can begin working on it.

esaber76 commented 5 months ago

@ergjustin You just need to give yourself access to Big Bend (ORIS 645), go to Emissions and Open and Checkout BB06. Then select Import Data, Import From Historical Data, Reporting Period = 2023 Q3, and Import. The error will eventually come up.

ergjustin commented 5 months ago

As per the discussion with Dwayne, my understand is: MODC 47 (or 46 or 48) should not occur when FLOW exists, so this should not be an issue for CO2 CEMS. I believe it should only affect the NOXR CEMS view and that view filters out rows with MODC 47 or 48.

WHERE EXITSTS ( SELECTE 1 FROM camdecmpswks.DERIVED_HRLY_VALUE WHERE HOUR_ID= dhv. HOUR_ID AND MON_LOC_ID = dhv.MON_LOC_ID AND RPT_PERIOD_ID=dhv.RPT_PERIOD_ID AND PARAMETER_CD =ANY(ARRAY: ['CO2']) ) AND EXISTS ( SELECT 1 FROM camdecmpswks.MONITOR_ HRLY_VALUE WHERE HOUR_ID= mhv. HOUR_ID AND MON_LOC_ID = mhv.MON_LOC_ID AND RPT_PERIOD_ID=mhv.RPT_PERIOD_ID AND PARAMETER_CD =ANY(ARRAY: ['FLOW']) )

djw4erg commented 5 months ago

Justin is correct.

Restating and adding:

The ECMPS 1.0 version of the query for CO2 CEMS would not produce rows for hours where either a CO2 DHV or FLOW MHV row did not exist, and in this case the FLOW rows do not exist. The logic to improve efficiency that flattens the rows for DHV parameters and separately MHV parameters, into a single row inadvertently eliminated the requirement that CO2 DHV and FLOW MHV rows must exist to produce a row for the CO2 CEMS view.

Possible Fixes:

  1. Change the use of the dhvParamCodes and mhvParamCodes variables in the where clause to use just 'CO2' and 'FLOW', which will require each but not require the other parameters to exist.
  2. Replace the where clause to check that dhv.co2_hour_id and mhv.flow_hour_id are not null.

Additional MODC 47 and 48 Information:

The 'duplicate' MODC 47 (and 48) rows should only matter for the NOXR CEMS view, which filters out the MODC 47 and 48 rows. So no duplicate rows for a location and hour should exist for any emission view.

ergjustin commented 5 months ago

Fixed this issue in dev with help of Dwayne, Now able to Import Big Bend BB06 2023 Q3 emissions from historical. Thanks Dwayne.

import-dev.PNG