Stored Emissions non-calculation

[OpenOilUG]

About 31,000 projects out of 72,000 in the total do not contain values for emissions. So far there are three identified causes:

1. Projects producing coal - there are no coal projects which have emissions entries in the co2e_information field
2. Projects which only have production data by data_year and not year (overwhelmingly using GEM or Opgee data source_id = 15,16)
3. Projects where the latest year in project_data_point (pdp) has volume = 0 although earlier years did have production volumes (in other words where a project has stopped producing). This could be addressed by deleting all rows from project_data_point (pdp) where volume = 0 (i was going to do this but count suggests there are 48,000 records so i thought i would leave it for you in case i made a mistake!)

PLUS:

1. Need simple way for me to export values into a spreadsheet in a form that is directly useable in calculations

@niklaserik

• [x] Post current excel model in issue
• [ ] Delete rows with latest year = 0

[niklaserik]

@OpenOilUG

GRFF_220607_0943_dev.xlsx

[OpenOilUG]

Confirmed the model is intact. Worked comparisons attached for reference. GRFF_220607_0943_dev_230331.xlsx

[OpenOilUG]

UPDATE: of points above 1 and 2 now dealt with. 3 is dealt with indirectly by the separate "is visible" switch.

REMAINING: export mechanism to allow json to be imported as a table structure in Excel.

ADDITIONAL:

• add boolean field to json to indicate whether the year used is an actual year (from "Year") or the year the data were collected ("Data Year")
• add also project_id in the json object so that the object can be treated separately, sent to others etc.

[OpenOilUG]

Sample json file as csv is OK. Would like addition of two fields above - indication of whether year number is from data_Year or not, and project_id.

Also remove production_co2e and methane_m3_ton from csv export as these relate to earlier mechanism to grade projects in web orderings and will only confuse this dataset.

Question: how is the csv file stored within the database? Is it a single field, like the JSON?

Test excel from json 230703.xlsx

[niklaserik]

Sounds good. I'll update that.

Question: how is the csv file stored within the database? Is it a single field, like the JSON? No, I need to generate the CSV myself. Do you want to save it to the database? I thought is was just for verifying that the data is correct?

[OpenOilUG]

Sounds good. I'll update that.

Question: how is the csv file stored within the database? Is it a single field, like the JSON? No, I need to generate the CSV myself. Do you want to save it to the database? I thought is was just for verifying that the data is correct?

No, there are two purposes:

1) Immediately, to be able to supply all fields and their latest production and emissions totals to an open source initiative OS-Climate. 2) I wonder if in the long run it is best to run the emissions calculations on a cycle and then call the values instea dof dynamically calculating them on the page. However that is a longer term project and discussion.

[OpenOilUG]

On review, i am wondering if the volumes have been converted accurately. for instance i only see one unit used for the oil fields which is e3bbls. but when i look in the project_data_point i find lots of different units in there (each preserved as in original source). so i assume some conversion has been done, otherwise there wouldn't be only one end unit. but then i get some numbers which look very high... making me wonder how the conversion proce ss has worked. I cannot make the file available to OS-Climate until we are sure the volumes are correct.