search

Klimatbyran / garbo

Klimatkollen's data pipeline, processing company sustainability reports
MIT License
5 stars 11 forks source link

Need to link multiple pdfs to the same company #110

Open Almendra-ab opened 6 months ago

Almendra-ab commented 6 months ago

We need to find a way to link multiple reports to the same company. There are two situations that can have this need, one is that the data is not all presented in the same pdf (example: banks such as Handelsbanken and Arion have a separate report for their financed emissions), and the other case is when we have reports from different years. This can be taking data from the 2022 report to get more historical data, or when we will add the 2024 report in the future.

I can see how this could be complex as we in the case with Arion, have financial data from 2021 that is updated in the latest report. But the earlier report has data for 2019, which is not included in the latest report. Basically the prompt would be to take all the data available from the latest report, and use older reports to fill in the gaps. A new report would override the old where there is data, but leave the old data where no new is found.

Greenheart commented 2 weeks ago

We now have support for this in the database, since we can save and change individual data points :)

However, to fully resolve this issue, we also need to update the garbo prompts as well as validation in the API to make sure we don't overwrite for example 2021 emissions data from the 2021 report with 2021 emissions data from the 2023 report.