carbonplan / offsets-db-api

accounting of voluntary and compliance offset programs
https://carbonplan.org/research/offsets-db
MIT License
1 stars 0 forks source link

Add distinct count query for pagination #75

Closed andersy005 closed 9 months ago

andersy005 commented 9 months ago

This PR address the issue with total_entries being incorrect when using joined tables. When we join tables SQL's COUNT includes duplicate rows created by the join, leading to an inflated total count.

So, instead of counting rows directly in the joined query, we count distinct instances using an unique identifer e.g. primary key from one of our tables.

❯ http 'http://127.0.0.1:8000/projects/?search=kariba'                                     ─╯
HTTP/1.1 200 OK
content-length: 2090
content-type: application/json
date: Thu, 07 Dec 2023 19:46:34 GMT
server: uvicorn

{
    "data": [
        {
            "category": [
                "forest"
            ],
            "clips": [
                {
                    "date": "2023-03-23T00:00:00Z",
                    "id": 3,
                    "is_waybacked": false,
                    "notes": null,
                    "source": "Bloomberg",
                    "tags": [
                        "baseline",
                        "additionality"
                    ],
                    "title": "Faulty Credits Tarnish Billion-Dollar Carbon Offset Seller",
                    "type": "press",
                    "url": "https://www.bloomberg.com/news/features/2023-03-24/carbon-offset-seller-s-forest-protection-projects-questioned"
                },
                {
                    "date": "2023-01-19T00:00:00Z",
                    "id": 8,
                    "is_waybacked": false,
                    "notes": null,
                    "source": "ZEIT",
                    "tags": [
                        "additionality"
                    ],
                    "title": "Phantom Offsets and Carbon Deceit",
                    "type": "press",
                    "url": "https://www.zeit.de/wirtschaft/2023-01/co2-certificates-fraud-emissions-trading-climate-protection-english"
                },
                {
                    "date": "2023-01-27T00:00:00Z",
                    "id": 10,
                    "is_waybacked": false,
                    "notes": null,
                    "source": "Follow the Money",
                    "tags": [
                        "additionality",
                        "revenue-sharing"
                    ],
                    "title": "Showcase project by the world’s biggest carbon trader actually resulted in more carbon emissions ",
                    "type": "press",
                    "url": "https://www.ftm.eu/articles/south-pole-kariba-carbon-emission"
                },
                {
                    "date": "2023-10-16T00:00:00Z",
                    "id": 12,
                    "is_waybacked": false,
                    "notes": null,
                    "source": "Follow the Money",
                    "tags": [
                        "environmental-integrity"
                    ],
                    "title": "Shooting elephants and selling fictitious CO₂ rights: South Pole facilitates both",
                    "type": "press",
                    "url": "https://www.ftm.eu/articles/south-pole-kariba-big-game-hunting"
                },
                {
                    "date": "2023-10-16T00:00:00Z",
                    "id": 13,
                    "is_waybacked": false,
                    "notes": null,
                    "source": "The New Yorker",
                    "tags": [
                        "additionality",
                        "revenue-sharing"
                    ],
                    "title": "The Great Cash-for-Carbon Hustle",
                    "type": "press",
                    "url": "https://www.newyorker.com/magazine/2023/10/23/the-great-cash-for-carbon-hustle"
                }
            ],
            "country": "Zimbabwe",
            "first_issuance_at": "2013-12-23",
            "first_retirement_at": "2013-12-23",
            "is_compliance": false,
            "issued": 29016364,
            "listed_at": null,
            "name": "KARIBA REDD+ PROJECT",
            "project_id": "VCS902",
            "project_url": "https://registry.verra.org/app/projectDetail/VCS/902",
            "proponent": "Carbon Green Investments (Guernsey)",
            "protocol": [
                "vm0009"
            ],
            "registry": "verra",
            "retired": 21523313,
            "status": "unknown"
        }
    ],
    "pagination": {
        "current_page": 1,
        "next_page": null,
        "total_entries": 1,
        "total_pages": 1
    }
}

Cc @katamartin