digitaldutch / roaddanger.org

A website and database for news reports about traffic crashes around the world.
https://www.roaddanger.org/
MIT License
12 stars 3 forks source link

Data export seems incomplete, limited at 10000 crashes #12

Closed b-a0 closed 7 months ago

b-a0 commented 8 months ago

Sorry, another issue... (I think).

It seems as if not all data is exported in the gzip JSON file.

When exporting the data via https://nl.roaddanger.org/export/ I get:

However, when I visit https://nl.roaddanger.org/statistics/general I see there are approximately 14k crashes (this corresponds with the current crash ID's):

image

Due to the neat amount of 10000 items I suspect there is some limiting going on. I don't know PHP, but the export function does mention $maxRows = 10000; and this variable is used later in the SQL query. Could this be the culprit?

digitaldutch commented 7 months ago

This is fixed in latest version. There is an artificial limit. It has now been set to 100,000 crashes for the export.