This is based strongly on ALUM, but also tempered by experience with validation in Northland.
I didn't actually know that ALUM had this process when doing this for Northland, so it's encouraging that we developed a pretty similar process. James and I took a stratified random sample of 1,000, and split it in half. We used MapAccuracy to do it efficiently. Where we deviated from ALUM is that we quite explicitly didn't do a validation or confusion matrix, but rather a "consistency" check where a land use was considered either "consistent" or "inconsistent" with aerial imagery. e.g. a field is not consistent with heavy industry, but various forms of agriculture are all consistent with a field, but we didn't assess anything stricter than "consistency". We did this because we only had ~4 year old aerial imagery and no local knowledge to go on. We would have preferred to have tasked others with validation, but we lacked any resource to do this.
For the framework, I think we ought to aim for actual validation, but bear in mind the principle of "practicality".
This is based strongly on ALUM, but also tempered by experience with validation in Northland.
I didn't actually know that ALUM had this process when doing this for Northland, so it's encouraging that we developed a pretty similar process. James and I took a stratified random sample of 1,000, and split it in half. We used MapAccuracy to do it efficiently. Where we deviated from ALUM is that we quite explicitly didn't do a validation or confusion matrix, but rather a "consistency" check where a land use was considered either "consistent" or "inconsistent" with aerial imagery. e.g. a field is not consistent with heavy industry, but various forms of agriculture are all consistent with a field, but we didn't assess anything stricter than "consistency". We did this because we only had ~4 year old aerial imagery and no local knowledge to go on. We would have preferred to have tasked others with validation, but we lacked any resource to do this.
For the framework, I think we ought to aim for actual validation, but bear in mind the principle of "practicality".