Species detection from pictures

[SebastianGer]

The species of the tree on the picture is supposed to be identified.

Open questions:

1. What exactly is the purpose of this feature? Do we assume that people generally don't know the species of tree they plant, do we want to take work off their hands by not having them manually choose the species from a list, or is it supposed to classify older trees, which were not planted by the person caring for it, so that they might not know the species?
2. For which points in its lifetime is the species detection supposed to work? A sapling looks very different from a full-grown tree.
3. Which solutions already exist and can we use them instead of creating our own ones?

I'll begin to work on the last point, to see which solutions exist and whether we can use them.

[SebastianGer]

Existing solutions:

• PlantSnap: Has a free and a paid app. Presumably won't give up their business by providing us services for free. Various others paid apps exist, which for the reason likely aren't good options.
• LeafSnap: Offers a dataset containing single images on white background of tree species from the northeastern USA. Probably too far from any real-life pictures to be useful to us.
• Flora Incognita: 4800 plant species focused on central europe, including trees. Developed at a university, so likely to be open to cooperation for good causes.
• Pl@ntNet: Offers API access to their classification service. 50 requests free per day, more for educational or research initiatives. Almost 30k plant species, including trees.
• Google Lense: Going by the prices of their Cloud Vision API their service would cost $1 per 1000 images. However, their label detection service does not seem to predict any plant species so far. During testing it only predicted very generic terms like 'plant' or 'flower'. Not sure if they offer this service separately anywhere or whether it's product-specific without a public API so far. I have also not found any details on the number of classes they predict.
• iNaturalist: They have a huge dataset of manually-tagged images, updated weekly. Those would however need to be downloaded one-by-one and filtered for acceptable licenses. They also held a Kaggle competition in 2019, for which the more successful approaches are shortly presented here.

[Davidezrajay]

1. "What exactly is the purpose of this feature? Do we assume that people generally don't know the species of tree they plant, do we want to take work off their hands by not having them manually choose the species from a list, or is it supposed to classify older trees, which were not planted by the person caring for it, so that they might not know the species?" It is to reduce workload for the tracker on the ground and improve the quality of our data. People often miss tag things. We also need this data to solve the update feature. It is meant to augment the taggers with a pre-tagging to make it easier for them as we are setting up to deal with millions of trees.
2. This is for the first two years of the tree's life