plantnet / PlantNet-300K

[NeurIPS2021] A plant image dataset with high label ambiguity and a long-tailed distribution
https://doi.org/10.5281/zenodo.5645731
BSD 2-Clause "Simplified" License
167 stars 31 forks source link

Plant Category Name #11

Open nomoneyExpection opened 1 year ago

nomoneyExpection commented 1 year ago

Are you sure that. json file is correct? Why did I display the ID of the first image as 4 when running the demo, but an error was reported indicating that this key does not have a corresponding value? Also, if possible, could you upload a version of the. txt key and value that corresponds to each other? Additionally, from the code, there are over 1080 classes in total, right

maximiliense commented 1 year ago

There are 1081 classes in total. None of them has ID 4. The classes ID starts at 1355868 and ends up at 1718287. However, note that deep learning frameworks label classes from 0 to the number of classes minus one (here 1080). I guess that 4 means that it is the $4^{th}$ in the deep learning framework. You should check the mapping from "PyTorch id" to "dataset id". It is given in the attributes returned by the get_data function.

Milk-Cool commented 1 year ago

It is given in the attributes returned by the get_data function.

It requires folders 'train', 'val' and 'test', which are not referenced by any other scripts. How do I get these?

Milk-Cool commented 1 year ago

Alright so I did a bit of searching and found out that you'd have to split the dataset into these three folders. Do I need to use the splitfolders library here or should I use something else?

Milk-Cool commented 1 year ago

Also something's wrong with the pretrained model. Here's my code: https://pastebin.com/jvFjJsxc (don't look at how I tried mapping indexes to ids, it probably won't work). It always returns index 204 for some reason. Or it is just me being dumb and not using the model properly. Should I create an issue on this?

Milk-Cool commented 1 year ago

@maximiliense sorry for pinging, but it's been three days and you still haven't answered so I have no clue if you've seen this or not.

garcinc commented 1 year ago

It requires folders 'train', 'val' and 'test', which are not referenced by any other scripts. How do I get these?

These are folders that already exist when you download the dataset, so you don't have to create them. Make sure to download the latest version in Zenodo : https://zenodo.org/record/5645731

garcinc commented 1 year ago

Also something's wrong with the pretrained model. Here's my code: https://pastebin.com/jvFjJsxc (don't look at how I tried mapping indexes to ids, it probably won't work). It always returns index 204 for some reason. Or it is just me being dumb and not using the model properly. Should I create an issue on this?

Which images are you testing on ? Are these images in the test set ? How many images did you try ? Thanks

Milk-Cool commented 1 year ago

These are folders that already exist when you download the dataset, so you don't have to create them. Make sure to download the latest version in Zenodo : https://zenodo.org/record/5645731

Thank you so much for replying! These folders did not show up in the archive preview, so I was a little confused there. I will try downloading the dataset.

Milk-Cool commented 1 year ago

Which images are you testing on ? Are these images in the test set ? How many images did you try ? Thanks

I am testing on the images from the README from this repository, plus one image of my plant. I'll send it when I get home

Milk-Cool commented 1 year ago

https://files.catbox.moe/x7meas.jpg

garcinc commented 1 year ago

Hi @Milk-Cool,

I have the same behaviour as you on these images. However, when I check the performances of the pre-trained model (resnet18) on the test set of Pl@ntNet-300K I get back the performances reported in the paper (i.e., ~70% of accuracy on the whole test set). Keep in mind that the macro-average accuracy , which is approximately 30%, is much lower than the accuracy , which means that if you draw a class at random and an image in that class, you will be right only 30% of time. This is because the classes in the long tail are very hard to discriminate. This is explained more thoroughly in the paper which I encourage you to read. This could explain what you observe. If you look at the top-5 predictions of the images you are talking about, you will see that they are different. The model might predict class 204 often because this is an overly represented class in the training set. To evaluate the model, I encourage you to test it on a large set of images belonging to the 1081 species covered by the dataset. I hope this helps.

Bests

Milk-Cool commented 1 year ago

Thanks for the answer @garcinc ! That explains pretty much everything.

dazmashaly commented 11 months ago

@Milk-Cool did you ever get better results?

Milk-Cool commented 11 months ago

@Milk-Cool did you ever get better results?

no, not really, i'm too lazy to train the model tbh