Updated tutorial

[V0XNIHILI]

SOLVED

@Maxtimer97 right now, the tutorial demonstrates how to download the dataset, but we can also just set the download is true flag when creating the MSWC class right?

[V0XNIHILI]

SOLVED

Also, some headings in the tutorial now end with ":" while others don't, I think we should pick one and use that one consistently:

[V0XNIHILI]

How does this work?

Is every batch guaranteed to only have samples from one class? If yes, I think this should be clarified because that is not an assumption users will make immediately I think.

[V0XNIHILI]

SOLVED with d76324b

BATCH_SIZE is defined here:

But only in a few data loaders and not in all + it's not clear why only in those.

[V0XNIHILI]

Not related to the tutorial, but why isn't query_classes part of query?

[V0XNIHILI]

Not clear why this is the case:

[Maxtimer97]

@Maxtimer97 right now, the tutorial demonstrates how to download the dataset, but we can also just set the download is true flag when creating the MSWC class right?

I didn't see this download code before, I agree that it is nicer to use it. However when trying to run it, it didn't work for me. I rearranged it in this new commit which works for me: c89735670a9a1d6917ab61544208de4da43eb0ae This now requires the root to be just the data/ folder and adds the MSWC/ manually.

Can you check if this works for you/ if you see some mistakes I incerted?

[V0XNIHILI]

Works @Maxtimer97!

[Maxtimer97]

Also, some headings in the tutorial now end with ":" while others don't, I think we should pick one and use that one consistently:

SOLVED: 4e692ed

[Maxtimer97]

How does this work?

Is every batch guaranteed to only have samples from one class? If yes, I think this should be clarified because that is not an assumption users will make immediately I think.

I provided an extra explanation when loading the dataloader. Do you think it's clear now? It is in this commit: d26e297

[Maxtimer97]

Not related to the tutorial, but why isn't query_classes part of query?

My point was that the list of all unique classes present in the dataset (query_classes) is not a usual argument to have in such an object so it is better to give it directly to users so they don't miss it (as it is quite useful in FSCIL). I would keep it like that if it's not a problem with you.

[Maxtimer97]

Not clear why this is the case:

I added a bit more explanation in commit c3da584 The idea is just that we want to define the Benchmark object only once fixing the metrics of interest and then change the data to test on over successive sessions.

[jasonlyik]

@V0XNIHILI @Maxtimer97 The tutorial and cleaned up script look really great! Merging it all to prep for v1.0.