search

brendel-group / objects-identifiability

https://brendel-group.github.io/objects-identifiability/
Other
3 stars 0 forks source link

Clarification #2

Open koriavinash1 opened 9 months ago

koriavinash1 commented 9 months ago

Hi @JackBrady, @zimmerrol,

Thanks for this interesting work and I very much appreciate the well written codebase.

I attempted to reproduce the results outlined in section 5.2, following the provided instructions. However, I'm facing some challenges as the maximum slot identifiability score achieved in my runs was less than 0.3. I'm curious if this variability could be attributed to random seeds, alone. If so, could you kindly share the seeds used for data generation and across all models?

Cheers, Avinash

JackBrady commented 9 months ago

Hi Avinsash,

Thanks for your message and interest in our work!

Could you tell me which model you were training in your runs i.e. Slot Attention, Monet, or the additive autoencoder?

Best, Jack

On Tue, Dec 5, 2023 at 11:45 AM Avinash @.***> wrote:

Hi @JackBrady https://github.com/JackBrady, @zimmerrol https://github.com/zimmerrol,

Thanks for this interesting work and I very much appreciate the well written codebase.

I attempted to reproduce the results outlined in section 5.2, following the provided instructions. However, I'm facing some challenges as the maximum slot identifiability score achieved in my runs was less than 0.3. I'm curious if this variability could be attributed to random seeds. If so, could you kindly share the seeds used for data generation and across all models?

Cheers, Avinash

— Reply to this email directly, view it on GitHub https://github.com/brendel-group/objects-identifiability/issues/2, or unsubscribe https://github.com/notifications/unsubscribe-auth/AEZY7U6WQDS3ZTEHIJUERCTYH33NHAVCNFSM6AAAAABAHOY2JCVHI2DSMVQWIX3LMV43ASLTON2WKOZSGAZDKOBYHEZTMMY . You are receiving this because you were mentioned.Message ID: @.***>

koriavinash1 commented 9 months ago

Hi Jack, I tried all three models, but the scores differed in all three models. PFA of obtained SIS scores for random seeds 807, 5270, and 8573 for AE, SA, and MONET, respectively.

SIS_Epochs.pdf Please let me know if I'm missing anything.

Thank you, Avinash

JackBrady commented 9 months ago

Hi Avinsash,

Thanks for the plot!

I will try executing the runs with the seeds you mentioned and see if I can debug the reason for the low scores.

Best, Jack