arthurdouillard / dytox

Dynamic Token Expansion with Continual Transformers, accepted at CVPR 2022
https://arxiv.org/abs/2111.11326
Apache License 2.0
132 stars 18 forks source link

imagenet-100 result cannot reproduce #6

Closed zhaoedf closed 2 years ago

zhaoedf commented 2 years ago

for cifar and imagenet-1000, i can get similar results as what paper reported. how ever, for imangenet-100 only, i cannot get the same result, with 3~4% deviation.

the followings are my running scripts and experiment results.

scipts

bash train.sh 6,7
--options options/data/imagenet100_10-10.yaml options/data/imagenet100_order1.yaml options/model/imagenet_dytox.yaml \
--name dytox \
--data-path ~/ilsvrc2012 \
--output-basedir ./checkpoints

results (i ran two times on different machines)

image

image

i focus on both avg and last acc.

arthurdouillard commented 2 years ago

Hello, I've saw your issue. I'll first look into https://github.com/arthurdouillard/dytox/issues/5, which may (or not) explain your issue and #2

I'll get back to you asap. Sorry for the inconvenience.

zhaoedf commented 2 years ago

Hello, I've saw your issue. I'll first look into #5, which may (or not) explain your issue and #2

I'll get back to you asap. Sorry for the inconvenience.

ok, i totally understand.

GengDavid commented 2 years ago

Hi @arthurdouillard
I also tried ImageNet-100 and got a similar result as @zhaoedf.

BTW, I wonder that is it possible to share your trained checkpoints with us?

GengDavid commented 2 years ago

Hi all, I also tried ImageNet-1000 but got a lower result:( I am using Pytorch 1.10.1, CUDA 10.1

dytox_imgnet

I'll try to use a different Pytorch version and try again.

arthurdouillard commented 2 years ago

Please see erratum.