efficacy of usfan and shot

[zwyang6]

Thanks for your contributions to the community

When I reproduce your code I found that 'source train' achieved the best results of Task: STL9, Accuracy: 73.79%. But when i launched the 'shot_lap' or 'shot' train, the results are worse than Accuracy: 73.79%. I only got 68.50% for 'shot_lap' and 66.0% for 'shot', respectively. Does the usfan help to alleviate the domain-shift problem from cifar9 -> stl9 in your experiments?

Look forward to your reply !

[roysubhankar]

Hi,

In our paper we actually report the cifar9->stl9 setting. If you run cifar9 as source and stl9 as target then you will see improvement over shot with our usfan method.

[zwyang6]

Thanks for your reply !!!

Following your work, I did get the improvement over shot with usfan method from 66.0% to 68,.5%. But I found the best results is not acquired by neither usfan nor shot. when i reproduce your code I found that 'source train' without usfan or shot, achieved the best results of Task: STL9, Accuracy: 73.79%. It confused me

Look forward to your further explaination.

[zwyang6]

I see the henssian file generated at the source model training is not updated during target adaptaion training step. Should the hessian file be updated when train on target domain ?

[roysubhankar]

Thanks for your reply !!!

Following your work, I did get the improvement over shot with usfan method from 66.0% to 68,.5%. But I found the best results is not acquired by neither usfan nor shot. when i reproduce your code I found that 'source train' without usfan or shot, achieved the best results of Task: STL9, Accuracy: 73.79%. It confused me

Look forward to your further explaination.

For the cifar->stl there is improvement with both shot and usfan over the source trained baseline. However shot's improvement is marginal wrt the baselines. Usfan i think gains around 2%.

Yes, you are right, stl->cifar indeed doesn't bring gain with both the source free methods over source baseline.

[roysubhankar]

I see the henssian file generated at the source model training is not updated during target adaptaion training step. Should the hessian file be updated when train on target domain ?

Ideally you can update the Hessian computation during target training. Since we train on one target we keep it fixed, otherwise support from the source model will be lost. However, in a continual learning scenario with several sequential target domains, you may want to update the Hessian after each target domain. We didn't explore this scenario.

[zwyang6]

For the cifar->stl there is improvement with both shot and usfan over the source trained baseline. However shot's Yes, you are right, stl->cifar indeed doesn't bring gain with both the source free methods over source baseline.

So it embarassinhgly means that both source free methods don't work in this scenario ？ Shouldn't the proposed method brings improvement to the source baseline ?