hello

[depcco]

I read your paper and found Table 1. Now I have some questions. I noticed that the concept shift has a higher value than the covariate shift, but in my opinion, the concept shift is a harder task. Could you explain why?

[depcco]

Is there a difference in difficulty between covariate shift and concept shift within the same dataset? I hope to get your explanation.

[CM-BF]

Hi @depcco,

It is true that concept shift is harder than covariate shift from some perspectives, but the performances are not comparable on difference shifts. The performance drop indicates the sensitivity of the model given a specific shift, but does not reflect the difficulty of the shifts. For example, models cannot perform well under covariate shift, but it does not mean that covariate shift is hard to solve vice versus.

Best, Shurui

[depcco]

Thank you very much, I have gained a lot of valuable knowledge from our discussions. I hope you can help me with another question regarding experimental data.I noticed that after the final training is completed, it outputs a "Loading best Out-of-Domain Checkpoint number" and a "Loading best in-Domain Checkpoint number". I would like to know which output results the data IDs IDID and OODOOD in the experimental data table correspond to. Also, i read you another paper-LECI，are the differences between different exp_round solely due to hyperparameters?

[CM-BF]

I would like to know which output results the data IDs IDID and OODOOD in the experimental data table correspond to

IDID uses the In-Domain checkpoint. OODOOD uses the Out-of-Domain checkpoint.

are the differences between different exp_round solely due to hyperparameters?

The difference mainly attributes to the change of random seeds.

[depcco]

@CM-BF Thank you for your response. However, when running LECI, I noticed that the data varies significantly between different exp_rounds. For example, in the fourth round, the OODOOD value is 41.47, whereas in the ninth round, the OODOOD value is 72.33. I would like to know if this is a normal situation. Did you take the average of the data over ten rounds or select the best round for the experimental data in your article? The data I reproduced comes from GOOD motif，size。Looking forward to your reply.

[CM-BF]

We take the average which was detailed in the paper. :)