search

godka / comyco-lin

Comyco for linear-based QoE
6 stars 0 forks source link

Reproducing results #1

Open ahmadhassandebugs opened 1 week ago

ahmadhassandebugs commented 1 week ago

Hi! I was trying to reproduce your results, but it didn't surpass the performance of Pensieve for all percentiles, as shown in the figure below. I trained the model for 500 epochs and it started to overfit after 480-490 epochs. I assume that is why you have set the total epochs to 500. The mean for comyco and pensieve is 0.936 and 0.925, respectively. Any idea how I can improve the performance further so that it is close to your pre-trained model performance. Thanks!

cdf

godka commented 1 week ago

Hi Ahmad,

https://github.com/godka/comyco-lin/blob/8cc945fb301a51f9c0cd7125fa648d2b3faef7c3/train.py#L115

Regarding the performances and overcoming the overfitting issues, the simplest way is to adjust the batch size via

actor_pool.get(2048) # batch_size = 2048, default: 256

Best, Tianchi