search

hongzimao / pensieve

Neural Adaptive Video Streaming with Pensieve (SIGCOMM '17)
http://web.mit.edu/pensieve/
MIT License
516 stars 279 forks source link

If you are using the same traces, why will the QoE value increase ? #121

Open mandannaresh8686bits opened 3 years ago

mandannaresh8686bits commented 3 years ago

Dear Sir,

I have a small doubt regarding QoE(reward) in Linear. I did the MAHIMAHI experiment two times successfully, But I got the different QoE's for the first and second experiments with respect to FCC and NORWAY traces.

In the first experiment, I received QoE of approximately 44 for both NORWAY and FCC traces. In the second experiment, I received QoE of approximately 45.64 for NORWAY and 49 for FCC traces.

Could you please let me know the reason why I got a higher QoE in the second experiment than in the first experiment even I am using same traces i.e FCC and NORWAY.

hongzimao commented 3 years ago

For the first one, what's your QoE?

The result looks quite similar actually (differ by only 2%). Notice that Mahimahi is a realistic network emulator. It's creating the network quality as replaying the trace. Now, when you do the experiment, all network events going in and out the mahimahi shell will experience the emulated network. That being said, in real-world experiments, some other network events may interfere, thus the small difference you see.

hongzimao commented 3 years ago

Btw, as a result of this noise in QoE, when comparing different ABR algorithms, you want to look at consistent QoE improvement that clearly stands out of the noise.