I want to ask a question about integrating ABR

[lilee97]

Hello, I am a graduate student. I want to ask a question about your work NAS. When you train the integrated ABR algorithm, how do you calculate the reward when the integrated ABR chooses to download DNN? The paper only mentioned reward when downloading video blocks.

[chaos5958]

We used the QoE metric consisting of 1) effective bitrate (described in the paper), 2) rebuffering time, and 3) quality difference. When downloading a DNN chunk, it increases the effective bitrate. So, the RL agent tries to find a policy that download DNNs chunks faster while avoiding rebuffering.

[lilee97]

We used the QoE metric consisting of 1) effective bitrate (described in the paper), 2) rebuffering time, and 3) quality difference. When downloading a DNN chunk, it increases the effective bitrate. So, the RL agent tries to find a policy that download DNNs chunks faster while avoiding rebuffering.

Thank you very much for your reply! I'm trying to reproduce the rest now. Could you explain it in more detail? In the A3C training architecture, when the actor decides to download the video chunk, it will get a reward. The paper also mentioned how to calculate this reward. However, when the actor decides to download a DNN chunk, it seems that the decision will also get a reward, but the paper does not mention how to calculate the reward.

[lilee97]

We used the QoE metric consisting of 1) effective bitrate (described in the paper), 2) rebuffering time, and 3) quality difference. When downloading a DNN chunk, it increases the effective bitrate. So, the RL agent tries to find a policy that download DNNs chunks faster while avoiding rebuffering.

In the picture, the paper only mentioned how to calculate the reward when the actor decides to download the video chunk, but did not mention how to calculate the reward when the actor decides to download the DNN chunk.

[chaos5958]

Here is our source code about how we give a reward for a DNN chunk. When an agent tries to select a DNN chunk after all the chunks are downloaded, we give an additional penalty (opt.dnn_penalty is set to 0)

            else:#reward for dnn
                if not last_end_of_dnn:
                    reward = -opt.log_rebuf * rebuf
                else:
                    reward = -opt.log_rebuf * rebuf-opt.dnn_penalty

[lilee97]

这是我们如何为 DNN 块提供奖励的源代码。 当代理在下载所有块后尝试选择一个 DNN 块时，我们会给予额外的惩罚（opt.dnn_penalty设置为 0）

            else:#reward for dnn
                if not last_end_of_dnn:
                    reward = -opt.log_rebuf * rebuf
                else:
                    reward = -opt.log_rebuf * rebuf-opt.dnn_penalty

Thank you very much for showing me some of the source code. But I have some doubts (1) If opt dnn_ penalty is set to 0, what is the difference between the upper and lower rewards in the picture? (2) What is the value of opt. log_rebuf?

[lilee97]

这是我们如何为 DNN 块提供奖励的源代码。 当代理在下载所有块后尝试选择一个 DNN 块时，我们会给予额外的惩罚（opt.dnn_penalty设置为 0）

            else:#reward for dnn
                if not last_end_of_dnn:
                    reward = -opt.log_rebuf * rebuf
                else:
                    reward = -opt.log_rebuf * rebuf-opt.dnn_penalty

I also try to modify the pensieve architecture, but I don't know (1) how much reward is appropriate when downloading a DNN chunk (2) when the number of remaining DNN chunks is 0, and the actor still decides to download DNN, how much reward is appropriate at this time

[chaos5958]

(1) It is shown in the code snippet. We used the same penalty for rebuffering as in Pensive (2) That's why you need to give an penalty when the number of remaining DNN chunks become 0.

[lilee97]

(1) 如代码片段所示。我们对重新缓冲使用了与 Pensive (2) 中相同的惩罚， 这就是为什么当剩余 DNN 块的数量变为 0 时需要给予惩罚。

Thank you for your reply. In my previous tests, I found that content-aware DNN seems to have less super-resolution effect on video than it does on pictures. This phenomenon is mainly reflected in the super-resolution of 720p video. The SSIM and PSNR values of 720p video after super-resolution are not as good as those of 720p pictures. Would you please tell me the same is true in your test?

[chaos5958]

That's true if you super-resolve an image from 720p to 1080p. The benefit increases as the gap btw input and output resolution becomes larger. Also, the benefit is smaller for videos than (lossless) pictures since compression artifacts generally negates the super-resolution effect.

[lilee97]

That's true if you super-resolve an image from 720p to 1080p. The benefit increases as the gap btw input and output resolution becomes larger. Also, the benefit is smaller for videos than (lossless) pictures since compression artifacts generally negates the super-resolution effect.

Please let me ask one more question. Generally, QoE only considers the influence of video chunks. When calculating QoE, do you consider the factor of DNN chunks? Is your reward the same as QoE? Or does your QoE only consider video chunks?

[chaos5958]

We also consider the QoE of DNN chunks in two ways: give penalty for occurring rebuffering or choosing to download a DNN chunk even after downloading the entire DNN.

[lilee97]

We also consider the QoE of DNN chunks in two ways: give penalty for occurring rebuffering or choosing to download a DNN chunk even after downloading the entire DNN.

I see that you have four quality DNN models. Does this mean that you need to train four integrated ABR algorithms?Is a quality DNN model corresponding to an integrated ABR algorithm?

[chaos5958]

Yes, that's right. We trained an ABR algorithm per DNN quality.