QoE Metrics Reporting : Support for name space identifier in metrics property

dsilhavy commented 7 months ago

Description

TS 26.512 Table 7.8.3-1 defines the metrics property as a non-empty list of metrics which shall be collected and reported. In the case of downlink media streaming and for the 3GPP scheme urn:3GPP:ns:PSS:DASH: QM10 the listed metrics shall correspond to the metrics specified in TS 26.247.

Our current client-side implementation uses the following constants to identify which metrics shall be reported by the Media Session Handler to the 5GMS Application Function. Note our implementation does not support all metrics defined in TS 26.247 yet.

object Metrics {
    const val BUFFER_LEVEL = "BufferLevel"
    const val HTTP_LIST = "HTTPList"
    const val REP_SWITCH_LIST = "RepSwitchList"
    const val MPD_INFORMATION = "MPDInformation"
}

26.512 Annex E.2.1 defines the reporting parameters for 3GPP-DASH metrics. In the provided examples the identifiers are prefixed with the name space identifier defined as : urn:3GPP:ns:PSS:DASH:QM10.

Consequently, to report all metrics of the AvgThroughput type the identifier looks like the following urn:3GPP:ns:PSS:DASH:QM10#AvgThroughput

Required changes

We need to adjust our implementation to support the name space identifier: urn:3GPP:ns:PSS:DASH:QM10.

Optional changes

For robustness reasons, we might decide to support the prefixed and non-prefixed version of the metrics strings. However, this will lead to problems once there are multiple metrics with the same name but different identifier.

Related Issue

https://github.com/5G-MAG/Standards/issues/128

dsilhavy commented 7 months ago

@rjb1000 @davidjwbbc @jordijoangimenez : Do you have an opinion on this one?

For robustness reasons, we might decide to support the prefixed and non-prefixed version of the metrics strings. However, this will lead to problems once there are multiple metrics with the same name but different identifier.

rjb1000 commented 7 months ago

Could you expand on what you mean by "robustness reasons", @dsilhavy , so we can better assess the pros and cons?

dsilhavy commented 7 months ago

Could you expand on what you mean by "robustness reasons", @dsilhavy , so we can better assess the pros and cons?

For me, it was not obvious that the metric strings like BufferLevel need to be prefixed with a name space identifier: urn:3GPP:ns:PSS:DASH:QM10#BufferLevel. Only after you pointed out the requirements defined in Annex E.2.1 I was aware of this.

So I am wondering if we should accept prefixed and non-prefixed versions of the metrics as an input, e.g.

  "metrics": [
    "HTTPList",
    "BufferLevel",
    "RepSwitchList",
    "MPDInformation"
  ]

  "metrics": [
    "urn:3GPP:ns:PSS:DASH:QM10#HTTPList",
    "urn:3GPP:ns:PSS:DASH:QM10#BufferLevel",
    "urn:3GPP:ns:PSS:DASH:QM10#RepSwitchList",
    "urn:3GPP:ns:PSS:DASH:QM10#MPDInformation"
  ]

That way we would still report the specified QoE metrics even if the configuration done via M1 did not include the urn:3GPP:ns:PSS:DASH:QM10 prefix.

Our current implementation registers an instance of a QoEMetricsReporter based on the clientMetricsReporting Configuration.scheme for each clientMetricsReportingConfiguration.metricsReportingConfigurationId. Consequently, we are operating only within the urn:3GPP:ns:PSS:DASH:QM10 scheme anyways within a single reporting instance. However, as you pointed out in https://github.com/5G-MAG/Standards/issues/128 there might be metrics related to VR with the same scheme (urn:3GPP:ns:PSS:DASH:QM10). This can potentially lead to collisions. For example, there might be a urn:3gpp:metadata:2020:VR:metrics#BufferLevel metric. At this point, we would need the prefix to identify which metric to report.

rjb1000 commented 7 months ago

My interpretation of the specification is that:

scheme determines the file format for reports.
- urn:3GPP:ns:PSS:DASH:QM10 means the XML Schema in TS 26.247.
- This was originally defined for DASH metrics reporting, but has subsequently been adopted for VR metrics and by other SA4 specifications too.
metrics identifies which metrics are to be included in the reports.
- The set of metrics is different for each SA4 usage of the report and each one has its own vocabulary.

Turning to your implementation, your mapping of a provisioned metrics reporting configuration to a "reporter" instance in the 5GMS Client sounds good.

The bit that's not quite aligned is that a given "reporter" instance might need to collect and report metrics that are not within the scope of scheme. That is never the case with DASH reporting, as currently specified, but would occur if/when the VR project wants to use 5GMS-based QoE metrics reporting mechanism. Given this, I would be inclined to make preparations in the reference implementation for that eventuality.

rjb1000 commented 7 months ago

That said, if you can come up with a convincing technical argument about why it's advantageous to have unqualified term identifiers listed in metrics, we might be able to accommodate that in the specification update.

(For the avoidance of doubt, "this is how my current implementation works" is not a convincing argument, so you will need to do better than that!)

dsilhavy commented 7 months ago

My interpretation of the specification is that:

scheme determines the file format for reports.

urn:3GPP:ns:PSS:DASH:QM10 means the XML Schema in TS 26.247.

This was originally defined for DASH metrics reporting, but has subsequently been adopted for VR metrics and by other SA4 specifications too.

metrics identifies which metrics are to be included in the reports.

The set of metrics is different for each SA4 usage of the report and each one has its own vocabulary.

Turning to your implementation, your mapping of a provisioned metrics reporting configuration to a "reporter" instance in the 5GMS Client sounds good.

The bit that's not quite aligned is that a given "reporter" instance might need to collect and report metrics that are not within the scope of scheme. That is never the case with DASH reporting, as currently specified, but would occur if/when the VR project wants to use 5GMS-based QoE metrics reporting mechanism. Given this, I would be inclined to make preparations in the reference implementation for that eventuality.

Ok understood, based on your example from https://github.com/5G-MAG/Standards/issues/128:

{
  "metricsReportingConfigurationId": "7bdfb0b0-fe4a-41ee-8d9e-cb36a16378a2",
  "serverAddresses": [
    "http://192.168.2.7:7778/3gpp-m5/v2/"
  ],
  "scheme": "urn:3GPP:ns:PSS:DASH:QM10",
  "dataNetworkName": "",
  "reportingInterval": 10,
  "samplePercentage": 100,
  "urlFilters": [],
  "samplingPeriod": 5,
  "metrics": [
    "urn:3gpp:metadata:2020:VR:metrics#RenderedViewports"
  ]
}

In this case the scheme (urn:3GPP:ns:PSS:DASH:QM10) differs from the metrics prefix (urn:3gpp:metadata:2020:VR:metrics).

For now, I will change the implementation to check for the right prefixes.

The current implementation approach would still work if we extend our reporter to support VR metrics (or any other metric reported with the scheme urn:3GPP:ns:PSS:DASH:QM10) . As an example, right now we have a class QoeMetricsReporterExoplayer that implements an interface IQoeMetricsReporter. The output is generated in urn:3GPP:ns:PSS:DASH:QM10 format. As part of creating the report, QoeMetricsReporterExoplayer or any other implementation of the IQoeMetricsReporter interface must implement the method getQoeMetricsReport. Inside that method, the individual metrics are derived and added to the final report. For example:

if (shouldReportMetric(Metrics.BUFFER_LEVEL, qoeMetricsRequest.metrics)) {
    if (bufferLevel.entries.size > 0) {
        qoeMetricsReport.bufferLevel = arrayListOf(bufferLevel)
    }
}

QoeMetricsReporterExoplayer can also be responsible for deriving VR metrics.

5G-MAG / rt-5gms-media-stream-handler