Single IPv6 addresses within device

jlurien commented 8 months ago

Problem description

When redesigning device, for IPv4 addresses it was considered just a single IP address, and subnets were only kept for applicationServer. But for IPv6 addresses we are keeping the same schema in both device and applicationServer, with same pattern that allows generic masks for subnets, without a clear differentiation.

Expected behavior Behaviour should be consistent for Ipv4 and Ipv6 and only allow IPv6 addresses in device which identify a single device.

We may need to differentiate schemas with dedicated descriptions and maybe patterns. Defining an exact pattern for Ipv6 is complex, so an option would be to remove them from spec and just rely on the RFC 5952 reference.

As this model is reused by other APIs in other WGs, they may be impacted as well.

eric-murray commented 8 months ago

Hi @jlurien

In my (limited) experience with IPv6, mobile devices tend to expect a /64 subnet, and some devices (naming no names) will randomly assign an address from that subnet no matter what specific interface identifier is allocated by the network. And the device will then go on to periodically change the interface identifier throughout the day.

So I think the choices are:

to only allow a /64 subnet (network prefix) to be specified; or
to allow single IPv6 addresses to be specified, but mandate that the resulting session applies to any address with the same network prefix as this address (for mobile devices, at least)

@RandyLevensalor It would be useful to know expectations of fixed network devices regarding the allocated network prefix and interface identifier. Do fixed network devices pick their own interface identifier?

RandyLevensalor commented 8 months ago

@RandyLevensalor It would be useful to know expectations of fixed network devices regarding the allocated network prefix and interface identifier. Do fixed network devices pick their own interface identifier?

@eric-murray I don't have a complete list of all of the permutations that are supported for IPv6 subnet allocations. For DOCSIS networks the Cable Modem Termination System (CMTS) could typically get a subnet in the /40-48 range and then allocate those to routers which it's managing. Usually each CPE router gets a /64 or larger (usually a /56 or /60) prefix delegation. Most routers will just use the first /64 in whatever it gets. Also a single /128 could be assigned to a single devices. IPv6 MAP-T is also used to map IPv4 over IPv6.

Commercial customers could have larger IPv6 subnets. There's nothing in the spec that limits this.

PON specs are layer 2 and would use CGNAT or a similar method as DOCSIS to assign an IPv6 subnet to a customer's gateway. They could also use a BNG or even an AGF with a 5G-RG to look identical to a 5G UE.

So an individual IPv6 address could be assigned to a device connected to Wi-Fi behind a CM or all devices behind a the router on a customer gateway.

jlurien commented 8 months ago

I have also limited knowledge about IPv6 assignment to devices, but it seems it is quite challenging to provide a regex pattern that covers all the cases and only those cases. I think it would be a safer approach for the specification to create separate schemas for devices and application servers and rely on descriptions, rather than providing a regex pattern that may not be totally accurate. I couldn't find an "standard" pattern for this. For implementation, there are utilities and libraries to check for IpV6 integrity that are more complex than just a regex.

jlurien commented 8 months ago

One simple solution may be to rely on the already defined format: ipv4 and format: ipv6 for the single IP addresses, instead of defining a pattern. IIRC, we considered this at the very beginning of the project but it was claimed that those formats do not cover values with masks to define subnets, but in the case of IP addresses within device is enough, and each implementation may use standard format validators. cc @eric-murray @RandyLevensalor

sfnuser commented 7 months ago

One simple solution may be to rely on the already defined format: ipv4 and format: ipv6 for the single IP addresses, instead of defining a pattern. IIRC, we considered this at the very beginning of the project but it was claimed that those formats do not cover values with masks to define subnets, but in the case of IP addresses within device is enough, and each implementation may use standard format validators. cc @eric-murray @RandyLevensalor

I agree and prefer the format: ipv6 for single IP address.

eric-murray commented 7 months ago

Yes, I agree that specifying a single IPv6 address for the device using format: ipv6 alone is the most unambiguous when it comes to supported formats. But how to interpret this?

For a mobile device, I think this should be interpreted as "any flow to/from any IPv6 address within the same network prefix as the specified IPv6 address". For otherwise, the session could not be established until the address chosen by the device was known and in use, and then the session would no longer apply as soon as the device randomised its interface identifier.

But for fixed network devices, is it possible that they will be using multiple IPv6 addresses with the same network prefix, but want the QoS profile only to apply to the specified IPv6 address and not any other?

Maybe an additional separate field is required for the device subnet to which the QoS flow applies.

jlurien commented 7 months ago

@eric-murray In the context of device object, IP addresses are intended to identify the UE. This object is shared with other CAMARA APIs, such as DeviceLocation, DeviceStatus... Characterization of flows for QoD may require dedicated properties and descriptions, but I would favor a generic schema for Device identification which can be reused in all CAMARA APIS

eric-murray commented 7 months ago

@jlurien If it is understood that the single IPv6 address identifies the device, and therefore that any other IPv6 addresses the device uses (or goes on to use) should also be included in the QoS session, then I'm happy to restrict the definition to a single IPv6 address.

jlurien commented 7 months ago

To me it is important to remark the differentiation between device identification and QoD flows. For QoD specifically we may need to discuss further the proper way to define flows and all its implications.

eric-murray commented 7 months ago

At the moment, both the device identification and device flow specification are bound together in a common device field. Is the proposal being discussed in this issue only to replace the current device IPv6 pattern with format: ipv6 (in which case, how should a single IPv6 address be interpreted?), or to properly separate device and flow identification so that device can only be considered to identify the device?

jlurien commented 7 months ago

The motivation of this issue is to have a common model for device across all CAMARA APIs which reuse it (indeed the proposal was first discussed in DeviceLocation). As the model in other APIs was copied from QoD, I started the discussion here, but it may be something worth to be discussed in Commonalities.

The only formats allowed in device would be those considered in JSON-schema:

IP Addresses

"ipv4": IPv4 address, according to dotted-quad ABNF syntax as defined in RFC 2373, section 3.2.
"ipv6": IPv6 address, as defined in RFC 2373, section 2.2.

The considerations about flows should be the same as if the device would be identified by other means, with a phone number for example.

eric-murray commented 7 months ago

In that case, we need to consider the issues of both device and device flow specification jointly. The existing device property can identify the device and a new optional property (perhaps repurposing the existing devicePorts to something like deviceFlows) to be specified if the flows to which the session should apply are only a subset of all flows to/from the device.

jlurien commented 7 months ago

In that case, we need to consider the issues of both device and device flow specification jointly. The existing device property can identify the device and a new optional property (perhaps repurposing the existing devicePorts to something like deviceFlows) to be specified if the flows to which the session should apply are only a subset of all flows to/from the device.

That makes sense to me, possibly redefining devicePorts.

hdamker commented 7 months ago

@jlurien @eric-murray Just a reminder of the (closed) issue https://github.com/camaraproject/QualityOnDemand/issues/146 ("Format 'ipv4' is more restrictive than the pattern in Data Type 'Ipv4Address'"). I don't think that the restriction of the JSON schema has an impact on the definition of device. But for applicationServer we should keep the option to define address ranges.

jlurien commented 7 months ago

@jlurien @eric-murray Just a reminder of the (closed) issue #146 ("Format 'ipv4' is more restrictive than the pattern in Data Type 'Ipv4Address'"). I don't think that the restriction of the JSON schema has an impact on the definition of device. But for applicationServer we should keep the option to define address ranges.

Yes, the intention is just to apply it to device, so it can be reused here and in the rest of APIs. I will propose a PR for this

hdamker commented 7 months ago

@jlurien

In that case, we need to consider the issues of both device and device flow specification jointly. The existing device property can identify the device and a new optional property (perhaps repurposing the existing devicePorts to something like deviceFlows) to be specified if the flows to which the session should apply are only a subset of all flows to/from the device.

I think that @eric-murray is right here - we can't do the change in device without updating also the way how the flow is specified in the QoS session. With the change of device there is no longer an option to specify a flow description which includes a range of of IPv6 addresses out of the prefix used by the device.

jlurien commented 7 months ago

What I don't get about the problem with IPv6 and ranges within device, is why do we have to consider IPV6 ranges and subnet in the case of IPv6 and not for IPV4, where we are already only allowing Single IPv4 addresses? I have limited experience with IPv6, sorry. And in case that a client is only providing a phone_number, which considerations about IPv6 ranges should the implementation assume? It is clear to me that we should differentiate device identification from flow description, and I understand that the second one may require further work. Can anyone suggest a proposal with flow description which is not coupled with device identification?

hdamker commented 7 months ago

What I don't get about the problem with IPv6 and ranges within device, is why do we have to consider IPV6 ranges and subnet in the case of IPv6 and not for IPV4, where we are already only allowing Single IPv4 addresses?

@jlurien The difference is that one device gets within the mobile network typically only one a single IPv4 address assigned - therefore it is totally ok to restrict to a single IPv4 address. But in IPv6 a device gets always a prefix (e.g. 64), so a device has an /64 address range from which it can pick more or less randomly (e.g. for privacy reasons) the actual used IP address. Any single IPv6 address out of this range can be used to identify the device, but for the flow we need either a separate definition or - in absence of such definition - an interpretation as proposed by Eric above:

For a mobile device, I think this should be interpreted as "any flow to/from any IPv6 address within the same network prefix as the specified IPv6 address". For otherwise, the session could not be established until the address chosen by the device was known and in use, and then the session would no longer apply as soon as the device randomised its interface identifier.

jlurien commented 7 months ago

What I don't get about the problem with IPv6 and ranges within device, is why do we have to consider IPV6 ranges and subnet in the case of IPv6 and not for IPV4, where we are already only allowing Single IPv4 addresses?

@jlurien The difference is that one device gets within the mobile network typically only one a single IPv4 address assigned - therefore it is totally ok to restrict to a single IPv4 address. But in IPv6 a device gets always a prefix (e.g. 64), so a device has an /64 address range from which it can pick more or less randomly (e.g. for privacy reasons) the actual used IP address. Any single IPv6 address out of this range can be used to identify the device, but for the flow we need either a separate definition or - in absence of such definition - an interpretation as proposed by Eric above:

For a mobile device, I think this should be interpreted as "any flow to/from any IPv6 address within the same network prefix as the specified IPv6 address". For otherwise, the session could not be established until the address chosen by the device was known and in use, and then the session would no longer apply as soon as the device randomised its interface identifier.

Thanks, I was having a look into this. On one hand, UEs are assigned a /64 subnet, but they should choose an specific IPv6 within that range to communicate, and a client trying to identify an specific device will see the single IPV6 address. Hypothetically we could allow a /64 to be used for identification purposes but not any arbitrary mask.

The interpretation provided by Eric makes sense, but that interpretation has to be assumed already when a phone number is used to identify a device, so for flow description, we could generalise it as "any flow to/from any address assigned to the device".

Moreover, looking into the current specification

    Ipv6Address:
      type: string
      allOf:
        - pattern: '^((:|(0?|([1-9a-f][0-9a-f]{0,3}))):)((0?|([1-9a-f][0-9a-f]{0,3})):){0,6}(:|(0?|([1-9a-f][0-9a-f]{0,3})))(\/(([0-9])|([0-9]{2})|(1[0-1][0-9])|(12[0-8])))?$'
        - pattern: '^((([^:]+:){7}([^:]+))|((([^:]+:)*[^:]+)?::(([^:]+:)*[^:]+)?))(\/.+)?$'
      example: "2001:db8:85a3:8d3:1319:8a2e:370:7344"
      description: |
        IPv6 address, following IETF 5952 format, may be specified in form <address/mask> as:
          - address - The /128 subnet is optional for single addresses:
            - 2001:db8:85a3:8d3:1319:8a2e:370:7344
            - 2001:db8:85a3:8d3:1319:8a2e:370:7344/128
          - address/mask - an IP v6 number with a mask:
            - 2001:db8:85a3:8d3::0/64
            - 2001:db8:85a3:8d3::/64

This has 2 problems for implementation:

allOf asks for both patterns to be supported simultaneously, which is not possible. It should be oneOf or anyOf to implement a logical XOR/OR.
In the description we ask for RFC 5952, but this RFC proposes a recommendation to write IPv6 in a human-friendly way, and it explicitly mentions that "all implementations must accept and be able to handle any legitimate RFC 4291 format."

I think we should review all of this device vs flow specification carefully as a priority for the next release. For this v0.10, we should try to find a compromise to fix the current problems. If both format: ipv6 or the current set of pattern are not totally correct, we may just rely on the description, explaining what is to be supported, for example:

      description: |
        IPv6 address, following RFC 4291 format, which may be specified as:
          - a single IPV6 address - The /128 subnet is optional for single addresses:
            - 2001:db8:85a3:8d3:1319:8a2e:370:7344
            - 2001:db8:85a3:8d3:1319:8a2e:370:7344/128
          - a IPv6 address with a /64 mask:
            - 2001:db8:85a3:8d3::0/64
            - 2001:db8:85a3:8d3::/64

hdamker commented 7 months ago

Reading the comment of @jlurien I got that we have to do something now (for v0.10) regarding the IPv6 address format.

Two options are discussed:

format: ipv6 (current PR)
- it will restrict accepted format to a single IPv6 address (which is sufficient to identify the device, also fine if the API customer knows only the prefix for some reason, can use just the first one out if of the range, e.g. ::0)
- within device it will not restrict the flows to/from the identified device. The logic is (which we might want to add to the documentation): "any flow to/from any address assigned to the device to/from the applicationServer addresses (if not restricted further by optional parameters like devicePorts and applicationServerPorts)" ... i have added the italic part, that will be still valid if we add further parameters which will define the flow more precise in later releases.
no format and pattern for IPv6 addresses beyond string, just with the description as proposed above by @jlurien
- I'm not in favor of this option, it would postpone a potential breaking change to a later release

Can we decide short-term on one of the options?

@eric-murray any thoughts about it?

hdamker commented 7 months ago

This has 2 problems for implementation:

allOf asks for both patterns to be supported simultaneously, which is not possible. It should be oneOf or anyOf to implement a logical XOR/OR.

In the description we ask for RFC 5952, but this RFC proposes a recommendation to write IPv6 in a human-friendly way, and it explicitly mentions that "all implementations must accept and be able to handle any legitimate RFC 4291 format."

Got the point, that this patterns are an implementation challenge. Within your PR you haven't eliminated the schema, as it is stilled used for applicationServer (where an address range makes sense, to allow e.g. a cluster). What would be your recommendation here? Just use string and the description?

jlurien commented 7 months ago

This has 2 problems for implementation:

allOf asks for both patterns to be supported simultaneously, which is not possible. It should be oneOf or anyOf to implement a logical XOR/OR.

In the description we ask for RFC 5952, but this RFC proposes a recommendation to write IPv6 in a human-friendly way, and it explicitly mentions that "all implementations must accept and be able to handle any legitimate RFC 4291 format."

Got the point, that this patterns are an implementation challenge. Within your PR you haven't eliminated the schema, as it is stilled used for applicationServer (where an address range makes sense, to allow e.g. a cluster). What would be your recommendation here? Just use string and the description?

As we potentiallly need support for masks and format: ipv6 is not enough, I would suggest to remove the pattern and document the string with description.

hdamker commented 7 months ago

As we potentiallly need support for masks and format: ipv6 is not enough, I would suggest to remove the pattern and document the string with description.

@jlurien Would you extend the PR #237 accordingly?

eric-murray commented 7 months ago

@eric-murray any thoughts about it?

For the device, I would favour option 1 (format: ipv6), with an associated description along the following lines:

"Any single IPv6 address from within the subnet allocated to the device. The session shall apply to all IP flows between the device subnet and the specified application server, unless further restricted by the optional parameters devicePorts or applicationServerPorts."

But I would check this interpretation with @sfnuser, as I recall that SFN were looking at scenarios where different QoS sessions would be applied to different flows between the device and application server. Are we breaking something here, or can their scenario be accommodated entirely by specifying different port restrictions for each flow?

An additional question is whether we continue to allow both an IPv4 and IPv6 address to be specified, Currently, I would interpret specifying both to mean that the session should apply to both IPv4 and IPv6 flows, assuming that IPv4 and IPv6 addresses are also specified for the application server. But given this assumption, it would be equally valid to say that the session would apply between the device and the specified application server addresses (whether IPv4, IPv6 or both) irrespective of how the device is identified, in which case a single IP address (or other identifier) for the device would be sufficient.

eric-murray commented 7 months ago

allOf asks for both patterns to be supported simultaneously, which is not possible. It should be oneOf or anyOf to implement a logical XOR/OR.

It is entirely possible for a string to simultaneously match two different patterns. Implementing this using code generators is a different problem. of course.

jlurien commented 7 months ago

allOf asks for both patterns to be supported simultaneously, which is not possible. It should be oneOf or anyOf to implement a logical XOR/OR.

It is entirely possible for a string to simultaneously match two different patterns. Implementing this using code generators is a different problem. of course.

Yes, you are right. Strings in the "intersection" of both expressions would be compliant. The problem with current pattern is that some valid IPv6 addresses are not compatible with both, such as "2001:0db8:85a3:0000:0000:8a2e:0370:7334", which is not valid for the first Regex. This may be related to not be according to the guidelines of RFC 5952

jlurien commented 7 months ago

@eric-murray any thoughts about it?

For the device, I would favour option 1 (format: ipv6), with an associated description along the following lines:

"Any single IPv6 address from within the subnet allocated to the device. The session shall apply to all IP flows between the device subnet and the specified application server, unless further restricted by the optional parameters devicePorts or applicationServerPorts."

I will modify the PR. Please review it, specially the descriptions and help me with the wording.

But I would check this interpretation with @sfnuser, as I recall that SFN were looking at scenarios where different QoS sessions would be applied to different flows between the device and application server. Are we breaking something here, or can their scenario be accommodated entirely by specifying different port restrictions for each flow?

An additional question is whether we continue to allow both an IPv4 and IPv6 address to be specified, Currently, I would interpret specifying both to mean that the session should apply to both IPv4 and IPv6 flows, assuming that IPv4 and IPv6 addresses are also specified for the application server. But given this assumption, it would be equally valid to say that the session would apply between the device and the specified application server addresses (whether IPv4, IPv6 or both) irrespective of how the device is identified, in which case a single IP address (or other identifier) for the device would be sufficient.vice

To me, in order to identify a device it makes more sense to allow just a single IP address (v4 or v6). But to express that we allow only one of the IPv4 or IPv6 addresses + possibly other identifiers (phoneNumber, externalId), complicates the schema. I remember some discussion months ago with the use of oneOf. We probably have to review this along with proper flow identification in the coming release.

jlurien commented 7 months ago

PR #237 updated

sfnuser commented 7 months ago

But I would check this interpretation with @sfnuser, as I recall that SFN were looking at scenarios where different QoS sessions would be applied to different flows between the device and application server. Are we breaking something here, or can their scenario be accommodated entirely by specifying different port restrictions for each flow?

@eric-murray: Thanks for checking. We are OK with the proposed option 1 and the associated description. As described in the RFC 6459, with a single IPv6 address also we can identify the subnet. The individual IP flows can further be identified using the device and application server ports. So the proposal sounds good.

jlurien commented 7 months ago

I will open an issue in Commonalities to inform other WGs of this change, as the model for Device was copied from QoD and replicated in many other APIs.

bnsmith240 commented 6 months ago

I seem to recall that there was a change made in the RFCs to allow a certain use case for IPv6 in LTE. The issue was the /64 being added and had something to do with allowing this to route to an address space behind.... hence allowing a more efficient use of IPv6. Not sure if this was propagated to 5G

camaraproject / QualityOnDemand

Single IPv6 addresses within device #230

IP Addresses