search

osgi / bugzilla-archive

Archive of OSGi Alliance Specification Bugzilla bugs. The Specification Bugzilla system was decommissioned with the move to GitHub. The issues in this repository are imported from the Specification Bugzilla system for archival purposes.
0 stars 1 forks source link

the mapping table, which RFC149 is going to define #1564

Closed bjhargrave closed 13 years ago

bjhargrave commented 14 years ago

Original bug ID: BZ#1672 From: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp> Reported version: R4 V4.2

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

The attached file represents the mapping table, whjch RFC149 is going to define.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 136 Mapping table b/w MIME types and FORMAT

Attached file: MimeTypeMapping_20100615.ods (application/vnd.oasis.opendocument.spreadsheet, 10303 bytes) Description: Mapping table b/w MIME types and FORMAT

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 138 Mapping table b/w dataType in TR069 and Format in DmtAdmin

I had completely revised the mapping table.

  1. When PA receives SetParameterValues RPC, PA needs to set the value to the node in the DMTree. How to set it ?

  2. When PA receives GetParameterValues RPC, PA needs to get the value from the DMTree and set the dataType in GetParameterValuesResponse RPC. How to choose dataType ?

To be determined are in yellow background:

In "For SetParameterValues RPC" sheet

A: What Should PA do in case of ERROR ? for example, the Format of the leaf node is FORMAT_BASE64 but the dataType specified in TR-069 XML is boolean. It is assumed as ERROR.

B: For example, Format is FORMAT_UNSIGNED_INTEGER and the dataType is int. "If the value is negative, ERROR. Otherwise, cast FORMAT_UNSIGNED_INT." Is is OK ? If GetParameterValues is called for the node after setting the value to DM Tree, the GetParameterValuesResponse will indicate the value is "unsignedInt".

C: If the Format of the leaf node is FORMAT_UNSIGNED_LONG, dataType to be specified in SetParameterValues is unsignedLong, What to do if the value specified in the SetParameterValues is greater than Long.MAX_VALUE ?

D: is the mapping FORMAT_REFERENCE and string correct ?

In "For GetParameterValues RPC" sheet

Manys are uncertain. See the yellow background cells.

Please correct the sheet and post it.

Attached file: DataTypeMapping_20100831.zip (application/zip, 134475 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

If we agree with my proposal in https://www.osgi.org/members/bugzilla/show_bug.cgi?id=1728#c0 "C" will not be a problem. Please ignore it.

C: If the Format of the leaf node is FORMAT_UNSIGNED_LONG, dataType to be specified in SetParameterValues is unsignedLong, What to do if the value specified in the SetParameterValues is greater than Long.MAX_VALUE ?

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

Created attachment 139 Comments about the mapping table b/w dataType in TR069 and Format in DmtAdmin

Attached file: DataTypeMapping_20100902_CommentedByProSyst.ods (application/vnd.oasis.opendocument.spreadsheet, 19610 bytes) Description: Comments about the mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

Please check our comments in the attached doc file (DataTypeMapping_20100902_CommentedByProSyst.ods). We can distinguish these issues to:

  1. For simplicity, when there is a mapping like FORMAT_LOG <-> long and FORMAT_INT <-> int, we prefer to remove the mapping like FORMAT_LONG <-> int.
  2. When TR-069 type can be mapped to more than one Dmt format, we must specify priorities. For example, if TR-069 string can be mapped to: FORMAT_STRING, FORMAT_RAW_STRING and FORMAT_XML. DMT node can support all formats i.e. the node value can be one of the specified. PA must know the preferred format.
  3. Minor notes, which can be checked in the attached document.
bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Created attachment 140 Updated version of GetParameterValues spreadsheet

Added a "WFL comment" column. No other changes.

Attached file: DataTypeMapping_20100831_WL.xls (application/vnd.ms-excel, 45571 bytes) Description: Updated version of GetParameterValues spreadsheet

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Please see "Updated version of GetParameterValues spreadsheet. This provides answers to all the questions but still leaves open issues around raw binary and raw string (both these have an associated format name that could be used in deciding to which TR-069 type to map).

For the SetParameterValues spreadsheet, we think that the mapping needs just to be the inverse of the GetParameterValues mapping, i.e. all are 1-1 mappings, e.g. int -> FORMAT_INTEGER etc. We don't think that any conversions between numeric types are needed.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 141 Mapping table b/w dataType in TR069 and Format in DmtAdmin

Thank you for your comments, Evgeni and William ! Basically I had updated the sheets based on the comments.

First, I would like to remark that the following discussion is based on my proposal in
https://www.osgi.org/members/bugzilla/show_bug.cgi?id=1728#c0

Now, I listed up issues to be determined.

========================================================== [GetParameterValues RPCs]

For FORMAT_BINARY

William, you wrote "have to choose one". Why do you prefer hexBinary to base64 ?

For FORMAT_RAW_BINARY

William, why you choose base64 instead of hexBinary ? In addition, I don't understand your comment: " (or specify which format names map to base64 and hexBinary)" Let me ask basics in TR069. Can arbitrary type be specified in ParameterValueStruct[] ?

If we decide to choose "base64",
PA cam get the original format of the RAW_BINARY by DmtData#getFormatName(). PA tries to transcode it from the original format into base64. If succeeded, transcoded byte[] will be used as base64. Otherwise (if failed), what PA to do ?

I had checked TR-069. The fault for GetParameterValues I found is only 9003 Invalid arguments. Is it appropriate fault code to be returned to ACS ?

For FORMAT_RAW_STRING

I investigated the DmtAdminSpec again. Just string seems enough.

========================================================== [SetParameterValues RPCs]

Q1. What PA should do if the ERROR in the spreadsheet (if the data type specified in RPC cannot be mapped to the Format of the node in the DM Tree) ?

Evgeni wrote "PA can return a fault message to the ACS with a code 9003/9007." I had checked TR-069. There are several candites 9003 Invalid arguments 9006 Invalid parameter type (associated with SetParameterValues) 9007 Invalid parameter value (associated with SetParameterValues)

William or John, could you tell which one is appropriate ?

The last issue to be discussed is what to do for row of "string".

Evgeni wrote in Comment#5,

  1. When TR-069 type can be mapped to more than one Dmt format, we must specify priorities. For example, if TR-069 string can be mapped to: FORMAT_STRING, FORMAT_RAW_STRING and FORMAT_XML. DMT node can support all formats i.e. the node value can be one of the specified. PA must know the preferred format.

I don't understand what it means.

There are two cases: the target node has already existed in DM Tree or not.

Case1: The node specified in SetParameterValues RPC has already existed in DM Tree.

The PA can get DmtData of the node and get the Format of the node and If the Format is any of the following GROUP, PA can set the node value. GROUP: FORMAT_STRING, FORMAT_XML, FORMAT_DATE, FORMAT_TIME,FORMAT_UNSINGED_LONG, FORMAT_UNSIGNED_INT,FORMAT_DATETIME, and FORMAT_REFERENCE.

Else if the Format is FORMAT_RAW_STRING, I think, it is an error condition because PA cannot get the info of the formatName to call DmtData(formatName,data).

Case2: The node specified in SetParameterValues RPC does not exist.

The PA tries to get MetaNode of the node. If failed, it is error condition. (fault code among 9003, 9006, or 9007). It succeeded, PA gets the format from the MetaNode. If the Format is any of the GROUP above, PA can create the node value. Else if the Format is FORMAT_RAW_STRING, I think, it is an error condition because of the same reason.

I think the priorities has no relationships with them.

On the other hand, I have a problem to be discussed:

See G18 cell, set "string" dataType to the node of FORMAT_UNSIGNED_LONG. Do we need it ?

It would break the one to one relationships, as William requested because GetParameterValues against the node of FORMAT_UNSIGNED_LONG will return unsignedLong dataType. The same is for G16, "string" and "FORMAT_UNSINGED_INTEGER".

Attached file: DataTypeMapping_20100903.zip (application/x-zip-compressed, 140520 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

Evgeni wrote in Comment#5,

  1. When TR-069 type can be mapped to more than one Dmt format, we must specify priorities. For example, if TR-069 string can be mapped to: FORMAT_STRING, FORMAT_RAW_STRING and FORMAT_XML. DMT node can support all formats i.e. the node value can be one of the specified. PA must know the preferred format.

I don't understand what it means.

There are two cases: the target node has already existed in DM Tree or not.

Case1: The node specified in SetParameterValues RPC has already existed in DM Tree.

The PA can get DmtData of the node and get the Format of the node and If the Format is any of the following GROUP, PA can set the node value. GROUP: FORMAT_STRING, FORMAT_XML, FORMAT_DATE, FORMAT_TIME,FORMAT_UNSINGED_LONG, FORMAT_UNSIGNED_INT,FORMAT_DATETIME, and FORMAT_REFERENCE.

Else if the Format is FORMAT_RAW_STRING, I think, it is an error condition because PA cannot get the info of the formatName to call DmtData(formatName,data).

Case2: The node specified in SetParameterValues RPC does not exist.

The PA tries to get MetaNode of the node. If failed, it is error condition. (fault code among 9003, 9006, or 9007). It succeeded, PA gets the format from the MetaNode. Exactly, the problem is here. The javadoc of info.dmtree.MetaNode.getFormat() says: " If there are multiple formats allowed for the node then the format constants are OR-ed. " If the node format is FORMAT_DATE | FORMAT_DATETIME, which one can PA use? The format priorities can say: FORMAT_DATA precedes FORMAT_DATETIME and PA will try with FORMAT_DATE. If the Format is any of the GROUP above, PA can create the node value. Else if the Format is FORMAT_RAW_STRING, I think, it is an error condition because of the same reason.

I think the priorities has no relationships with them. On the other hand, I have a problem to be discussed:

See G18 cell, set "string" dataType to the node of FORMAT_UNSIGNED_LONG. Do we need it ?

It would break the one to one relationships, as William requested because GetParameterValues against the node of FORMAT_UNSIGNED_LONG will return unsignedLong dataType. The same is for G16, "string" and "FORMAT_UNSINGED_INTEGER".

We prefer to return an error in this case.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Exactly, the problem is here. The javadoc of info.dmtree.MetaNode.getFormat() says: " If there are multiple formats allowed for the node then the format constants are OR-ed. " If the node format is FORMAT_DATE | FORMAT_DATETIME, which one can PA use? The format priorities can say: FORMAT_DATA precedes FORMAT_DATETIME and PA will try with FORMAT_DATE.

Now I understand. Thank you for the explanation. However, I cannot imagine any use case for multiple format for a node. Do you have any idea ?

On the other hand, I have a problem to be discussed:

See G18 cell, set "string" dataType to the node of FORMAT_UNSIGNED_LONG. Do we need it ?

It would break the one to one relationships, as William requested because GetParameterValues against the node of FORMAT_UNSIGNED_LONG will return unsignedLong dataType. The same is for G16, "string" and "FORMAT_UNSINGED_INTEGER".

We prefer to return an error in this case.

OK. William, do you agree ?

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

Exactly, the problem is here. The javadoc of info.dmtree.MetaNode.getFormat() says: " If there are multiple formats allowed for the node then the format constants are OR-ed. " If the node format is FORMAT_DATE | FORMAT_DATETIME, which one can PA use? The format priorities can say: FORMAT_DATA precedes FORMAT_DATETIME and PA will try with FORMAT_DATE.

Now I understand. Thank you for the explanation. However, I cannot imagine any use case for multiple format for a node. Do you have any idea ? In the current state of RMT, only ServiceState//Properties//Values/ node has multiple formats. That node is not a problem, because the possible formats are:

  • String
  • bin
  • int
  • bool
  • float and all of them are well mapped.
bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Oh dear there are a lot of things to comment on, starting with comment BZ#8

  1. the 2010-09-03 version of the spreadsheets:
  1. Ikuo questions in comment BZ#8:

For FORMAT_BINARY William, you wrote "have to choose one". Why do you prefer hexBinary to base64?

  • For no strong reason, just feeling that hexBinary is better for data that might make some sense when read byte by byte (e.g. TLV) and base64 is better for data that makes no sense byte by byte (e.g. compressed data). As I said, we have to pick one.

For FORMAT_RAW_BINARY William, why you choose base64 instead of hexBinary?

  • Again for no strong reason, just a feeling that "raw" suggested more the "makes no sense byte by byte" case.

In addition, I don't understand your comment: " (or specify which format names map to base64 and hexBinary)"

  • Maybe my ignorance and I misunderstood, but we have extra information here, i.e. the format name, so if the format names are "well known" (I don't know whether they are) we could specify whether a given format name should be represented as base64 or hexBinary.

Let me ask basics in TR069. Can arbitrary type be specified in ParameterValueStruct[] ?

  • No, only supported TR-069 primitive types (a subset of SOAP types); to add additional primitive types requires a TR-106 revision (and DM Schema revision).

If we decide to choose "base64", Otherwise (if [convert to base64] failed), what PA to do ?

  • Why would convert to base64 fail (or convert to hexBinary)? It's a well-defined mapping that can encode any binary data.

I had checked TR-069. The fault for GetParameterValues I found is only 9003 Invalid arguments. Is it appropriate fault code to be returned to ACS?

  • You mean if conversion from internal representation to interchange representation fails? I think if this is a system-level failure, e.g. "out of memory", 9004 (resources exceeded) might be more appropriate. 9003 should indicate that the ACS made a mistake.

SetParameterValues RPC ... What PA should do if the ERROR in the spreadsheet (if the data type specified in RPC cannot be mapped to the Format of the node in the DM Tree)?

  • Now that we have 1-1 mapping, this can probably occur only for the various string mappings, is that right, e.g. string to FORMAT_DATE? I think 9007 (invalid parameter value).
  1. Ikuo questions in comment BZ#10:

See G18 cell, set "string" dataType to the node of FORMAT_UNSIGNED_LONG. Do we need it? ... We prefer to return an error in this case. ... OK. William, do you agree?

  • yes
  1. Evgeni's comment BZ#11:

In the current state of RMT, only ServiceState//Properties//Values/ node has multiple formats. That node is not a problem, because the possible formats are {string, bin, int, bool, float} and all of them are well mapped.

  • I assume that you have listed these in priority order, i.e. string has highest priority?
  • Who specifies the priorities and are they are fixed (the same for all nodes)?
  • I hope that this means that on SetParameterValues, all values would be stored as string, and on GetParameterValues, values would be converted to string?
  • If not, and the conversion is a function of actual value, I see ambiguities because the value spaces overlap, e.g. "1" is a valid string, int, bool(?) and float(?).
  • Please clarify.
bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 142 Mapping table b/w dataType in TR069 and Format in DmtAdmin

Oh dear there are a lot of things to comment on, starting with comment BZ#8

  1. the 2010-09-03 version of the spreadsheets:
  • GetParameterValues: looks good apart from the open issues that you have raised (see below).

Got it. I'll comments on them below.

  • SetParameterValues: looks good apart from no FORMAT_BINARY mapping and why do we have mappings from TR-069 string to some (not all) numeric types (we would prefer not to allow conversion from string to numeric)?

So there are two open issues:

Issue-A: What to do if SetParameterValues is called against a leaf node of FORMAT_BINARY ?

Do you have any idea, William ?

Issue-B: G16 and G18 Issues, which is discussed below.

  1. Ikuo questions in comment BZ#8:

For FORMAT_BINARY William, you wrote "have to choose one". Why do you prefer hexBinary to base64?

  • For no strong reason, just feeling that hexBinary is better for data that might make some sense when read byte by byte (e.g. TLV) and base64 is better for data that makes no sense byte by byte (e.g. compressed data). As I said, we have to pick one.

I got it. That sounds reasonable.

For FORMAT_RAW_BINARY William, why you choose base64 instead of hexBinary?

  • Again for no strong reason, just a feeling that "raw" suggested more the "makes no sense byte by byte" case.

I got it. That sounds reasonable.

In addition, I don't understand your comment: " (or specify which format names map to base64 and hexBinary)"

  • Maybe my ignorance and I misunderstood, but we have extra information here, i.e. the format name, so if the format names are "well known" (I don't know whether they are) we could specify whether a given format name should be represented as base64 or hexBinary.

I got it. I want to say just "base64" in D12.

Let me ask basics in TR069. Can arbitrary type be specified in ParameterValueStruct[] ?

  • No, only supported TR-069 primitive types (a subset of SOAP types); to add additional primitive types requires a TR-106 revision (and DM Schema revision).

Thanks. I got it.

If we decide to choose "base64", Otherwise (if [convert to base64] failed), what PA to do ?

  • Why would convert to base64 fail (or convert to hexBinary)? It's a well-defined mapping that can encode any binary data.

The original encoding format, which can be retrieved by getFormatName() might be unknown for the PA. In that case, the PA cannot convert to base64, can it ?

I had checked TR-069. The fault for GetParameterValues I found is only 9003 Invalid arguments. Is it appropriate fault code to be returned to ACS?

  • You mean if conversion from internal representation to interchange representation fails? I think if this is a system-level failure, e.g. "out of memory", 9004 (resources exceeded) might be more appropriate. 9003 should indicate that the ACS made a mistake.

Can 9004 be used for GetParameterValues RFC

SetParameterValues RPC ... What PA should do if the ERROR in the spreadsheet (if the data type specified in RPC cannot be mapped to the Format of the node in the DM Tree)?

  • Now that we have 1-1 mapping, this can probably occur only for the various string mappings, is that right, e.g. string to FORMAT_DATE?

No, there are a lot of cases, which are specified "ERROR" in the sheet. For example, see G4 cell. If the Format of the leaf node in DM Tree is FORMAT_BASE64 but SetParameterValues RPC asks to set value as "string", PA should recognize it is an ERROR. Then what fault code to be delivered to ACS should be used ?

I think 9007 (invalid parameter value).

Should 9007 be used for all ERROR in the spreadsheet ? (IMO, it is OK).

  1. Ikuo questions in comment BZ#10:

See G18 cell, set "string" dataType to the node of FORMAT_UNSIGNED_LONG. Do we need it? ... We prefer to return an error in this case. ... OK. William, do you agree?

  • yes

Now all of us agreed to change G16, G18 and G19 to ERROR.

I had updated the sheet. Modified part from the latest one is written in red.

Attached file: DataTypeMapping_20100904.zip (application/x-zip-compressed, 140662 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

  1. Evgeni's comment BZ#11:

In the current state of RMT, only ServiceState//Properties//Values/ node has multiple formats. That node is not a problem, because the possible formats are {string, bin, int, bool, float} and all of them are well mapped.

  • I assume that you have listed these in priority order, i.e. string has highest priority?
  • Who specifies the priorities and are they are fixed (the same for all nodes)?
  • I hope that this means that on SetParameterValues, all values would be stored as string, and on GetParameterValues, values would be converted to string?
  • If not, and the conversion is a function of actual value, I see ambiguities because the value spaces overlap, e.g. "1" is a valid string, int, bool(?) and float(?).
  • Please clarify.

I also want Evgeni to clarify.

SetParameterValues RPC specifies the value to be set and its dataType defined in TR069. I expect the following situation: If string dataType is specified in SetParameterValues RPC, it can be set to leaf node whose Format is any of GROUP (which I listed up in comment#8) or FORMAT_NULL. All of them will be return as string when GetParameterValues RFC is called. We need to define some algorithm (or precedence) how PA choose the Format to Construct DmtData If the Format of the node in DM Tree has multiple FORMATs.

The following algoritm is an idea:

  1. If the value to be set is empty string, then FORMAT_NULL will be chosen.
  2. Else if DmtData(String value, int format) with any of FORMAT_DATE, FORMAT_TIME, FORMAT_DATETIME, FORMAT_UNSIGNED_LONG, FORMAT_UNSIGNED_INT, and FORMAT_REFERENCE does not throw IllegalArgumentException, the format will be chosen.
  3. Else There is no good way to choose one among FORMAT_STRING and FORMAT_XML. So we need to have precedence. i.e. (high precendence) FORMAT_STRING, FORMAT_XML (low precedence) (There is no specific reason of this precedence.)

How about the idea ?

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Replies to questions from Ikuo in comment BZ#13:

SetParameterValues Issue-A: What to do if SetParameterValues is called against a leaf node of FORMAT_BINARY ? Do you have any idea, William ?

Given that we have agreed that FORMAT_BINARY is converted to hexBinary when reading, I think that we should allow only hexBinary when writing. If we allow base64 when writing, we could have a situation where the ACS uses base64 to set the value and then reads it back and gets hexBinary. I don't think we want that.

WL: GetParameterValues Why would convert to base64 fail (or convert to hexBinary)? It's a well-defined mapping that can encode any binary data. IY: The original encoding format, which can be retrieved by getFormatName() might be unknown for the PA. In that case, the PA cannot convert to base64, can it ?

This is the FORMAT_RAW_BINARY case, right? My original suggestion was that the PA might use the format name in order to choose between hexBinary and base64, but I thought that your proposal (with which I am fine) was that the PA would ignore the format name here. So I don't see any problem with conversion TO base64 (for GetParameterValues).

Conversion FROM base64 (for SetParameterValues) could definitely fail though. The appropriate failure code here would be 9007 "invalid parameter value".

Note that this case can only occur for non-BBF data models, because all BBF primitive data types have explicit formats. The ACS will need out-of-band knowledge of how to format (and interpret) such values.

Can 9004 be used for GetParameterValues RPC ?

Yes.

WL: SetParameterValues ... Now that we have 1-1 mapping, this can probably occur only for the various string mappings, is that right, e.g. string to FORMAT_DATE?
IY: No, there are a lot of cases ... what fault code to be delivered to ACS should be used ? IY: Should 9007 be used for all ERROR in the spreadsheet ? (IMO, it is OK).

Sorry, you are of course correct. Yes, think that 9007 is OK for all the ERROR cases.

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

SetParameterValues RPC specifies the value to be set and its dataType defined in TR069. I expect the following situation: If string dataType is specified in SetParameterValues RPC, it can be set to leaf node whose Format is any of GROUP (which I listed up in comment#8) or FORMAT_NULL. All of them will be return as string when GetParameterValues RFC is called.

Referring to the most version of the spreadsheet, the formats to which a TR-069 can be converted are: DATE, NULL, STRING, TIME, XML, REFERENCE, NODE (where the last one applies only to lists), i.e. we no longer have the string to numeric conversions.

We need to define some algorithm (or precedence) how PA choose the Format to Construct DmtData If the Format of the node in DM Tree has multiple FORMATs.

The following algoritm is an idea:

  1. If the value to be set is empty string, then FORMAT_NULL will be chosen.
  2. Else if DmtData(String value, int format) with any of FORMAT_DATE, FORMAT_TIME, FORMAT_DATETIME, FORMAT_UNSIGNED_LONG, FORMAT_UNSIGNED_INT, and FORMAT_REFERENCE does not throw IllegalArgumentException, the format will be chosen.

Ref the earlier comment, DATETIME, UNSIGNED_LONG and UNSIGNED_INT shouldn't be in the above list? This leaves DATE, TIME, REFERENCE. This should be OK, as long as there is no value space overlap (which there probably isn't).

  1. Else There is no good way to choose one among FORMAT_STRING and FORMAT_XML. So we need to have precedence. i.e. (high precendence) FORMAT_STRING, FORMAT_XML (low precedence) (There is no specific reason of this precedence.)

Supporting both STRING and XML on a node seems very much an edge case. I think I would put XML first in the precedence list because if STRING is first, it's always going to match! So I'd say something like "XML if well-formed XML, else STRING" in this case.

Note that this perhaps illustrates the only valid use case for allowing both STRING and XML, i.e. to allow invalid XML not to be regarded as an error.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 143 Mapping table b/w dataType in TR069 and Format in DmtAdmin

Replies to questions from Ikuo in comment BZ#13:

SetParameterValues Issue-A: What to do if SetParameterValues is called against a leaf node of FORMAT_BINARY ? Do you have any idea, William ?

Given that we have agreed that FORMAT_BINARY is converted to hexBinary when reading, I think that we should allow only hexBinary when writing. If we allow base64 when writing, we could have a situation where the ACS uses base64 to set the value and then reads it back and gets hexBinary. I don't think we want that.

Got it.

Please confirm to check L5 cell in the "For SetParameterValues RPC" sheet.

WL: GetParameterValues Why would convert to base64 fail (or convert to hexBinary)? It's a well-defined mapping that can encode any binary data. IY: The original encoding format, which can be retrieved by getFormatName() might be unknown for the PA. In that case, the PA cannot convert to base64, can it ?

This is the FORMAT_RAW_BINARY case, right?

Right.

My original suggestion was that the PA might use the format name in order to choose between hexBinary and base64, but I thought that your proposal (with which I am fine) was that the PA would ignore the format name here. So I don't see any problem with conversion TO base64 (for GetParameterValues).

Conversion FROM base64 (for SetParameterValues) could definitely fail though. The appropriate failure code here would be 9007 "invalid parameter value".

Now I understand your original suggestion, which is very reasonable. I'll take your basic idea as followings:

[GetParameterValues for FORMAT_RAW_BINARY node] The original format can be retrieved by DmtData#getFormatName(). If the original one equals (ignore case) with either "base64" or "hexBinary", "base64" or "hexBinary" is used as dataType, respectively. Otherwise, error (9004). (no conversion occures)

Please confirm to check D12 and E12 cells in the "For GetParameterValues RPC" sheet.

[SetParameterValues for FORMAT_RAW_BINARY node] If the dataType specfied in RPC is "base64", FORMAT_RAW_BINARY with the formatName = "base64" is adopted. Else If the dataType specfied in RPC is "hexBinary", FORMAT_RAW_BINARY with the formatName = "hexBinary" is adopted. Else error (9007).

Please confirm to check D11 and L11 cells in the "For SetParameterValues RPC" sheet.

Note that this case can only occur for non-BBF data models, because all BBF primitive data types have explicit formats. The ACS will need out-of-band knowledge of how to format (and interpret) such values.

Can 9004 be used for GetParameterValues RPC ?

Yes.

Got it.

WL: SetParameterValues ... Now that we have 1-1 mapping, this can probably occur only for the various string mappings, is that right, e.g. string to FORMAT_DATE?
IY: No, there are a lot of cases ... what fault code to be delivered to ACS should be used ? IY: Should 9007 be used for all ERROR in the spreadsheet ? (IMO, it is OK).

Sorry, you are of course correct. Yes, think that 9007 is OK for all the ERROR cases.

Got it.

SetParameterValues RPC specifies the value to be set and its dataType defined in TR069. I expect the following situation: If string dataType is specified in SetParameterValues RPC, it can be set to leaf node whose Format is any of GROUP (which I listed up in comment#8) or FORMAT_NULL. All of them will be return as string when GetParameterValues RFC is called.

Referring to the most version of the spreadsheet, the formats to which a TR-069 can be converted are: DATE, NULL, STRING, TIME, XML, REFERENCE, NODE (where the last one applies only to lists), i.e. we no longer have the string to numeric conversions.

You are right. Sorry it was my mistake.

We need to define some algorithm (or precedence) how PA choose the Format to Construct DmtData If the Format of the node in DM Tree has multiple FORMATs.

The following algoritm is an idea:

  1. If the value to be set is empty string, then FORMAT_NULL will be chosen.
  2. Else if DmtData(String value, int format) with any of FORMAT_DATE, FORMAT_TIME, FORMAT_DATETIME, FORMAT_UNSIGNED_LONG, FORMAT_UNSIGNED_INT, and FORMAT_REFERENCE does not throw IllegalArgumentException, the format will be chosen.

Ref the earlier comment, DATETIME, UNSIGNED_LONG and UNSIGNED_INT shouldn't be in the above list? This leaves DATE, TIME, REFERENCE. This should be OK, as long as there is no value space overlap (which there probably isn't).

Right.

  1. Else There is no good way to choose one among FORMAT_STRING and FORMAT_XML. So we need to have precedence. i.e. (high precendence) FORMAT_STRING, FORMAT_XML (low precedence) (There is no specific reason of this precedence.)

Supporting both STRING and XML on a node seems very much an edge case. I think I would put XML first in the precedence list because if STRING is first, it's always going to match! So I'd say something like "XML if well-formed XML, else STRING" in this case.

I'll take William's comments and revised the algorithm:

  1. If the MetaNode supports FORMAT_NULL and the value to be set is empty string, then FORMAT_NULL will be chosen.

  2. Else if the MetaNode supports XXX and DmtData(value, XXX) does not throw IllegalArgumentException where XXX is any of FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE, XXX will be chosen.

  3. Else "FORMAT_XML" if the value is well-formed XML, else "FORMAT_STRING".

William, I have one question : How can PA judge whether the value is "well-formed XML" ? This might be related with the following your comment.

Note that this perhaps illustrates the only valid use case for allowing both STRING and XML, i.e. to allow invalid XML not to be regarded as an error.

Attached file: DataTypeMapping1_20100907.zip (application/x-sdlc, 141829 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

  1. Evgeni's comment BZ#11:

In the current state of RMT, only ServiceState//Properties//Values/ node has multiple formats. That node is not a problem, because the possible formats are {string, bin, int, bool, float} and all of them are well mapped.

  • I assume that you have listed these in priority order, i.e. string has highest priority? No priority order here. By definition of that node, its type can be one of the given. It depends on the type of the service property value.
  • Who specifies the priorities and are they are fixed (the same for all nodes)?
  • I hope that this means that on SetParameterValues, all values would be stored as string, and on GetParameterValues, values would be converted to string? When the node type is int, we will use the integer type and so on, nothing to convert here.
  • If not, and the conversion is a function of actual value, I see ambiguities because the value spaces overlap, e.g. "1" is a valid string, int, bool(?) and float(?).
  • Please clarify.
bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 144 Mapping table b/w dataType in TR069 and Format in DmtAdmin

Now I'm implementing utility classes which PA can use (RFC149 is going to define). I found a problem in the Mapping table posted at 2010-09-07 01:24 UTC.

In GetParameterValues sheet, we put "string" in D7 (FORMAT_FLOAT). However, in SetParameterValues sheet, cell G8 (FORMAT_FLOAT and string) is ERROR. I think cell G8 should be FORMAT_FLOAT or ERROR. I had updated the sheet.

Attached file: DataTypeMapping1_20100907-2.zip (application/x-sdlc, 141954 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

I'll take William's comments and revised the algorithm:

  1. If the MetaNode supports FORMAT_NULL and the value to be set is empty string, then FORMAT_NULL will be chosen.

  2. Else if the MetaNode supports XXX and DmtData(value, XXX) does not throw IllegalArgumentException where XXX is any of FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE, XXX will be chosen. Yes, but I am afraid that FORMAT_DATA will be checked before FORMAT_TIME,... The sequence needs to be specified.

  3. Else "FORMAT_XML" if the value is well-formed XML, else "FORMAT_STRING".

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

  1. Else if the MetaNode supports XXX and DmtData(value, XXX) does not throw IllegalArgumentException where XXX is any of FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE, XXX will be chosen. Yes, but I am afraid that FORMAT_DATA will be checked before FORMAT_TIME,... The sequence needs to be specified.

I don't think the sequence to check is not needed.

DmtData( String value, int format ) will validate if the format is FORMAT_DATE(pattern CCYYMMDD), FORMAT_TIME(pattern hhmmss or hhmmssZ) , FORMAT_REFERENCE(it must start with "./").

IMO, there is no overlap among them. That means, there is no value (dateType=string) which can be created DmtData with more than two format. (if DmtData(value, FORMAT_DATE) does not throw IllegalArgumentException, Both DmtData(value, FORMAT_TIME) and DmtData(value, FORMAT_REFERENCE) must throw IllegalArgumentException.

Is my understanding incorrect ?

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

In the conf call, REG agreed that there is no need to specify the precedence among FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE as specified in BZ#21.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

In GetParameterValues sheet, we put "string" in D7 (FORMAT_FLOAT). However, in SetParameterValues sheet, cell G8 (FORMAT_FLOAT and string) is ERROR. I think cell G8 should be FORMAT_FLOAT or ERROR. I had updated the sheet.

In the conf call on Sep 7th, REG agree as described in comment#19.

This change requires the update the algorithm in Comment#17 in the case that dataType SetParameterValues specifies is string and leaf node supports multiple Format. Ikuo will investigate it.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 146 Mapping table b/w dataType in TR069 and Format in DmtAdmin

In the conf call on Sep 7th, REG agreed as follows:

My original suggestion was that the PA might use the format name in order to choose between hexBinary and base64, but I thought that your proposal (with which I am fine) was that the PA would ignore the format name here. So I don't see any problem with conversion TO base64 (for GetParameterValues).

Conversion FROM base64 (for SetParameterValues) could definitely fail though. The appropriate failure code here would be 9007 "invalid parameter value".

Now I understand your original suggestion, which is very reasonable. I'll take your basic idea as followings:

[GetParameterValues for FORMAT_RAW_BINARY node] The original format can be retrieved by DmtData#getFormatName(). If the original one equals (ignore case) with either "base64" or "hexBinary", "base64" or "hexBinary" is used as dataType, respectively. Otherwise, error (9004). (no conversion occures)

Please confirm to check D12 and E12 cells in the "For GetParameterValues RPC" sheet.

[SetParameterValues for FORMAT_RAW_BINARY node] If the dataType specfied in RPC is "base64", FORMAT_RAW_BINARY with the formatName = "base64" is adopted. Else If the dataType specfied in RPC is "hexBinary", FORMAT_RAW_BINARY with the formatName = "hexBinary" is adopted. Else error (9007).

Please confirm to check D11 and L11 cells in the "For SetParameterValues RPC" sheet.

We agreed:

[GetParameterValues for FORMAT_RAW_BINARY node] PA converts the byte array returned by DmtData#getRawBinary() into base64. The dataType of base64 will be used for GetParameterValuesResponse.

[SetParameterValues for FORMAT_RAW_BINARY node] If the dataType is base64, PA will call and the first element of the returned array will be adopted as formatName. PA will create DmtData(byte[] value, String formatName) where value is converted from string in RPC into as base64 and formatName is the first element of returned array of MetaNode#getFormatNames(). Here convertion from string to byte[] could efinitely fail. The appropriate failure code here would be 9007 "invalid parameter value".

William, I have one question : How can PA judge whether the value is "well-formed XML" ? This might be related with the following your comment.

Note that this perhaps illustrates the only valid use case for allowing both STRING and XML, i.e. to allow invalid XML not to be regarded as an error.

We agreed: RFC149 and Spec will not mention the definition of "well-formed XML".

This change requires the update the algorithm in Comment#17 in the case that dataType SetParameterValues specifies is string and leaf node supports multiple Format. Ikuo will investigate it.

I revised the algorithm:

  1. If the MetaNode supports FORMAT_NULL and the value to be set is empty string, then FORMAT_NULL will be chosen.

  2. If the MetaNode supports FORMAT_FLOAT and Float.parseFloat() with the value string does not throw Exception, then FORMAT_FLOAT will be chosen.

  3. Else if the MetaNode supports XXX and DmtData(value, XXX) does not throw IllegalArgumentException where XXX is any of FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE, XXX will be chosen.

  4. Else if the value is well-formed XML and the MetaNode supports FORMAT_XML, then FORMAT_XML will be chosen,

  5. Else if the MetaNode supports FORMAT_STRING, FORMAT_STRIN will be chosen.

  6. Else ERROR (fault code 9007)

Regarding SetParameterValues, as multiple Format can support string dataType, multiple Format can support either base64 or hexBinary.

[base64 dataType in SetParameterValues]

  1. If the MetaNode supports FORMAT_BASE64, then FORMAT_BASE64 will be chosen.

  2. Else If the MetaNode supports FORMAT_RAW_BINARY and MetaNode#getFormatNames() returns more than 0 elements, then FORMAT_RAW_BINARY will be chosemn.

  3. Else ERROR (fault code 9007)

    [hexBinary dataType in SetParameterValues]

  4. If the MetaNode supports FORMAT_HEXBINARY, then FORMAT_HEXBINARY will be chosen.

  5. Else If the MetaNode supports FORMAT_BINARY, then FORMAT_BINARY will be chosen.

  6. Else ERROR (fault code 9007)

Attached spreadsheet has been reflected by all my comments. Please check it !

Attached file: DataTypeMapping1_20100908.zip (application/x-sdlc, 137939 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

Attached spreadsheet has been reflected by all my comments. Please check it ! For completeness in SetParameterValues, we can map FORMAT_RAW_STRING to TR string type. So, if ACS knows what exactly the string means, it can set it.

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Attached spreadsheet has been reflected by all my comments. Please check it !

I reviewed the latest spreadsheets and:

  1. GetParameterValues one: 100% OK

  2. SetParameterValues one: a few minor comments:

2a. for conversion from string, most show "FORMAT_xxx or ERROR"; when multiple formats are supported it's a little more complicated than this because rather than ERROR it will fall back on the next supported format; could add a note that the algorithms in this case are specified separately? or could order the rows so that they are in precedence order, and add a note explaining what happens when multiple formats are supported (in which case the algorithms probably do not need defining separately)?

2b. for conversion from string to FORMAT_XML, should show "FORMAT_XML or ERROR" because not all strings are valid XML

2c. for conversion from string to FORMAT_REFERENCE, there could be an ERROR case, e.g. if the value is not a valid TR-069 path, or the result (after conversion) is not a valid DMTAdmin path?

2d. for conversion from string to FORMAT_RAW_STRING I agree with Evgeni; this is exactly comparable with base64 to FORMAT_RAW_BINARY, so will use the formatName which is the first element of the metanode getFormatNames()

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Minor additional comment re my comment BZ#26 2a: I realise that there is more info in the algorithm descriptions than just the precedence, e.g. they include the constructors to be used and indicate the error criteria. So feel free to ignore part or all of 2a!

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 147 Mapping table b/w dataType in TR069 and Format in DmtAdmin

  1. SetParameterValues one: a few minor comments:

2b. for conversion from string to FORMAT_XML, should show "FORMAT_XML or ERROR" because not all strings are valid XML

OK, since DmtData(String value, int format) with FORMAT_XML will not validate the format, PA should check whether the value is well-formed XML as the node supports multiple formats.

2c. for conversion from string to FORMAT_REFERENCE, there could be an ERROR case, e.g. if the value is not a valid TR-069 path, or the result (after conversion) is not a valid DMTAdmin path?

OK. I would say: DmtData(String value, int format) with FORMAT_REFERENCE will check validity. It means, if the result (after conversion) is not a valid DMTAdmin URI, ERROR. (no check for the validation of TR-069 path).

2d. for conversion from string to FORMAT_RAW_STRING I agree with Evgeni; this is exactly comparable with base64 to FORMAT_RAW_BINARY, so will use the formatName which is the first element of the metanode getFormatNames()

OK. I updated the spread sheet and the algorithm for string dataType in SetParameterValues:

[string dataType in SetParameterValues]

  1. If the MetaNode supports FORMAT_NULL and the value to be set is empty string, then FORMAT_NULL will be chosen.

  2. If the MetaNode supports FORMAT_FLOAT and Float.parseFloat() with the value string does not throw Exception, then FORMAT_FLOAT will be chosen.

  3. Else if the MetaNode supports XXX and DmtData(value, XXX) does not throw IllegalArgumentException where XXX is any of FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE, XXX will be chosen.

  4. Else if the MetaNode supports FORMAT_XML and the value is well-formed XML, then FORMAT_XML will be chosen.

  5. Else if the MetaNode supports FORMAT_RAW_STRING XML, then FORMAT_RAW_STRING will be chosen.

  6. Else if the MetaNode supports FORMAT_STRING, FORMAT_STRING will be chosen.

  7. Else ERROR (fault code 9007)

Attached file: DataTypeMapping_20100909.zip (application/x-sdlc, 138669 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

Created an attachment (id=147) [details] Mapping table b/w dataType in TR069 and Format in DmtAdmin

  1. SetParameterValues one: a few minor comments:

2b. for conversion from string to FORMAT_XML, should show "FORMAT_XML or ERROR" because not all strings are valid XML

OK, since DmtData(String value, int format) with FORMAT_XML will not validate the format, PA should check whether the value is well-formed XML as the node supports multiple formats. XML content can depend on the data model. Some plug-ins can use only a XML fragment. This will break the well-formed XML rules. I suppose that PA must not validate the XML content.

2c. for conversion from string to FORMAT_REFERENCE, there could be an ERROR case, e.g. if the value is not a valid TR-069 path, or the result (after conversion) is not a valid DMTAdmin path?

OK. I would say: DmtData(String value, int format) with FORMAT_REFERENCE will check validity. It means, if the result (after conversion) is not a valid DMTAdmin URI, ERROR. (no check for the validation of TR-069 path). DmtData constructor will be called: DmtData("./A/B", FORMAT_REFERENCE) Right? TR to DMT URI transformation should be done in PA, because it is TR specific.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

2b. for conversion from string to FORMAT_XML, should show "FORMAT_XML or ERROR" because not all strings are valid XML

OK, since DmtData(String value, int format) with FORMAT_XML will not validate the format, PA should check whether the value is well-formed XML as the node supports multiple formats. XML content can depend on the data model. Some plug-ins can use only a XML fragment. This will break the well-formed XML rules. I suppose that PA must not validate the XML content.

Hmm. It matches the reason why DmtData(String value, int format) with FORMAT_XML does not check the validity.

So I there are three alternatives:

Alternative1: PA always does NOT check the validity of XML.

Alternative2: PA does NOT check the validity of XML, if the node supports only one Format "FORMAT_XML", while PA checks the validity if the node supports multiple Formats (algorithm in comment BZ#28).

Alternative3: PA always checks the validity of XML in both cases that the node supports only one Format "FORMAT_XML" and multiple format.

According the Evgeni's comment, we should choose either Alternative1 and Alternative2. In Alternative2, the validity check is done because there is no clear way to choose one between FORMAT_XML and FORMAT_STRING.

Which do you support ?

2c. for conversion from string to FORMAT_REFERENCE, there could be an ERROR case, e.g. if the value is not a valid TR-069 path, or the result (after conversion) is not a valid DMTAdmin path?

OK. I would say: DmtData(String value, int format) with FORMAT_REFERENCE will check validity. It means, if the result (after conversion) is not a valid DMTAdmin URI, ERROR. (no check for the validation of TR-069 path). DmtData constructor will be called: DmtData("./A/B", FORMAT_REFERENCE) Right?

Right.

TR to DMT URI transformation should be done in PA, because it is TR specific.

Rigth. PA can do that transformation by utility class defined by RFC149.

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

(In reply to comment BZ#28 including follow-up on comment BZ#29 and comment BZ#30)

Mapping table b/w dataType in TR069 and Format in DmtAdmin

SetParameterValues table: the hexBinary -> FORMAT_RAW_BINARY cell should be the same as the base64 -> FORMAT_RAW_BINARY cell

2b. for conversion from string to FORMAT_XML, should show "FORMAT_XML or ERROR" because not all strings are valid XML

Yes, I had missed that the spec says that the format isn't checked. I agree with Alternative2 (i.e. check only when the node supports both XML and STRING).

Regardless of whether it is a fragment, surely the angle brackets will match, so the implementation could just scan the string character by character and keep a counter that increments for "<" and decrements for ">". The value of the counter should always be in the range [0,1] and should be 0 at end of string. But I guess the details are up to the implementation?

[string dataType in SetParameterValues]

  1. Else if the MetaNode supports XXX and DmtData(value, XXX) does not throw IllegalArgumentException where XXX is any of FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE, XXX will be chosen.

There is no value space overlap, but should we nevertheless specify the order in which these are checked?

There is one slight extra wrinkle for REFERENCE in that TR-106 A.2.3.4 allows the Root or Service Object name to be omitted (this was probably a mistake ... but it's in the standard):

<String parameters whose values are path names are subject to the rules of section 3.2.5, so object names do not include a trailing dot. The parameter value (or each list item if the parameter is list-valued) MUST always be a full path name, with the single exception that a path that begins with a dot is relative to the Root or Service Object. For example, in the Device Root Object, a parameter value of “.DeviceInfo”always means “Device.DeviceInfo”>

...so there is an initial processing step that will use the parameter NAME to prepend the Root or Service Object name if a reference begins with ".", e.g. in "Device.A", a reference to ".B" will become "Device.B", or in "Device.Services.CService.1.D", a reference to ".E" will become "Device.Services.CService.1.E".

I would say that there is no requirement to remove such a prefix when reading the parameter value. Yes, the string value of the reference might not be identical to the value that was written, but both values reference the same node. This is equivalent to allowing an integer to be written as "01" but read back as "1" (canonical representation).

  1. Else if the MetaNode supports FORMAT_RAW_STRING XML, then FORMAT_RAW_STRING will be chosen.

Is the word "XML" above just a typo?

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

(In reply to comment BZ#28 including follow-up on comment BZ#29 and comment BZ#30)

Mapping table b/w dataType in TR069 and Format in DmtAdmin

SetParameterValues table: the hexBinary -> FORMAT_RAW_BINARY cell should be the same as the base64 -> FORMAT_RAW_BINARY cell

It was my mistake. The cell should have updated to "ERROR". The dataType for the Response of GetParameterValues against the node of FORMAT_RAW_BINARY is only base64.It means, the dataType of hexBinary to the node of FORMAT_RAW_BINARY should not be allowed. (If it allowed, ACS will get the value as base64 even if the value is set as hexBinary.)

2b. for conversion from string to FORMAT_XML, should show "FORMAT_XML or ERROR" because not all strings are valid XML

Yes, I had missed that the spec says that the format isn't checked. I agree with Alternative2 (i.e. check only when the node supports both XML and STRING).

Regardless of whether it is a fragment, surely the angle brackets will match, so the implementation could just scan the string character by character and keep a counter that increments for "<" and decrements for ">". The value of the counter should always be in the range [0,1] and should be 0 at end of string. But I guess the details are up to the implementation?

IMO, Those kind of things should be specified in RFC149, just as a guideline. I'll add it.

[string dataType in SetParameterValues]

  1. Else if the MetaNode supports XXX and DmtData(value, XXX) does not throw IllegalArgumentException where XXX is any of FORMAT_DATE, FORMAT_TIME, and FORMAT_REFERENCE, XXX will be chosen.

There is no value space overlap, but should we nevertheless specify the order in which these are checked?

There is one slight extra wrinkle for REFERENCE in that TR-106 A.2.3.4 allows the Root or Service Object name to be omitted (this was probably a mistake ... but it's in the standard):

<String parameters whose values are path names are subject to the rules of section 3.2.5, so object names do not include a trailing dot. The parameter value (or each list item if the parameter is list-valued) MUST always be a full path name, with the single exception that a path that begins with a dot is relative to the Root or Service Object. For example, in the Device Root Object, a parameter value of “.DeviceInfo”always means “Device.DeviceInfo”>

...so there is an initial processing step that will use the parameter NAME to prepend the Root or Service Object name if a reference begins with ".", e.g. in "Device.A", a reference to ".B" will become "Device.B", or in "Device.Services.CService.1.D", a reference to ".E" will become "Device.Services.CService.1.E".

I would say that there is no requirement to remove such a prefix when reading the parameter value. Yes, the string value of the reference might not be identical to the value that was written, but both values reference the same node. This is equivalent to allowing an integer to be written as "01" but read back as "1" (canonical representation).

Let me confirm: If SetParameterValues the value ".B" to the path "Device.A" is invoked and the node "./Device/A" in DM Tree has FORMAT_REFERENCE, what value should be used for DmtData(String value, int format) ? "./Device/B" ?

  1. Else if the MetaNode supports FORMAT_RAW_STRING XML, then FORMAT_RAW_STRING will be chosen.

Is the word "XML" above just a typo?

Sorry, it was typo.

  1. Else if the MetaNode supports FORMAT_RAW_STRING, then FORMAT_RAW_STRING will be chosen.
bjhargrave commented 14 years ago

Comment author: jblackford@2wire.com

Just an additional comment regarding the GetParameterValuesResponse when converting from FORMAT_XML to string... we need to note in RFC-149 that the XML needs to be escaped and we should provide a reference to TR-106 Section 3.2.1, which states:

"When transporting a string value within an XML document, any characters which are special to XML MUST be escaped as specified by the XML specification [8]. Additionally, any characters other than printable ASCII characters, i.e. any characters whose decimal ASCII representations are outside the (inclusive) ranges 9-10 and 32-126, SHOULD be escaped as specified by the XML specification."

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Let me confirm: If SetParameterValues the value ".B" to the path "Device.A" is invoked and the node "./Device/A" in DM Tree has FORMAT_REFERENCE, what value should be used for DmtData(String value, int format) ? "./Device/B" ?

yes

bjhargrave commented 14 years ago

Comment author: Evgeni Grigorov <e.grigorov@prosyst.com>

2b. for conversion from string to FORMAT_XML, should show "FORMAT_XML or ERROR" because not all strings are valid XML

OK, since DmtData(String value, int format) with FORMAT_XML will not validate the format, PA should check whether the value is well-formed XML as the node supports multiple formats. XML content can depend on the data model. Some plug-ins can use only a XML fragment. This will break the well-formed XML rules. I suppose that PA must not validate the XML content.

Hmm. It matches the reason why DmtData(String value, int format) with FORMAT_XML does not check the validity.

So I there are three alternatives:

Alternative1: PA always does NOT check the validity of XML. Sounds good to me. XML validation is time-consuming and doesn't provide clear solution, only assumptions.

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Alternative1: PA always does NOT check the validity of XML. Sounds good to me. XML validation is time-consuming and doesn't provide clear solution, only assumptions.

I just want to point out that, if there is no check, a node that supports XML and STRING will always be stored as XML, regardless of validity. So there is no point ever allowing a node to support both XML and STRING. Is that want we want? If so, fine.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Let me confirm: If SetParameterValues the value ".B" to the path "Device.A" is invoked and the node "./Device/A" in DM Tree has FORMAT_REFERENCE, what value should be used for DmtData(String value, int format) ? "./Device/B" ? yes

I got it.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Alternative1: PA always does NOT check the validity of XML. Sounds good to me. XML validation is time-consuming and doesn't provide clear solution, only assumptions. I just want to point out that, if there is no check, a node that supports XML and STRING will always be stored as XML, regardless of validity. So there is no point ever allowing a node to support both XML and STRING. Is that want we want? If so, fine.

We should not restrict the data model. In other words, MetaNode of a node should be able to support both FORMAT_STRING and FORMAT_XML. However, Alternative2 has some ambiguous points. Therefore, I would like to propose Alternative4.

Alternative4: PA always does NOT check the validity of XML. If the node supports both FORMAT_XML and FORMAT_STRING and PA needs to choose one, FORMAT_STRING will be chosen.

From the point of view of TR-069, either FORMAT_XML or FORMAT_STRING is fine. However, FORMAT_STRING seems more generic. Assume that SetParameterValue against a node with string dataType, FORMAT_STRING will be chosen. If GetParameterValue against the node is called, the value as string dataType will be returned. It seems to keep consistency.

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

Alternative4: PA always does NOT check the validity of XML. If the node supports both FORMAT_XML and FORMAT_STRING and PA needs to choose one, FORMAT_STRING will be chosen.

From the point of view of TR-069, either FORMAT_XML or FORMAT_STRING is fine. However, FORMAT_STRING seems more generic. Assume that SetParameterValue against a node with string dataType, FORMAT_STRING will be chosen. If GetParameterValue against the node is called, the value as string dataType will be returned. It seems to keep consistency.

I think that storing it as FORMAT_XML and FORMAT_STRING are equally consistent from the TR-069 point of view aren't they?

I also think that the most natural hierarchy is from most specific to least specific, which would put FORMAT_STRING last.

Personally I am OK with Alternative2, which performs a check only in the (very unlikely!) event that the node supports both FORMAT_XML and FORMAT_STRING.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Alternative4: PA always does NOT check the validity of XML. If the node supports both FORMAT_XML and FORMAT_STRING and PA needs to choose one, FORMAT_STRING will be chosen.

From the point of view of TR-069, either FORMAT_XML or FORMAT_STRING is fine. However, FORMAT_STRING seems more generic. Assume that SetParameterValue against a node with string dataType, FORMAT_STRING will be chosen. If GetParameterValue against the node is called, the value as string dataType will be returned. It seems to keep consistency. I think that storing it as FORMAT_XML and FORMAT_STRING are equally consistent from the TR-069 point of view aren't they?

Right. I agree.

I also think that the most natural hierarchy is from most specific to least specific, which would put FORMAT_STRING last. Personally I am OK with Alternative2, which performs a check only in the (very unlikely!) event that the node supports both FORMAT_XML and FORMAT_STRING.

I understand what you mean. However, Alternative2 makes the implementation of TR-069 PA diverse because the algorithm of judging whether the value is well-formed XML or not is ambiguous: an implementation-A will use FORMAT_XML but other will use FORMAT_STRING. It should be avoided, shouldn't it ?

bjhargrave commented 14 years ago

Comment author: wlupton@2wire.com

(In reply to comment BZ#38 and comment BZ#40)

Alternative4: PA always does NOT check the validity of XML. If the node supports both FORMAT_XML and FORMAT_STRING and PA needs to choose one, FORMAT_STRING will be chosen.

I accept your arguments and your Alternative4.

Do we then need to note somewhere that there is no point supporting FORMAT_XML and FORMAT_STRING on a node, because the value can never be stored as FORMAT_XML?

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

(In reply to comment BZ#38 and comment BZ#40)

Alternative4: PA always does NOT check the validity of XML. If the node supports both FORMAT_XML and FORMAT_STRING and PA needs to choose one, FORMAT_STRING will be chosen.

I accept your arguments and your Alternative4.

Thank you for your understanding.

Do we then need to note somewhere that there is no point supporting FORMAT_XML and FORMAT_STRING on a node, because the value can never be stored as FORMAT_XML?

OK. Let me clarify: From "JUST" TR-069 perspective, the node which contains MetaNode with both FORMAT_XML and FORMAT_STRING is just as same as the node which contains MetaData with FORMAT_STRING. However, the data model might be used for another remote management protocol or local managers. Therefore, it is up to the implementation of a data model (DataPlugin) what FORMAT a MetaNode of a node supports. I can add those notes in RFC149.

bjhargrave commented 14 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Created attachment 149 Mapping table b/w dataType in TR069 and Format in DmtAdmin

I had updated the sheet. The updated part is written in red.

Attached file: DataTypeMapping_20100916.zip (application/x-sdlc, 26910 bytes) Description: Mapping table b/w dataType in TR069 and Format in DmtAdmin

bjhargrave commented 13 years ago

Comment author: Ikuo Yamasaki <yamasaki.ikuo@lab.ntt.co.jp>

Close this bug, because the RFC149 ver. 1.5.0 has reflect the result of it.