ovis-hpc / ldms

OVIS/LDMS High Performance Computing monitoring, analysis, and visualization project.
https://github.com/ovis-hpc/ovis-wiki/wiki
Other
99 stars 51 forks source link

List data types should probably be declared in the ldms schema #1097

Open morrone opened 1 year ago

morrone commented 1 year ago

The way that array types were added to ldms, we have no way to declare the data type(s) that the list will contain. The data type is only assigned at metric set creation time. So this essentially means that the schema no longer expresses the full schema of a metric set. In the worst case, we could have two different metric sets that share a common schema, but have different data types in their lists.

If we restricted lists to only containing a single data type and making that declaration in the schema, that would address this issue.

tom95858 commented 1 year ago

@morrone, @nichamon I believe that this is the way it works now. @nichamon please confirm and if true please close this issue.

morrone commented 1 year ago

I do not think so. We can only tell the schema from the metric set where lists are concerned. Two metric sets that declare they are using the same schema can, in fact have different schemas. This is why we need to make the data type of the array contents part of the schema.

nichamon commented 1 year ago

@tom95858 No, the current code does not work as @morrone's suggestion yet.

We discussed and planned to add the restriction that all list entries must be of the same value type. By doing this, we could add the value type of the list entries to the schema. However, we have not gotten to this item yet.

tom95858 commented 2 weeks ago

@nichamon what is the status of this? There are two parts of it, declaring what the list will contain, and verifying that the list contains only what it is supposed to (i.e. the 1st element). I think we don't have the the former which makes decomposition an issue with empty lists.