Hierarchical structure for hierarchies

[oalt]

Issue

When SpecIf is used as data format in a backend database, you will have many hierarchy objects containing requirement documents, BOMs etc. Currently the hierarchy objects are just an array of hierarchies. Normally engineering data is organized and assigned to projects or specific systems. So it would be helpful, when you have a possibility to organize hierarchies in a tree structure with folders (headings), perhaps paragraphs and hierarchies, too - similar to file systems or repositories of MBSE tools.

Proposal

Extend SpecIF with a new type called "repositories" on the same level as the other data types (e.g. hierarchies) and use the same concept as used for hierarchies to link to hierarchies using recursive nodes. Because of a hierarchy can also have properties, we can use a hierarchy object as well to represent a folder/directory.

[odungern]

@oalt : thanks for your thoughts! There are some associated questions with this issue. Let me start the discussion with some ideas crossing my mind:

1. We had a similar thought when implementing the "Interactive Spec": We did not want to list multiple projects (=SpecIF files) in a flat list, but to organize them in a hierarchy. Towards this end, we added a UNIX-style path to the project title, such as "productName/componentName/projectName". When listing the projects, we parsed the project titles and created a corresponding hierarchy. Advantage: The project files are self-contained, i.e. there is no separate structural information. Disadvantage: If you want to rename a path, you need to modify multiple files. Also it is not possible to have a different project hierarchy in a different context.
2. We agree that we should be able to manage a specification consisting of multiple components. Each component spec is consistent (= passes the schema and consistency check) and a certain team can assume the responsibility. This corresponds to the "Domain Driven Design" approach by Eric Evans and agile development of "Micro-Services" (in case of software). Then, we need to think about "public" and "private" elements and how to reference public elements of another component.
3. In the context of OEM and supplier collaboration/negotiation, there can be multiple threads of model exchange in parallel. The ReqIF Implementor Forum (ProSTEP iViP) has defined a ReqIF extension to specify a "conversation" in order to distinguish the threads and to define the respective "views", as a supplier shall not see the conversation of any other.
4. In case of ReqIF it is possible to group several specs in a common ZIP-container, where the filename of the ZIP is considered the name of the group. Perhaps it makes sense to create a "manifest" with a description of the contained specs and their hierarchical structure. This brings us back to your original question ...

I would like to discuss all these aspects and develop an overarching concept. Any idea is appreciated.

[oalt]

I think it is not necessary to have all data in one SpecIF file. You can have multiple files and the superset is the complete specification. Because of every element in SpecIF has a unique ID, you can use these IDs to reference the elemnt you want. So you can have a set of SpecIF files containing the data (e.g. a set of hierarchies) and another SpecIF file for example referencing these hierarchies using "repository"-objects as proposed above to define a hierarchical context. So it is possible to have a different ordering of hierarchies for different users or contexts. If you do not want to bring the hierarchies in a structure, you just do not use the repository-element.

[odungern]

@oalt : In fact, I must admit that I do not quite get your point. Could you give an example of hierarchically structured hierarchies?

[oalt]

Here is an example how it could look like:

"repositories": [
  {
    "id": "REP-69c8a7730000bca80137509a49b1218b",
      "title": "Requirement repository",
      "description": "This is the repository for all requirement specifications (hierarchies) in our comnpany.",
      "class": "RepositoryClass-ID",
      "revision": 1,
      "hierarchyNodes": [
        {
          "id": "RepoH-Fld-5a5f54090000bca801375b04a668f1a7",
          "hierarchy": {
            "id": "SP-59c8a7730000bca80137509a49b1218b",
            "revision": 0
          },
          "revision": 1,
          "hierarchyNodes": [
            {
              "id": "SH-Pln-5a4755dd0000bca801375293a62c90a8",
              "hierarchy": {
                "id": "SP-5aa4755dd0000bca801375293a62c90a8",
                "revision": 0
              },
              "revision": 1,
              "changedAt": "2017-06-19T20:14:47+02:00"
            }
          ],
          "changedAt": "2017-06-19T20:14:47+02:00"
        }
  }
]

It is the same principla as for hierarchies referencing Resources with the difference that a repository references hierarchies to bring them in a tree structure. I am not sure if it is necessary to create a repository class. This depends of if you want to give a repository additional attributes (properties). If a hierarchy is defined of class "folder", you can reference it to create "folder hierarchies" as parents for hierarchies containing content (e.g. a SpecIF:Hierarchy) in the tree structure. The JSON schema could look like that:

 "hierarchyNodes": {
      "description": "A list of pointers to hierarchies; may be nested to build a tree.",
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": {
            "$ref": "#/definitions/id"
          },
          "hierarchy": {
            "description": "The pointer to the hierarchy to be displayed at this position.",
            "$ref": "#/definitions/key"
          },
          "hierarchyNodes": {
            "description": "The next repository level.",
            "$ref": "#/definitions/hierarchyNodes"
          },
          "revision": {
            "$ref": "#/definitions/revision"
          },
          "changedAt": {
            "$ref": "#/definitions/dateTime"
          }
        },
        "required": [ "id", "hierarchy" ]
      }
    },

 "repositories": {
      "description": "Repository trees, each referencing hierarchies to constitute a tree structure for hierarchies.",
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "id": {
            "$ref": "#/definitions/id"
          },
          "title": {
            "type": "string"
          },
          "description": {
            "type": "string"
          },
          "class": {
            "$ref": "#/definitions/key"
          },
          "properties": {
            "$ref": "#/definitions/properties"
          },
          "hierarchyNodes": {
            "$ref": "#/definitions/hierarchyNodes"
          },
          "revision": {
            "$ref": "#/definitions/revision"
          },
          "changedAt": {
            "$ref": "#/definitions/dateTime"
          },
          "changedBy": {
            "type": "string"
          }
        },
        "required": [ "id", "title", "class", "hierarchyNodes", "changedAt" ]
      }
    }

[odungern]

.. excuse me please, I rather meant an application example.

In my eyes, a SpecIF project (= file) has perhaps 2.000-10.000 resources and let's say 20.000 statements. The product management, requirement engineers and product architects are the authors. There are perhaps 3 or 4 target groups which get a tailored view, provided by a hierarchy each. A target group may be the functional security specialists for review, the the product management and marketing group for input and review and finally the product developers, internal or external (supplier).

A complete product may consist of 10 to 1.000 such component specifications represented by a SpecIF file (or logical database). The components hide most of their internal detail. Some resources, statements and diagrams are exposed and may be referenced by other higher-level components. Or the other way around.

So I see more need to provide a structure for a multitude of SpecIF components/projects than to provide a hierarchy of hierarchies. This would ask for an external hierarchy of SpecIF components/projects.

What is your view?

[odungern]

@oalt : You have told me that you have a solution in which you present multiple hierarchies from multiple sources in a common environment. Yes, that's a perfect example of model integration. In this situation it makes sense to have a hierarchy for the hierarchy roots. However, something bothers me when thinking of hierarchies of hierarchies. Perhaps a hierarchy root should be just a fat node (with properties) and you can place it in another tree wherever you want. In the end you want a combined tree, don't you?

[odungern]

@oalt : Thinking even further, I come to believe that we don't need any fat nodes (hierarchy roots). A hierarchy then has just nodes and in case I want to add some information to a specific hierarchy, then I create a resource with the information and point to it. Just the same way we are using resources for folders. Then, we don't need any hierachyClasses, either. I love it. The bidirectional transformation between SpecIF and ReqIF is still possible: Any ReqIF HIERARCHY-TYPE is represented by a resourceClass ... and a HIERARCHY-ROOT is transformed to a node pointing to a resource with the information content of the HIERARCHY-ROOT. Coming to your original proposal: You should be fine to create a hierarchy of hierarchies using the nodes ... What do you think?

[odungern]

.. so a node could be allowed not only to point to an internal resource, but alternatively to a SpecIF container ... which is expanded, when it is in reach.

[odungern]

Have a look - first step:

• schema v0.10.7
• no more hierarchyClasses.
• hierarchy roots are now nodes
• previous hierarchyClasses are now represented by resourceClasses
• previous hierarchies are now represented by resources to which the hierarchy root-nodes point.

[odungern]

Schema of SpecIF-Level-3 has been enhanced, see https://specif.de/v0.11.7/schema. The consistency checker will be updated accordingly in the coming weeks.

[odungern]

Consistency Checkers for v0.10.8 and v0.11.8 have been updated.

[odungern]

Thanks a lot, @oalt , for your proposal. Getting rid of the hieryClasses was a very good move!