jastram commented 10 years ago

Design Spec: Subset-26 conversion to ProR

The objective of this tool is to convert the MS Word-based Subset-26 to the ProR-based ReqIF format. Important is the ability to do subsequent updates with an updated Subset and the availability of section numbers as human- and machine-readable IDs.

Note that the question of Traceability is outside the scope of this issue.

TODO

[x] Find someone who has access to DOORS 9.4, or find an academic partner who is willing to acquire (and install) the free academic version of DOORS 9.4.
[x] Convert Subset26 to ReqIF.
[ ] Find someone to create a plug-in that processes the generated ReqIF to extract the section numbers (Michael if no one else is found).
Stakeholders
Product Owner: Bernd
V&V: Jan Welte
Modeling: Uwe Steinke
Major Use Cases

Note: UC-Initial-Subset-Import needs to be provided urgently, but UC-Subset-Update can be delivered much later. We don't expect updates more often than once or twice a year.

Actor(s):

Administrator
UC-Initial-Subset-Import

Description: Convert a Word-based document into a ProR-readable ReqIF-Model.

UC-Subset-Update

Description: Update a ReqIF-Model with an updated Word document. We assume that the section numbering has not changed and expect traces (if any) to stay intact.

Possible solutions

As discussed with Bernd on 5-Dec-2013, possible solutions include (the favorite being selected):

[ ] Adapt existing scripts DB has for importing the Subset into DOORS. Reexport as ReqIF with DOORS 9.4.
[ ] Adapt existing scripts DB has for importing the Subset into DOORS. Reexport as RIF with DOORS 9.2. Convert RIF to ReqIF.
[ ] Create a specialized Plug-In for openETCS that reads Word and writes ReqIF.
[ ] Create a generic Plug-In for ProR that reads Word and writes ReqIF.
[x] Import the Subset directly into DOORS (no further scripts required). Export as ReqIF with DOORS 9.4. Process further with a plug-in (to be written), to extract section headings, to be used as IDs.

jastram commented 10 years ago

@stanpinte wrote:

As mentioned already several times, we have all requirements of Subset-026 3.3.0 and 027 encoded in xml, in ERTMSFormalSpecs requirement format.

These are available on the repo https://github.com/openETCS/ERTMSFormalSpecs , for easy translation to another XML format, like ReqIF.

Feel free to ask for support if you need help using it.

Very kind regards

Stan.

jastram commented 10 years ago

Moved from https://github.com/openETCS/toolchain/issues/199#issuecomment-30311170

I want to pick-up the string arround Laurents comment for the SRS contribution to ProR: https://github.com/openETCS/toolchain/issues/199#issuecomment-28907550

In order to have this as an practical approach I need the following properties:

[ ] The import has to be reproducable and without manual intervention, or
[ ] There has to be a test on the correctness of the result
[ ] I see value in the pictures and in formating issues.
[ ] The export from EFS to REQIF also needs to be reproducable and without manual intervention.

@LaurentFerier how much manual action was involved in the import?

br Bernd

LaurentFerier commented 10 years ago

Hi Bernd,

According to your question

@LaurentFerier how much manual action was involved in the import?

I agree with you that best would be to have a fully automated importation process, since it would solve the problem once and for all. We unfortunately did not reach that goal because out first XML version of Subset-026 had been modified manually.

However, we have an automatic process which tracks differences between two versions. It works as follows

Read one version of the file and classify all requirements by paragraph numbers
Read the second version of the file and also classify requirements by paragraph numbers
Compare those paragraphs (ignoring minor differences, such as white-spaces and control character differences) and create a diff report between those two files
Import those differences in our XML document, and mark the corresponding requirements as needing revision (manual review of what the importer did). As a matter of fact, we also use traceability information to mark where the model should be updated according to these requirements changes (impact analysis)

So, Importation in the XML is not completely automatic. The manual actions were the followings

Review all modified requirements (and flag them as reviewed to keep track of this action)
Manually modify the requirements that were not correctly imported (because the original format did not allow it)
Use the diff report to handle all changes that could not be automatically taken care of

The major problem we encountered were due to paragraph number changes, typically because a row was inserted in the middle of a table (each row is considered as a specific requirement in EFS). Also note that we do not consider images in this process.

Hoping this helps, Cheers, Laurent

BerndHekele commented 10 years ago

Thanks for clarification. I'll take this poition into tomorrow's WP3 meeting.

BerndHekele commented 10 years ago

@LaurentFerier We would give it a try with the EFS solution Can you (based on some instruction by @jastram ) prode the data in REQIF format?

LaurentFerier commented 10 years ago

I will contact Michael about this REQIF format.

jastram commented 10 years ago

Hi Laurent,

The easiest is to provide an example. Check below for my comments:

<?xml version="1.0" encoding="UTF-8"?>
<REQ-IF xmlns="http://www.omg.org/spec/ReqIF/20110401/reqif.xsd" xmlns:configuration="http://eclipse.org/rmf/pror/toolextensions/1.0">
<THE-HEADER>
<REQ-IF-HEADER IDENTIFIER="rmf-4c6e09e8-e0ab-4c5b-b273-ed4a9fc5a3f2">
  <COMMENT>Created by: jastram</COMMENT>
  <CREATION-TIME>2013-12-17T10:46:26.093+01:00</CREATION-TIME>
  <REQ-IF-TOOL-ID>ProR (http://pror.org)</REQ-IF-TOOL-ID>
  <REQ-IF-VERSION>1.0.1</REQ-IF-VERSION>
  <SOURCE-TOOL-ID>ProR (http://pror.org)</SOURCE-TOOL-ID>
</REQ-IF-HEADER>
</THE-HEADER>
<CORE-CONTENT>
<REQ-IF-CONTENT>
  <DATATYPES>
    <DATATYPE-DEFINITION-STRING IDENTIFIER="_GJB8cGcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00" LONG-NAME="T_String32k" MAX-LENGTH="32000"/>
    <DATATYPE-DEFINITION-STRING IDENTIFIER="_KHHioGcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:49.476+01:00" LONG-NAME="T_ID"/>
  </DATATYPES>
  <SPEC-TYPES>
    <SPEC-OBJECT-TYPE IDENTIFIER="_GJB8cWcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00" LONG-NAME="Requirement Type">
      <SPEC-ATTRIBUTES>
        <ATTRIBUTE-DEFINITION-STRING IDENTIFIER="_GJB8cmcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00" LONG-NAME="Description">
          <TYPE>
            <DATATYPE-DEFINITION-STRING-REF>_GJB8cGcAEeOuAMNg8dJKYg</DATATYPE-DEFINITION-STRING-REF>
          </TYPE>
        </ATTRIBUTE-DEFINITION-STRING>
        <ATTRIBUTE-DEFINITION-STRING IDENTIFIER="_JV6vMGcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:38.580+01:00" LONG-NAME="ID">
          <TYPE>
            <DATATYPE-DEFINITION-STRING-REF>_KHHioGcAEeOuAMNg8dJKYg</DATATYPE-DEFINITION-STRING-REF>
          </TYPE>
        </ATTRIBUTE-DEFINITION-STRING>
      </SPEC-ATTRIBUTES>
    </SPEC-OBJECT-TYPE>
    <SPECIFICATION-TYPE IDENTIFIER="_GJB8c2cAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00" LONG-NAME="Specification Type">
      <SPEC-ATTRIBUTES>
        <ATTRIBUTE-DEFINITION-STRING IDENTIFIER="_GJB8dGcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00" LONG-NAME="Description">
          <TYPE>
            <DATATYPE-DEFINITION-STRING-REF>_GJB8cGcAEeOuAMNg8dJKYg</DATATYPE-DEFINITION-STRING-REF>
          </TYPE>
        </ATTRIBUTE-DEFINITION-STRING>
      </SPEC-ATTRIBUTES>
    </SPECIFICATION-TYPE>
  </SPEC-TYPES>
  <SPEC-OBJECTS>
    <SPEC-OBJECT IDENTIFIER="_GJB8dWcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:48:11.540+01:00">
      <VALUES>
        <ATTRIBUTE-VALUE-STRING THE-VALUE="ITEM_1.1.1">
          <DEFINITION>
            <ATTRIBUTE-DEFINITION-STRING-REF>_GJB8cmcAEeOuAMNg8dJKYg</ATTRIBUTE-DEFINITION-STRING-REF>
          </DEFINITION>
        </ATTRIBUTE-VALUE-STRING>
        <ATTRIBUTE-VALUE-STRING THE-VALUE="1.1.1">
          <DEFINITION>
            <ATTRIBUTE-DEFINITION-STRING-REF>_JV6vMGcAEeOuAMNg8dJKYg</ATTRIBUTE-DEFINITION-STRING-REF>
          </DEFINITION>
        </ATTRIBUTE-VALUE-STRING>
      </VALUES>
      <TYPE>
        <SPEC-OBJECT-TYPE-REF>_GJB8cWcAEeOuAMNg8dJKYg</SPEC-OBJECT-TYPE-REF>
      </TYPE>
    </SPEC-OBJECT>
    <SPEC-OBJECT IDENTIFIER="_V2ZNMGcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:48:26.564+01:00">
      <VALUES>
        <ATTRIBUTE-VALUE-STRING THE-VALUE="1.1.2">
          <DEFINITION>
            <ATTRIBUTE-DEFINITION-STRING-REF>_JV6vMGcAEeOuAMNg8dJKYg</ATTRIBUTE-DEFINITION-STRING-REF>
          </DEFINITION>
        </ATTRIBUTE-VALUE-STRING>
        <ATTRIBUTE-VALUE-STRING THE-VALUE="ITEM_1.1.2">
          <DEFINITION>
            <ATTRIBUTE-DEFINITION-STRING-REF>_GJB8cmcAEeOuAMNg8dJKYg</ATTRIBUTE-DEFINITION-STRING-REF>
          </DEFINITION>
        </ATTRIBUTE-VALUE-STRING>
      </VALUES>
      <TYPE>
        <SPEC-OBJECT-TYPE-REF>_GJB8cWcAEeOuAMNg8dJKYg</SPEC-OBJECT-TYPE-REF>
      </TYPE>
    </SPEC-OBJECT>
  </SPEC-OBJECTS>
  <SPECIFICATIONS>
    <SPECIFICATION IDENTIFIER="_GJB8dmcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00" LONG-NAME="Specification Document">
      <VALUES>
        <ATTRIBUTE-VALUE-STRING THE-VALUE="Requirements Document">
          <DEFINITION>
            <ATTRIBUTE-DEFINITION-STRING-REF>_GJB8dGcAEeOuAMNg8dJKYg</ATTRIBUTE-DEFINITION-STRING-REF>
          </DEFINITION>
        </ATTRIBUTE-VALUE-STRING>
      </VALUES>
      <TYPE>
        <SPECIFICATION-TYPE-REF>_GJB8c2cAEeOuAMNg8dJKYg</SPECIFICATION-TYPE-REF>
      </TYPE>
      <CHILDREN>
        <SPEC-HIERARCHY IDENTIFIER="_GJB8d2cAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00">
          <OBJECT>
            <SPEC-OBJECT-REF>_GJB8dWcAEeOuAMNg8dJKYg</SPEC-OBJECT-REF>
          </OBJECT>
        </SPEC-HIERARCHY>
        <SPEC-HIERARCHY IDENTIFIER="_V2jlQGcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:48:11.523+01:00">
          <OBJECT>
            <SPEC-OBJECT-REF>_V2ZNMGcAEeOuAMNg8dJKYg</SPEC-OBJECT-REF>
          </OBJECT>
        </SPEC-HIERARCHY>
      </CHILDREN>
    </SPECIFICATION>
  </SPECIFICATIONS>
</REQ-IF-CONTENT>
</CORE-CONTENT>
<TOOL-EXTENSIONS>
<REQ-IF-TOOL-EXTENSION>
  <configuration:ProrToolExtension>
    <configuration:specViewConfigurations>
      <configuration:ProrSpecViewConfiguration specification="_GJB8dmcAEeOuAMNg8dJKYg">
        <configuration:columns>
          <configuration:Column label="ID"/>
          <configuration:Column label="Description" width="530"/>
        </configuration:columns>
        <configuration:leftHeaderColumn>
          <configuration:Column label="Lead Header Column" width="30"/>
        </configuration:leftHeaderColumn>
      </configuration:ProrSpecViewConfiguration>
    </configuration:specViewConfigurations>
    <configuration:generalConfiguration>
      <configuration:ProrGeneralConfiguration>
        <configuration:labelConfiguration>
          <configuration:LabelConfiguration>
            <defaultLabel>Description</defaultLabel>
          </configuration:LabelConfiguration>
        </configuration:labelConfiguration>
      </configuration:ProrGeneralConfiguration>
    </configuration:generalConfiguration>
  </configuration:ProrToolExtension>
</REQ-IF-TOOL-EXTENSION>
</TOOL-EXTENSIONS>
</REQ-IF>

Okay, most of the XML, you don't have to touch. The actual requirements information is provided in the form of SpecObjects:

    <SPEC-OBJECT IDENTIFIER="_V2ZNMGcAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:48:26.564+01:00">
      <VALUES>
        <ATTRIBUTE-VALUE-STRING THE-VALUE="1.1.2">
          <DEFINITION>
            <ATTRIBUTE-DEFINITION-STRING-REF>_JV6vMGcAEeOuAMNg8dJKYg</ATTRIBUTE-DEFINITION-STRING-REF>
          </DEFINITION>
        </ATTRIBUTE-VALUE-STRING>
        <ATTRIBUTE-VALUE-STRING THE-VALUE="ITEM_1.1.2">
          <DEFINITION>
            <ATTRIBUTE-DEFINITION-STRING-REF>_GJB8cmcAEeOuAMNg8dJKYg</ATTRIBUTE-DEFINITION-STRING-REF>
          </DEFINITION>
        </ATTRIBUTE-VALUE-STRING>
      </VALUES>
      <TYPE>
        <SPEC-OBJECT-TYPE-REF>_GJB8cWcAEeOuAMNg8dJKYg</SPEC-OBJECT-TYPE-REF>
      </TYPE>
    </SPEC-OBJECT>

In this case, the ID is "1.1.2" and the value is "ITEM_1.1.2". Each SpecObjects needs a unique ID. It would be really cool if you could generate a unique one that is reproducible. I.e, instead of _V2ZNMGcAEeOuAMNg8dJKYg to use something like

SUBSET-26-3-1.1.2

Creating SpecObjects does not create the requirements structure. This is done with the SpecHierarchies:

        <SPEC-HIERARCHY IDENTIFIER="_GJB8d2cAEeOuAMNg8dJKYg" LAST-CHANGE="2013-12-17T10:46:26.093+01:00">
          <OBJECT>
            <SPEC-OBJECT-REF>_GJB8dWcAEeOuAMNg8dJKYg</SPEC-OBJECT-REF>
          </OBJECT>
        </SPEC-HIERARCHY>

You can just insert as many as you like to get a flat list. If you would like to get fancy, you can build a hierarchy.

That's it in a nutshell. If you have questions, just ask me. The ReqIF Specification may also be helpful. If you need more attributes, besides ID and Content, I can generate a new Stub for you. Fancy datatypes (enumerations, rich text, etc.) are available as well.

BerndHekele commented 10 years ago

@LaurentFerier @jastram Sorry for changing direction again. Please, don't spent effort right now. Jan has Doors 9.5 available now and started to trial the import os the SRS via Doors. The result looks really promissing. Since he can also handle pictures and since the result even with a first easy procedure looks realy good I believe we shall take this procedure as a basis for the SRS document.

Sorry for confusion Bernd

BerndHekele commented 10 years ago

@jastram Hi Michael, can you prepare the modelling repository for access with ProR?

Jan is about to upload the SRS from Doors to ProR and it would be nice to have it shared as soon as possible?

jastram commented 10 years ago

Hello Bernd,

OK, I'll go ahead with this. This can be tracked via #223 .

jastram commented 10 years ago

Hello Bernd,

To do this, I need to be committer to the modeling project. Could you please initiate the process? Thanks!

Michael

On 18.12.2013 12:40, Bernd Hekele wrote:

@jastram https://github.com/jastram Hi Michael, can you prepare the modelling repository for access with ProR?

Jan is about to upload the SRS from Doors to ProR and it would be nice to have it shared as soon as possible?

— Reply to this email directly or view it on GitHub https://github.com/openETCS/toolchain/issues/218#issuecomment-30835146.

formalmind science for systems engineering Dr. Michael Jastram Universitätsstr. 1 Building 25.12.01.28 D-40225 Düsseldorf Geschäftsführer / CEO +49 (162) 274 83 94 www.formalmind.com http://www.formalmind.com michael.jastram@formalmind.com mailto:michael.jastram@formalmind.com

janWelte commented 10 years ago

Dear Michael, dear Bernd,

I've loaded all 8 Chapters of Subset 26 in a Doors-DB via the Doors Word-Document import. The result looks relatively good in Doors (despite all the layout issues). Sadly, my first try of a ReqIF Export does not look satesfying in ProR. The Attributes are ForeigCreation and Modification inclide a long list of information, which are shown in plain text, while the grafics (OLE-Objects) are not included.

Maybe someone of you now, how the ProRintegration can be improved by some specific parameters at the ReqIF Export.

I have uploaded my first try of an ReqIF Export from Doors in my fork of the Modeling Repository (as I was not able to commit it to the main repository). Please feel free to check it out.

https://github.com/janWelte/modeling/tree/master/WorkDocuments/Subset26-ReqIF

Best regards Jan

jastram commented 10 years ago

Hi Jan,

I'll look what I can do today or tomorrow. The view can be adapted and doing so may improve things (didn't have a look yet). Stay tuned...

janWelte commented 10 years ago

Thanks Michael,

do not hurry. It just think you could have some ideas how the results gets much better with nearly no changes.

jastram commented 10 years ago

Michael to look at it and prettyfy

jastram commented 10 years ago

Jan,

So after having talked about this in Munich, how do we proceed? We certainly could simply move what you have to the modeling repository. But we still have to resolve how to deal with providing the IDs from the Word documents.

My suggestion:

[ ] Move the prettified ReqIF-Documents to the modeling requirement
[ ] Warn users that any modifications to the requirements may get lost, as we may replace these models with new ones eventually.

If you agree, can you take care of these things?

Thanks, Michael

janWelte commented 10 years ago

Dear Michael,

This sounds good.

I'll do it an check, whether I can easily post process the ReqIF File to create the fixed numbers.

Best regards Jan

BerndHekele commented 10 years ago

I dont see a big risk if we do it right. We can check on changes after with Github anyway.

But, we should keep the file organisation so that the SRS is seperated from other requirements.

janWelte commented 10 years ago

The main problem to upload the ReqIF file at the moment seams to be the size. I'll try to splitt the SRS requirements in a number of smaller ReqIFs and post them in a clearly marked area in the requirement part of the model repository.

jastram commented 10 years ago

@janWelte , do you want me to help you? I can split the file, if you would like me to.

jastram commented 10 years ago

@janWelte could you upload your .reqif files to gitHub somewhere, no matter what shape they are in, so that others can look at it? Thanks!

janWelte commented 10 years ago

I've uploaded the ReqIF files as seperate ReqIF for every Chapter (size problem). Please check whether you can open these files perfectly in ProR on your system. (I had problems to see the OLE files.)

Pull mrequest # 8 in modeling repository

After doing the Doors Import now from a RTF file the Headline should allways include the allocated ID from SUBSET 26. I had no time to change the ReqIF files so that the SUBSET 26 ID becomes an own attribute. If someone has the time to do this conversion, it would be great and should only be a small change.

Best regards Jan

jastram commented 10 years ago

@janWelte I can open the files, and they look broadly right to me. I propose to close this issue, but to open new ones for the issues you raise, which are:

OLE objects not visible (Note: To see OLE objects in the first place, ProR Essentials must be installed. But I didn't see the OLE objects, even with the Essentials)
Extract IDs from ReqIF.Text and put them in a separate attribute. We need a script to do this. Language doesn't really matter.
Clean up the Specification View by hiding columns and resizing the ones left.
Bonus: While writing a script, it would be nice to move the images in a subfolder, so that the .reqif files can be found easier.

I will reassign to Cecile for the next steps.

janWelte commented 10 years ago

That sounds great for me. As I'm not sure how to fix the next problems best, that would be good to go the way you proposed and close this issue.

cecilebraun commented 10 years ago

The import is now done

openETCS / toolchain

Subset-26 conversion to ProR #218

Design Spec: Subset-26 conversion to ProR

TODO

Stakeholders

Major Use Cases

Actor(s):

UC-Initial-Subset-Import

UC-Subset-Update

Possible solutions