itsallcode / openfasttrace

Open source requirement tracing suite
GNU General Public License v3.0
100 stars 21 forks source link

third party artefact tracing; includes noticing a traced section has changed but the sequence number hasn't #204

Open froh opened 5 years ago

froh commented 5 years ago

As an architect or requirements engineer tracing third party artifacts I want to specify a range of lines in other files and I want to be able to "freeze" these in order to know the traced third party traced object changed and a dependency needs a review.

For example, if I trace some open source code and I have documented the design of some function, I want to reference an external range of lines. Then the tool needs to "remember" the content of these lines (fingerprint?) so when I update the files, the tool can tell me which traced items have changed, so it helps me to know what I need to review.

Likewise, if I trace third party requirements without clear ids, I'd love to point to specific parts of these documents.

Ideally the memoized chunk is linked to the version id of the requirement. so if I update the id the tool captures the current chunk. which is okay as I now need to propagate the version id change in my tracing.

chku2469 commented 5 years ago

Hello froh,

I am not perfectly sure, if I understood correctly what you want to do.

if I trace some open source code

In my understanding you should not "trace" code, but rather documents. Source code in my eyes should rather be the leafs of a tracing hierarchy.

Another item that I am not sure whether I understood correctly is

when I update the files

Which files do you update? The source code of third party artifacts? Does that imply that you are the "third party" yourself?

redcatbear commented 5 years ago

Hi @froh,

let me re-phase in my own words so that I see if I understood the intent.

  1. You'd like to reference 3rd party artifacts that do not follow OFT's specification item definition.
  2. Between two trace runs you'd like to get informed if anything changed in the 3rd party artifact in order to see that you need to update the dependent specification items.

Is that correct?

In this case I'd like to propose a more robust mechanism then referencing line numbers:

In both cases this requires a dedicated importer for the specific format (e.g. a C++ code importer and an PDF importer).

As for realizing that the tracing chain needs an update:

If the document / code is sectioned and we reference a section / function then the importer can hash the section's / function's contents and make the hash part of the ID. In this case changes in the section / function break the chain of linked requirements.

Does this satisfy the user-level requirement you have in mind?

froh commented 5 years ago
  1. You'd like to reference 3rd party artifacts that do not follow OFT's specification item definition.
  2. Between two trace runs you'd like to get informed if anything changed in the 3rd party artifact in order to see that you need to update the dependent specification items.

Is that correct?

yup.

Regarding the more robust part: that is what's needed behind the scenes.

As a user, I just want to reference line numbers (or section numbers or html ids or function names, human readable identifiers).

So the specitem fingerprints are an openfasttrace internal information. If there are no fingerprints yet, OFT will generate those and store them for itself. If there are fingerprints, it will check if they still match the current code.

Not sure if excerpts are even more useful as they allow to diff. but then a diff could also be generated from the VCS underneath the traced external source. So OFT just identifies: this specitem has changed! and the user then uses other, more appropriate tooling to identify and review the actual change.

Actually I believe this power to memoize the contents of trace items between runs may also be helpful if the specitem tags are right in the document, OFT style. Not sure about the convention, by filetype, how a specitem label identifies it's context. upt to the next tag? or optionally also relative line numbers? what is more intuitive and convenient?

Whatever it is: knowing the refenced context allows openfasttrace to identify the trace item has changed, with the same mechanism as above, and it can help to identifyif the specitem version id needs a review. maybe the fingerprint needs an update (just editing, no semantic change). maybe the semantics have changed (update version).

froh commented 5 years ago

In my understanding you should not "trace" code, but rather documents. Source code in my eyes should rather be the leafs of a tracing hierarchy.

I see. Hm. Which verb do I use to express "I describe and then monitor the change of specitems, i.e. snippets of documents?" that's what I tried to say with "trace".

For source code, the compiler and the tests will do this for me. If I change stuff and there are mismatches, the compiler will tell me or the tests will tell me. They monitor everything is in place (all necessary items are there) and the semantics didn't change.

Now for human documents we use another mechanism. As the machine doesn't understand the human language cross references, and even less so semantics, we manually tag the cross references (specitem ids). The tool notices changes, and we review changes, and tell the tool: now everything is fine again.

This use case here focuses on reviewing changes, and on more specifically identifying the tagged specitem as a specific part of a document. So the changes int he document are related not to all specitems throughout the document, but only to the specitem the change is part of.

Another item that I am not sure whether I understood correctly is

when I update the files

Which files do you update? The source code of third party artifacts?

correct.

Does that imply that you are the "third party" yourself?

Not necessarily. For example I want to make a statement about say, openSSL. so I describe openSSL artefacts, reconstruct their design and some architecture. Now openSSL releases a new version. did they change the design? the architecture?

Now if openfasttrace "knows" which specific 3rd party parts are described in which of my arch and design documents (or test cases), it can help me to focus on just the parts that have changed.

chku2469 commented 5 years ago

Thanks, understood.