Support default values for options per file

[twodrops]

It would be nice to specify need option values not only globally but per file as well.

The default option values is currently available on a per-project basis in sphinx-needs:

needs_global_options = {
      'my_option': 'my_global_value'
}

However, in big projects, requirements with different "standard options" exist, and per-file grouping is a natural strategy to structure them. In projects with some mandatory requirements attributes, repeated options lead to visual noise (repetitive content repeated a lot with little additional information).

Now:

.. need:: 
   :id: NEED_ID_1
   :my_option: same_value
   :some-other-option: value1

.. need:: 
   :id: NEED_ID_2
   :my_option: same_value
   :some-other-option: value2

.. need:: 
   :id: NEED_ID_3
   :my_option: same_value
   :some-other-option: value2

Later:

.. needoptions:
  :my_option: same_value

.. need:: 
   :id: NEED_ID_2
   :some-other-option: value1

.. need:: 
   :id: NEED_ID_2
   :some-other-option: value2

.. need:: 
   :id: NEED_ID_3
   :some-other-option: value2

The needoptions shall apply to all needs in the current page starting where the directive appears. Probably there could be a filter option as well to filter certain need types (within the page).

Idea came from @arwedus :)

[danwos]

I see the benefit of having such a solution and maybe it can be done already with the current version.

But I have also a concern with such a directive like needoptions. If it get used several times on a page and the page is quite huge, it is hard to figure out, which needoptions is responsible for what need. The need may be defined on line 732 and the last needoptions was on line 126.

But let me explain, why I think it is also doable by today. You can use filtered global options: https://sphinxcontrib-needs.readthedocs.io/en/latest/configuration.html#filter-based-global-options. And in the filter you can check for the file (alias docname) or even for the section. Here an example (untested!):

needs_global_options = {
      'my_option': [
          # Value for my_module/index page. Sets also "my_default_value" for ALL not matched needs
          ('my_global_value', 'docname == "my_module/index"', 'my_default_value'),

          # Special value for "my_class" section on page "my_module/index
          # No default here, as it was set already earlier
          ('another_value', 'docname == "my_module/index" and "my_class" in sections'),

          # Value for another page
          ('another_value', 'docname == "my_module/my_class/function"'),
     ]
}

For sure, this list can get quite big. As an alternative, you could set your value with a dynamic_function via global_options.

This function sets the right value then based on docname and/or section name like above. But value and comparison can come from outside, no need to have it as Python code. E.g. a spreadsheet could store this information. Or a json file...

Would this approach also support your use case? Or have I missed some points?

[twodrops]

@danwos Thanks for making an alternate suggestion.

What I feel goes little bit against using needs_global_options in this way would be that, we might be mixing the concerns. conf.py and the needs configuration within it should be agnostic to the folders, files and sections that might be created in the project. It will be tough to keep it in sync if splitted in this way.

What @arwedus and me were thinking is more like an inheritance pattern with file as the scope. If file cannot be technically made the scope, we might have to define how to do the scoping.

I see the usecase in this way. The user wants to create 20 requirements in a page and realizes that, 15 options have the same values for all of them and only 3 options are different. This is a very dynamic expectation from the author which I feel is less of configuration, but done mostly as part of authoring itself. It would be nice if we can workout such an "inheritance pattern".

@arwedus Feel free to extend with your expectations/ideas.

[danwos]

Good points and I agree. conf.py should not be too related to the internal documentation structure.

I also thought that your case is somehow related to inheritance.

Here is an idea:

Need can inherit from any other need

.. need:: My basic need
   :id: basic_001
   :status: open
   :tags: tag_01
   :hide: true

.. need:: Another need
   :id: REAL_NEED_001
   :tags: my_tag
   :inherit: basic_001     

REAL_NEED_001 will have status: open and tags: my_tag.

The user can decide, if a need like basic_001 shall be shown or gets hidden. basic_001 could also get automatically inherited_from: True as option, so that all needs which are used for inheritance could get the hide-option automatically via needs_global_options.

This somehow feels like needextend, but only the other way around.

Beside the overall inheritance idea, wouldn't needextend be a solution for your case?

.. needextend:: docname == "this_page.rst"
  :my_option: same_value

.. need:: 
   :id: NEED_ID_2
   :some-other-option: value1

.. need:: 
   :id: NEED_ID_2
   :some-other-option: value2

.. need:: 
   :id: NEED_ID_3
   :some-other-option: value2

[twodrops]

@danwos I find the inherit idea good and I feel we can keep it aligned with the nested-needs concept, where nested-needs is a "containment" link between needs (parent-child) and inherit is more the "inheritance" link between the needs.

I am thinking if inherit could also help with the variant management concept, where you will have needs defined once and reuses them in variants. Maybe I am thinking a bit too far, but it did ring some bells :)

I also thought needextend is the reverse of inherit, but after seeing your example, I am thinking if this could be an alternative as well. Brilliant 👍🏽 Let me think about it and get back.

[twodrops]

@danwos The needextend trick above worked for me to define some default values for options within a page. You cannot later override them within the needs (to make it more like a inheritance usecase). But I guess this is good enough to start with. @arwedus Please confirm as the initial wish came from your side

[arwedus]

@danwos that trick unfortunately overwrites options which have been set in a certain requirements.

Tried it with 20 requirements of which one hat ASIL C and the other were QM - unfortunately that made my ASIL C requirement a QM requirement ;-).

need-default would be nice.

I do like the filter string syntax though. ad "docname" is not yet available, I helped myself with a requirement prefix that happened to match the file's scope.

[danwos]

Sorry for the really late response.

You are right, needextend overwrites every need, which matches the search string.

Maybe we can solve this by introducing an option like extend_safe, extend_empty or so. If given, only the options, which are not set/empty in the target need get the new value.

We could also introduce need-default, but as it does nearly the same as needextend I would like to keep the number of directives small.

So solution would look like:

.. needextend:: docname == "this_page.rst"
   :extend_empty:   #  <-- this is new
   :my_option: same_value

.. need:: 
   :id: NEED_ID_2
   :some-other-option: value1

.. need:: 
   :id: NEED_ID_2
   :some-other-option: value2

.. need:: 
   :id: NEED_ID_3
   :my_option: My own value   # <-- does not get "same_value" as it is already set

[arwedus]

okay, but "extend_empty" is kind of redundant. How about "if_empty"? Also, can I filter for a type?

To sum up, it would look like this:

.. work:: Component X requirement spec
   :id: WORK_SPEC_COMPONENT_X

   Just an example of another need type in the same document

.. needextend:: (docname == "this_page.rst") and (need_type == "req")
   :if_empty:
   :asil: ASIL_B

.. req:: Some ASIL B requirement
   :id: REQ_ID_2
   :some-other-option: value1

.. req:: Another ASIL B requirement
   :id: REQ_ID_2
   :some-other-option: value2

.. req:: Super safety critical requirement
   :id: REQ_ID_3
   :some-other-option: value2
   :asil: ASIL_D

[danwos]

Yeah, the name "extend_empty" is not perfect.

By the way, you can filter for empty fields already today:

.. needextend:: docname == "this_page.rst" and need_type == "req" and asil is False
   :asil: ASIL_B

Not sure if this filter string is correct. As alternative this one should also work:

.. needextend:: docname == "this_page.rst" and need_type == "req" and asil == ""
   :asil: ASIL_B

And regarding filtering, you can filter for any option or link. See https://sphinxcontrib-needs.readthedocs.io/en/latest/filter.html#filter-string for some ideas.

To get a complete list what is available in your project, simply set the layout of one need to debug:

.. req:: test req
   :id: REQ_0001
   :layout: debug

Details and example: https://sphinxcontrib-needs.readthedocs.io/en/latest/layout_styles.html#EX_DEBUG

So I'm not sure, if we really need an option like is_empty or so.

[arwedus]

@danwos: I already fail at the docnameoption. Example:

I have a document with relative path source/work_products/software_wp/sw_req_eng_wp/sw_requirements_specification_wp.rst and it contains:

Copied DoDs
~~~~~~~~~~~~~~~

.. dod::
    :id: DOD_WORK_SWRS_4

    Software requirements shall be traceable to its source to support bilateral traceability.

Common DoDs
~~~~~~~~~~~~~~~

.. dod:: Requirements linked to upper level
    :id: DOD_WORK_SWRS_23
    :status: draft

    Description
    -----------

    To mark requirements "approved" they need to have a link to an upper-level requirement,
    and/or a system design element.

    See also :need:`DOD_WORK_SWRS_4`.

.. WP requirements from XC compass are automatically approved dod's

.. needextend:: (type == "dod") and  (docname == "sw_requirements_specification_wp.rst")
   :status: approved

Seems like the docname variable just doesn't match anything. E.g. this also doesn't work:

.. needextend:: (type == "dod") and  (docname == "work_products/software_wp/sw_req_eng_wp/sw_requirements_specification_wp.rst")
   :status: approved

Other filters, like a common "links_back", successfully limit the needextend to a certain "document scope". Are you sure docname works as you describe it? Could you add a test, minimal example, and a documentation entry then?

[arwedus]

W.r.t. the original issue, "is False" doesn't work, but "is None" works as intended :-)

If I use this directive, all stati are overwritten (also the one that has explicit value draft):

.. needextend:: (type == "dod")
   :status: approved

If I use this directive filter, nothing is extended again:

.. needextend:: (type == "dod") and  (status is False)
   :status: approved

The following works, i.e. it adds "approved" status to all dod needs which have no status explicitly set, and leaves the others unaltered:

.. needextend:: (type == "dod") and  (status is None)
   :status: approved

[danwos]

Regarding the correct docname, you can check the set value by using the debug layout in one of your need objects:

.. dod::  An example
   :id: DOD_001
   :layout: debug 

Docs: https://sphinxcontrib-needs.readthedocs.io/en/latest/layout_styles.html#EX_DEBUG This shows you also all the other option values.

In your case I think the correct one is work_products/software_wp/sw_req_eng_wp/sw_requirements_specification_wp, without the file extension.

[twodrops]

As @danwos mentioned it works when the folder/filename is used without the file extension.

[arwedus]

@danwos : When I add that to an example need ("dod"), I get in the output no attributes at all, just:

Debug view off

Btw.: Seems to be a common issue, as it also shows in the same way in the sphinx-needs docu: https://sphinxcontrib-needs.readthedocs.io/en/latest/layout_styles.html#layouts

[danwos]

The Debug view off text is a button. Just click it :) Yeah, it is not the best UX and documentation :( I'll create an issue to improve this.

[arwedus]

works like a charm, we should add this to the sphinx-needs docu