🐛 [BUG] - Overwhelming Info Output from Unreferenced Definitions

Coffee2Bits commented 12 months ago

Pre-requisites

[X] I checked the documentation and found no answer.
[X] I checked to make sure that this issue has not already been filed.

What version of the AaC package are you using? Check your version with the command: aac version.

0.2.17

Description

The new validation that checks for unreferenced definitions is producing overwhelming output. It's a combination of issues including false-positives and some edge cases. When I run validation on the usecase for the alarm clock model I get this massive output:


Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_plant_uml/gen_plant_uml.yaml. 
No references to 'Generate PlantUML Diagrams' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/model/alarm_clock/usecase.yaml. 
No references to 'Trigger Alarm' in the language context.

Validation ERROR from 'Usecase sources and targets refer to defined participants' in 
/workspace/AaC/python/model/alarm_clock/usecase.yaml. Source 'alarm' of step 'The alarm is 
activated to emit a noise from the AlarmClock speaker' does not refer to a participant of the usecase.

Validation ERROR from 'Usecase sources and targets refer to defined participants' in 
/workspace/AaC/python/model/alarm_clock/usecase.yaml. Target 'alarm' of step 'The timer 
emits an alarm notification to the alarm' does not refer to a participant of the usecase.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/unique_requirement_ids/unique_requirement_ids.yaml. 
No references to 'Validate requirement IDs are not duplicated.' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_protobuf/gen_protobuf.yaml. No references 
to 'SchemaMessageOptions' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_design_doc/gen_design_doc.yaml. No 
references to 'Generate System Design Document' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/model/alarm_clock/structures.yaml. No references to 'my_alert' in the 
language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/spec/spec.yaml. No references to 'Backend' in the language 
context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/reference_fields/reference_fields.yaml. No 
references to 'Validate query reference format' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/root_keys/root_keys.yaml. No references to 
'Validate root keys' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_gherkin_behaviors/gen_gherkin_behaviors.yaml. 
No references to 'Generate Gherkin Feature Files' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/unique_names/unique_names.yaml. No references 
to 'Unique definition names' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/reference_targets/reference_targets.yaml. No 
references to 'Reference target valid' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_protobuf/gen_protobuf.yaml. No references 
to 'Map' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/exclusive_fields/exclusive_fields.yaml. No 
references to 'Mutually exclusive fields' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/unused_definitions/unused_definitions.yaml. No 
references to 'Validate definition references' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/validator_implementation/validator_implementation.yaml. 
No references to 'Validation definition has an implementation' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_plugin/gen_plugin.yaml. No references to 
'CommandArgumentFields' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/defined_references/defined_references.yaml. No 
references to 'Type references exist' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/validator_implementation/validator_implementation.yaml. 
No references to 'Validate implementations for validators' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/usecase_participants/usecase_participants.yaml. 
No references to 'Validate usecase participants' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/spec/spec.yaml. No references to 'Generation' in the language 
context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_protobuf/gen_protobuf.yaml. No references 
to 'Generate Protobuf Messages' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/unique_names/unique_names.yaml. No references 
to 'Validate names are not duplicated' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/spec/spec.yaml. No references to 'Query' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/spec/spec.yaml. No references to 'Internal' in the language 
context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/reference_fields/reference_fields.yaml. No 
references to 'Reference format valid' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/requirement_references/requirement_references.yaml. 
No references to 'Validate referenced IDs' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/model/alarm_clock/structures.yaml. No references to 
'AddTimerAlertRootNode' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/unused_definitions/unused_definitions.yaml. No 
references to 'Definition is referenced' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/required_fields/required_fields.yaml. No 
references to 'Required fields are present' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/subcomponent_type/subcomponent_type.yaml. No 
references to 'Subcomponents are models' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/required_fields/required_fields.yaml. No 
references to 'Validate required fields' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/usecase_participants/usecase_participants.yaml. 
No references to 'Usecase sources and targets refer to defined participants' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_plugin/gen_plugin.yaml. No references to 
'Generate Plugin' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/reference_targets/reference_targets.yaml. No 
references to 'Validate query reference targets' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/defined_references/defined_references.yaml. No 
references to 'Validate type references' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_protobuf/gen_protobuf.yaml. No references 
to 'ProtobufPrimitiveTypesExtension' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/spec/spec.yaml. No references to 'Management' in the language 
context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/root_keys/root_keys.yaml. No references to 
'Root key is defined' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/exclusive_fields/exclusive_fields.yaml. No 
references to 'Validate mutually exlusive fields' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/requirement_references/requirement_references.yaml. 
No references to 'Referenced IDs exist' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_json/gen_json.yaml. No references to 
'Generate JSON' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/subcomponent_type/subcomponent_type.yaml. No 
references to 'Validate model subcomponents' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/model/alarm_clock/usecase.yaml. No references to 'Set Alarm Time' in the 
language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/cli/builtin_commands/specifications/specifications.yaml. No 
references to 'Specifications' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/first_party/gen_protobuf/gen_protobuf.yaml. No references 
to 'EnumMessageOptions' in the language context.

Validation INFO from 'Validate definition references' in 
/workspace/AaC/python/src/aac/plugins/validators/unique_requirement_ids/unique_requirement_ids.yaml. 
No references to 'Requirement ID is unique' in the language context.

Reproduction URL

https://github.com/DevOps-MBSE/AaC

Reproduction steps

Please enter an explicit description of your issue
1. Open the repo
2. Run `aac validate model/alarm_clock/usecase.yaml'
3. Get overwhelmed.

Screenshots

No response

Logs

Not helpful

Pertinent Files

In repo

lizzcondrey commented 8 months ago

Not currently being experienced in 0.4.0 because the constrainer was not brought forward and converted. The conversion task for this is #741 . That potentially the conversion could make use of the dataref primitive constraint since it is the constraint that runs when checking referenced definitions.

If it is decided that the legacy validator needs to be reimplemented as a constraint, want to be sure that the output remains helpful and does not become overwhelming again.

Scope to only run within "user editable" (in legacy AaC there was a field entry for flagging this, not sure if that was carried forward in 0.4.0 work) files, or user defined files, not to entire language context and therefore ALL definitions within AaC when running check on a file. Scope to only checking within the passed in file and any imported files maybe?

lizzcondrey commented 1 day ago

Based on the results of the 0.4.0 implementation and the current thoughts on the unreferenced constraint not being needed currently, this issue is being closed.

DevOps-MBSE / AaC