DSL evaluation for TRADES - please take a look

[dzw1999]

Hello, greetings from the software engineering group from Beihang University, China. We have been working on the evaluation of Domain Specific Modeling Languages (DSMLs ), the main purpose is to find out the flaws of the design of the DSML. To test our evaluation framework, we evaluated your software design, mainly based on your ecore and odesign files, and found out that even though the overall quality of your software design is very good, there still exists some minor problems. The details are as follows:

rule-based evaluation

Firstly, we did some rule-based evaluation, mainly focusing on the graphic part.

Evaluation of Semiotic Clarity

We've checked your odesign file according to Goodman's theory of symbols[Goodman N. Languages of Art: An Approach to a Theory of Symbols[M]. Hackett publishing, Indianapolis, 1976.], and there are some minor problems that can decrease the semiotic clarity of your design as follows:

1. Excess

   There are some graphic symbols in your design that does not represent any of the elements of your language.

   • TRADES\bundles\dsm.oscal.design\icons\add.png
   • TRADES\bundles\dsm.oscal.design\icons\edit_template.png
   • TRADES\bundles\dsm.oscal.design\icons\folderType_filter.png

2. Deficit Elements

   There are some elements in your design that are not represented by any graphic symbols.

   Certainly, we know that some elements do not need to be assigned symbols to, but we believe it is best to check whether these elements really do not need symbols or they were just overlooked.

   • EEFExtHTMLRendererDescription
   • EEFExtMarkdownWidget
   • Group
   • ParameterConstraint
   • ParameterConstraintTest
   • ParameterGuideline
   • ParameterSelection
   • Address
   • BackMatter
   • BackMatterResource
   • Base64
   • Base64Type
   • ControlOwner
   • DateTimeType
   • DateTimeWithTzType
   • DateType
   • DateWithTzType
   • DocumentId
   • DocumentationComputer
   • ElementWithClazz
   • ElementWithId
   • ElementWithRemarks
   • ElementWithValue
   • ExternalId
   • Hash
   • IpV4AddressType
   • IpV6AddressType
   • LinkOwner
   • Location
   • MarkupLineType
   • MarkupMultilineType
   • OscalElement
   • ParameterOwner
   • PartOwner
   • Party
   • PropertyOwner
   • ResourceCitation
   • ResourceRlink
   • ResponsibleParty
   • ResponsibleRole
   • Revision
   • Role
   • TelephoneNumber
   • UUIDElement
   • UriReferenceType
   • UriType
   • UuidType
   • ExtHTMLRendererDescription
   • ExtMarkdownDescription
   • AssessmentENUM
   • AffectedENUM
   • threatTypeENUM
   • ScoreSystem
   • NamedElement
   • ExternalThreat
   • ExternalElement
   • ImpactConfiguration
   • RGBColor
   • ExternalControl
   • ThreatsOwner
   • ControlOwner
   • AbstractControlOwner
   • ControlStatusENUM
   • AbstractThreatOwner
   • IThreatDefinition
   • IControlDefinition
   • ICatalogDefinition
   • IMitigationLink
   • ColoredElement
   • IElementWithSource
   • ElementWithId

LLMS-based evaluation

Secondly, we did some LLMS-based evaluation, mainly focusing on the model part. Generally speaking, we try to make LLM act as a domain expert to provide multidimensional evaluations of your language design.

Model Completeness

We give your ecore design to LLM and ask the LLM to guess which domain your language is designed for. And then we ask LLM to add possible missing elements to your language (perhaps not taken into account in the first version of the design, but can be considered for inclusion in future versions). The results are as follows:

Based on the meta models you provided and their relationships, these models appear to belong to the field of information security, particularly those related to risk assessment, threat modeling, and security control. Next, I will supplement some key meta models and their relationships that may be lost according to the required format.

Element:

Vulnerability: description; Mitigation;
Risk: level; Description; Likelihood; Impact;
RiskAssessment: methodology; AssessmentDate; Assessor;
Security Requirement: category; Description;
Security Policy: description; Title;
MitigationStrategy: description; Effectiveness;
Incident Response Plan: procedure; ResponsibleParty;
Security Standard: standardName; Version; PublicationDate;

Relationship:

Vulnerability (1)<- effects ->Component (n)
Risk (1)<- identifiedIn ->Component (n)
Risk (1)<- associated With ->Vulnerability (n)
RiskAssessment (1)<- evaluations ->Risk (n)
Security Requirement (1)<- addresses ->Risk (n)
Security Requirement (1)<- derived From ->Security Policy (1)
MitigationStrategy (1)<- initiatives ->Vulnerability (n)
Component (1)<- compliesWith ->Security Standard (n)
Control (1)<- elements ->Security Requirement (n)
Incident Response Plan (1)<- activesInResponse To ->Incident (n)
Party (1)<- assignedResponsibilityFor ->MitigationStrategy (n)

These supplements reflect a structured approach to analyzing, responding to, and managing information security risks while ensuring compliance with specific security policies and standards. It covers the complete process of identifying threats and vulnerabilities, planning and implementing corresponding mitigation measures, and establishing accident response plans and security requirements.

We are not sure if these issues actually constitute a problem, the decision to fix them or not is still up to your team, looing forward to your respond, thanks!

[ASH-SYSTEMS]

Thank you for this. Please feel free to approach us using the email below if you wish to further discuss this. avi.shaked @ cs.ox.ac.uk

BTW, a more recent version exists in the new repository: https://github.com/UKRI-DSbD/TRADES