Validate `<<express:{entity},{render}>>` links that `entity` is an EXPRESS ENTITY before converting to concept mention

[ronaldtse]

The issue is described in https://github.com/metanorma/iso-10303-2/pull/102#issuecomment-1243148727 .

The express_reference_to_mention method needs to check whether the {entity} is really an EXPRESS ENTITY. If not, this reference does not exist in the ISO 10303-2 collection (because only ENTITY entries are included).

This requires stepmod-utils to parse the entire STEPmod repository initially (so we can query it) rather than parsing individual schema files one by one.

[ronaldtse]

From original issue:

Not ENTITY

Pre_defined_activity_relationship_types: NOT AN ENTITY Pre_defined_address_assignment_types: NOT AN ENTITY Pre_defined_approval_status_types: NOT AN ENTITY Pre_defined_approval_types: NOT AN ENTITY Pre_defined_date_or_date_time_assignment_types: NOT AN ENTITY Pre_defined_organization_or_person_in_organization_assignment_types: NOT AN ENTITY Pre_defined_security_classification_types: NOT AN ENTITY Pre_defined_work_request_status_types: NOT AN ENTITY Pre_defined_work_request_types: NOT AN ENTITY a3m_accuracy_associated_target_select: NOT AN ENTITY accuracy_associated_target_select: NOT AN ENTITY action_method_to_select_from: NOT AN ENTITY action_request_item: NOT AN ENTITY additional_contexts: NOT AN ENTITY affected_item_select: NOT AN ENTITY analysis_model_idealisation_item: NOT AN ENTITY ap242_assignment_object_select: NOT AN ENTITY assembly_product_design_object_category: NOT AN ENTITY assigned_document: NOT AN ENTITY assignment_object_select: NOT AN ENTITY associated_definition: NOT AN ENTITY base_solid: NOT AN ENTITY cartesian_transformation: NOT AN ENTITY characteristic_data_table_cell: NOT AN ENTITY classification_item: NOT AN ENTITY clgc_with_dimension: NOT AN ENTITY complex_product_select: NOT AN ENTITY connected_terminals: NOT AN ENTITY constraining_part: NOT AN ENTITY containing_set: NOT AN ENTITY count_measure: NOT AN ENTITY curve_length: NOT AN ENTITY curve_or_surface: NOT AN ENTITY date_and_time_item: NOT AN ENTITY date_item: NOT AN ENTITY date_or_date_time_select: NOT AN ENTITY depth: NOT AN ENTITY derived_from: NOT AN ENTITY derived_property_select: NOT AN ENTITY design_intent: NOT AN ENTITY dimension_value: NOT AN ENTITY directrix: NOT AN ENTITY document_identifier_assigned_item: NOT AN ENTITY document_reference_item: NOT AN ENTITY edge_end: NOT AN ENTITY edge_start: NOT AN ENTITY electrical_conductivity_characteristic: NOT AN ENTITY element_component_classification_item: NOT AN ENTITY event_occurrence_item: NOT AN ENTITY experience_item: NOT AN ENTITY experience_type_item: NOT AN ENTITY expression_assignment_item: NOT AN ENTITY external_identification_item: NOT AN ENTITY extrusion_axis: NOT AN ENTITY feature_or_non_feature_usage: NOT AN ENTITY feature_shape: NOT AN ENTITY generic_property_definition_select: NOT AN ENTITY geometric_constraint_element: NOT AN ENTITY groupable_item: NOT AN ENTITY hole_bottom_condition_select: NOT AN ENTITY identification_item: NOT AN ENTITY idrm_marked_item: NOT AN ENTITY initial_context: NOT AN ENTITY iso13584_expressions_schema: NOT AN ENTITY item_identified_representation_usage_definition: NOT AN ENTITY justification_item: NOT AN ENTITY justification_support_item: NOT AN ENTITY kinematic_topology_representation_select: NOT AN ENTITY lam_product_design_object_category: NOT AN ENTITY land_physical_template_or_inter_stratum_feature_template: NOT AN ENTITY length_value: NOT AN ENTITY limited_value: NOT AN ENTITY linear_geometry_element: NOT AN ENTITY location_groupable_item: NOT AN ENTITY location_item: NOT AN ENTITY location_representation_item: NOT AN ENTITY logical_literal: NOT AN ENTITY mapping_source: NOT AN ENTITY maths_function: NOT AN ENTITY measure_value: NOT AN ENTITY name_item: NOT AN ENTITY name: NOT AN ENTITY number_of_defects: NOT AN ENTITY number_of_inspected_instances: NOT AN ENTITY number_of_inspections: NOT AN ENTITY number_of_quality_defects_detected: NOT AN ENTITY organization_item: NOT AN ENTITY organization_type_item: NOT AN ENTITY person_and_organization_item: NOT AN ENTITY person_organization_item: NOT AN ENTITY person_type_item: NOT AN ENTITY plane_angle_measure: NOT AN ENTITY pocket_depth: NOT AN ENTITY position_in_organization_group_assignment_item: NOT AN ENTITY position_in_organization_group_item: NOT AN ENTITY position_in_organization_item: NOT AN ENTITY position_in_organization_type_item: NOT AN ENTITY pre_defined_type_qualifiers: NOT AN ENTITY product_data_equivalence_inspection_result_schema: NOT AN ENTITY product_select: NOT AN ENTITY project_group_name_item: NOT AN ENTITY quantity_criterion: NOT AN ENTITY radial_geometry_element: NOT AN ENTITY related_product_concept_feature: NOT AN ENTITY related_product_definition: NOT AN ENTITY related_shape_aspect: NOT AN ENTITY related_view: NOT AN ENTITY related: NOT AN ENTITY relating_product_concept_feature: NOT AN ENTITY relating_product_definition: NOT AN ENTITY relating_shape_aspect: NOT AN ENTITY relating_view: NOT AN ENTITY relation_type: NOT AN ENTITY representation_or_representation_reference: NOT AN ENTITY represented_definition: NOT AN ENTITY requirement_satisfaction_item: NOT AN ENTITY rigid_placement: NOT AN ENTITY security_classification_item: NOT AN ENTITY shape_model_element: NOT AN ENTITY shapeable_item: NOT AN ENTITY shell: NOT AN ENTITY simple_Boolean_expression: NOT AN ENTITY start_knot_index: NOT AN ENTITY sto_1: NOT AN ENTITY sto_2: NOT AN ENTITY stratum_assembly_relationship_or_stratum_technology_occurrence: NOT AN ENTITY stratum_concept: NOT AN ENTITY stratum_surface_designation: NOT AN ENTITY stratum_thickness: NOT AN ENTITY swept_area: NOT AN ENTITY swept_curve: NOT AN ENTITY target_annotated_3d_model_select: NOT AN ENTITY task_item: NOT AN ENTITY thickened_face_solid: NOT AN ENTITY thickness: NOT AN ENTITY time_interval_item: NOT AN ENTITY traces_from: NOT AN ENTITY traces_to: NOT AN ENTITY unary_Boolean_expression: NOT AN ENTITY unit: NOT AN ENTITY usage_concept: NOT AN ENTITY used_package: NOT AN ENTITY work_item: NOT AN ENTITY

[HassanAkbar]

@ronaldtse From what I understand we are converting references like <<express:Activity_method_arm.Activity_method,Activity Method>> to {{Activity_method_arm.Activity_method,Activity Method}} if it is an Express ENTITY.

My question is what should we do if it is not an Express ENTITY? e.g

<<express:Pre_defined_product_data_management_specializations_arm.Pre_defined_organization_or_person_in_organization_assignment_types,Pre_defined_organization_or_person_in_organization_assignment_types>>

Should I just remove it or skip the conversion and leave it as it is?

[TRThurman]

It needs to be converted.

[ronaldtse]

Yes @HassanAkbar , as mentioned by @TRThurman , all EXPRESS <<express:xxx,yyy>> need to be converted into {{xxx,yyy}}.

[HassanAkbar]

Validate <<express:{entity},{render}>> links that entity is an EXPRESS ENTITY before converting to concept mention

@ronaldtse I'm not following here. What is the purpose of this ticket? What should I do after validation if it is not an EXPRESS ENTITY?

[ronaldtse]

Actually, @HassanAkbar, our approach has changed since the original ticket was filed.

Right now, we just convert the non-Entities into {{xxx,yyy}}.

Previously, we had this flow for ISO 10303-2:

1. Convert STEPmod schemas into Annotated EXPRESS
2. From Annotated EXPRESS, we build the Metanorma AsciiDoc files.
3. From Metanorma AsciiDoc files, the ISO 10303-2 document is created

Now, the flow has changed ("new flow"):

1. Convert STEPmod schemas into Annotated EXPRESS
2. From Annotated EXPRESS, we build a Glossarist Concept Collection.
3. Metanorma reads the Glossarist Concept Collection, and builds the ISO 10303-2 document (This step is to be done, let's call this the "TODO")

The reason why we wanted to validate that the <<express:...>> references lead to an EXPRESS Entity is because:

• In ISO 10303-2, only EXPRESS Entities are present as "Concepts". This means that references for non-Entities will lead to Metanorma generating errors for "broken concept mentions" (because if the target is not an EXPRESS Entity, it is not present in ISO 10303-2)

In the "new flow" step 1, in the Annotated EXPRESS, we want to always preserve these links, so there is no need to remove these mentions.

With the new flow, this error check can be done in the "TODO" step (binding between Metanorma and the Glossarist Concept Collection), because it is expected that a Glossarist Collection may contain concepts not available in a Metanorma document.

Therefore there is no longer a need to validate that <<express:xxx...>> cross-references (which are turned into {{xxx...}) point to EXPRESS Entities. But we will still need to perform the conversion from <<express:xxx...>> into {{xxx...}.

[HassanAkbar]

@ronaldtse Just to make sure I'm going in the right direction.

What I need to do here is change the metanorma site generate command to generate the site using glossarist yaml files instead of .adoc files?

[ronaldtse]

What I need to do here is change the metanorma site generate command to generate the site using glossarist yaml files instead of .adoc files?

In effect yes, but the method of implementation is this:

• https://github.com/metanorma/metanorma-plugin-glossarist/issues/1

[HassanAkbar]

@ronaldtse the attached issue is complete, Is there anything left to do in this ticket?