Invalid files in STEPmod?

[ronaldtse]

From @zakjan

There are a few files that are invalid EXPRESS:

For example ballots/ballots/ap210_wg12/express/resources/risk_schema.exp:

ENTITY risk_consequence
SUBTYPE OF (representation);
SELF\representation.name : label;
SELF\representation.context_of_items : representation_context;
SELF\representation.items : SET[1:?] OF representation_item;
DERIVE
SELF\representation.description : text := get_description_value (SELF);
SELF\representation.id : identifier := get_id_value (SELF);
WHERE
: SIZEOF (USEDIN (SELF, 'BASIC_ATTRIBUTE_SCHEMA.' + 'ID_ATTRIBUTE.IDENTIFIED_ITEM')) <= 1;
: SIZEOF (USEDIN (SELF, 'BASIC_ATTRIBUTE_SCHEMA.' + 'DESCRIPTION_ATTRIBUTE.DESCRIBED_ITEM')) <= 1;
END_ENTITY;

WHERE domain_rule rule is domain_rule = [ rule_label_id ’:’ ] expression ., meaning that rule_label_id ’:’ is optional, but must be present together. : alone without rule_label_id is not allowed.

How should I handle this? Fail, or be tolerant and pass?

[ronaldtse]

@trthurman , are these exp files considered invalid?

WHERE
: SIZEOF (USEDIN (SELF, 'BASIC_ATTRIBUTE_SCHEMA.' + 'ID_ATTRIBUTE.IDENTIFIED_ITEM')) <= 1;
: SIZEOF (USEDIN (SELF, 'BASIC_ATTRIBUTE_SCHEMA.' + 'DESCRIPTION_ATTRIBUTE.DESCRIBED_ITEM')) <= 1;
END_ENTITY;

Are these entries in STEPmod to be fixed?

[zakjan]

As I go along, I'm finding a few more invalid files. I'll be posting them here for reference.

[zakjan]

empty select_list is not allowed

301 select_list = ’(’ named_types { ’,’ named_types } ’)’ .
302 select_type = [ EXTENSIBLE [ GENERIC_ENTITY ] ] SELECT [ select_list | select_extension ] .

ballots/ballots/ap233wd2/express/ecco_decision_support_arm_lf.exp

TYPE verification_evidence_item = SELECT 
  ();
END_TYPE;

ballots/ballots/annotated_text/associative_text_arm.exp

TYPE appearance_context = SELECT ();
END_TYPE;

TYPE appearance_select = SELECT ();
END_TYPE;

etc/ap210/tools/lksoft/longform_gen/mim_20050410v1_with_resources_longform.exp

TYPE requirement_source_item = SELECT ();
END_TYPE;

[zakjan]

git conflict marks

data/modules/ap239_ap242_combined_for_Domain_to_ARM_mapping_reference/ap242/concatenated_mim.exp data/modules/ap239_ap242_combined_for_Domain_to_ARM_mapping_reference/ap242/concatenated_arm.exp

[zakjan]

invalid order of SELECT EXTENSIBLE GENERIC_ENTITY

302 select_type = [ EXTENSIBLE [ GENERIC_ENTITY ] ] SELECT [ select_list | select_extension ] .

ballots/ballots/ap233_SC4-CD_build/express/mim_resources_20070821v1.exp ballots/ballots/ap233_SC4-CD_build/express/mim_20070821v1.exp

TYPE description_item = SELECT EXTENSIBLE GENERIC_ENTITY;
END_TYPE;   

[ronaldtse]

Also pinging @brandonsapp for this.

[zakjan]

extraneous ; between entity_id and subsuper

207 entity_head = ENTITY entity_id subsuper ’;’ .

ballots/ballots/ap233_SC4-CD_build/express/mim_resources_20070821v1.exp ballots/ballots/ap233_SC4-CD_build/express/mim_20070821v1.exp

ENTITY applied_description_text_assignment;
    SUBTYPE OF ( description_text_assignment );
    items : SET[1:?] OF description_item;
END_ENTITY;

[zakjan]

invalid usage of BY

The only reference of BY is in increment_control.

235 increment_control = variable_id ’:=’ bound_1 TO bound_2 [ BY increment ] .

data/modules/ap210_electronic_assembly_interconnect_and_packaging_design/dvlp/ap210_electronic_assembly_interconnect_and_packaging_design_arm_lf-patched-5589.exp data/modules/ap210_electronic_assembly_interconnect_and_packaging_design/dvlp/ap210_electronic_assembly_interconnect_and_packaging_design_arm_lf-patchedscript.exp

(p\Product_view_definition.shape_type = product_shape_type_enumeration by product_shape_type_enumeration.nominal_shape)

WR2: SELF\Shape_element.associated_definition.shape_type = product_shape_type_enumeration by product_shape_type_enumeration.nominal_shape;

[zakjan]

INTYPEOF is not a recognized keyword

missing space between IN and TYPEOF

data/modules/ap210_electronic_assembly_interconnect_and_packaging_design/dvlp/pmi/electronic_assembly_interconnect_and_packaging_design_mim.lfepm_500010000.exp

('AP210_ELECTRONIC_ASSEMBLY_INTERCONNECT_AND_PACKAGING_DESIGN_MIM_LF.EXTERNALLY_DEFINED_CLASS' INTYPEOF ( edir.related_item ) )

[zakjan]

missing schema_id

296 schema_decl = SCHEMA schema_id [ schema_version_id ] ’;’ schema_body END_SCHEMA ’;’ .

data/modules/ap210_electronic_assembly_interconnect_and_packaging_design/dvlp/mim_lf-pretty.exp data/modules/ap210_electronic_assembly_interconnect_and_packaging_design/dvlp/arm_lf_seded_7255-pretty.exp data/modules/ap242_managed_model_based_3d_engineering/mim_lf-pretty.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/mim_lf-pretty.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/arm_lf_seded_7255-pretty.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/arm_lf_4662.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/mim_lf-pretty-5056.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/mim_lf-pretty-5690.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/mim_lf-pretty-5004.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/arm_lf_seded_7255_pretty_grepped.exp data/modules/ap242_managed_model_based_3d_engineering/dvlp/mim_lf_greped.exp

SCHEMA ;

[zakjan]

extraneous , in the end of the list

336 use_clause = USE FROM schema_ref [ ’(’ named_type_or_rename { ’,’ named_type_or_rename } ’)’ ] ’;’ .

ballots/ballots/ap233wd2/express/mim_20060418v1.exp ballots/ballots/ap233wd2/express/mim_resources_20060418v1.exp

USE FROM ISO13584_expressions_schema(
    abs_function,
    acos_function,  
    asin_function,
    atan_function,  
    cos_function,
    exp_function,
    int_value_function,
    integer_defined_function,   
    length_function,    
    log_function,
    log10_function,
    log2_function,
    maximum_function,
    minimum_function,   
    minus_function,     
    numeric_defined_function,   
    odd_function,   
    real_defined_function,  
    sin_function,   
    square_root_function,   
    tan_function,   
    unary_function_call,    
    value_function, 
    );

[zakjan]

free-form text outside of remark

ballots/ballots/ap233wd2/express/mim_20060418v1.exp ballots/ballots/ap233wd2/express/mim_resources_20060418v1.exp

Error IF-3: The reference parameter for risk_schema has not been specified. Add @reference (e.g. ISO 10303-***) to data/resources/risk_schema/risk_schema.xml.

data/modules/ap210_electronic_assembly_interconnect_and_packaging_design/concatenated_mim.exp

This file is obsolete. see file mim_concatenated.exp

data/modules/ap210_electronic_assembly_interconnect_and_packaging_design/concatenated_arm.exp

This file is obsolete. see file arm_concatenated.exp

data/modules/solid_with_local_modification/Revisedent.exp

Alternative proposal for variable radius entity: 

[zakjan]

missing OF between ENUMERATION and enumeration_items

213 enumeration_type = [ EXTENSIBLE ] ENUMERATION [ ( OF enumeration_items ) | enumeration_extension ] .

data/resources/quantities_and_units_schema/quantities_and_units_schema.exp

TYPE unit_multiplier = ENUMERATION (
  ... 
  );
END_TYPE;

TYPE iso80000_quantity = ENUMERATION (
...
);
END_TYPE;

TYPE iso80000_unit = ENUMERATION (
...
);
END_TYPE;

[zakjan]

extraneous ; between WHERE and domain_rule

338 where_clause = WHERE domain_rule ’;’ { domain_rule ’;’ } .

data/resources/quantities_and_units_schema/quantities_and_units_schema.exp

WHERE;
  wr1: TRUE; -- nu

[zakjan]

space instead of '_' in enumeration_id

210 enumeration_id = simple_id .
211 enumeration_items = ’(’ enumeration_id { ’,’ enumeration_id } ’)’ .

data/resources/quantities_and_units_schema/quantities_and_units_schema.exp

kilogram metre_squared_per_second, -- 4-12.a

[TRThurman]

space instead of '_' in enumeration_id

210 enumeration_id = simple_id . 211 enumeration_items = ’(’ enumeration_id { ’,’ enumeration_id } ’)’ . data/resources/quantities_and_units_schema/quantities_and_units_schema.exp

kilogram metre_squared_per_second, -- 4-12.a

Generally speaking, Unless an EXPRESS file is published, there may be errors that will not be corrected, because the files are dormant. In some cases, there are errors in the published EXPRESS, but that is much rarer.

[zakjan]

Thanks @TRThurman for the explanation. I'll continue listing the issues here for reference only.

[zakjan]

} instead of ] in bound_spec

185 bound_spec = ’[’ bound_1 ’:’ bound_2 ’]’ .

data/resources/quantities_and_units_schema/quantities_and_units_schema.exp

us : SET[1:?} OF physical_unit;

[zakjan]

missing ; at the end of entity_head

207 entity_head = ENTITY entity_id subsuper ’;’ .

data/resources/quantities_and_units_schema/quantities_and_units_schema.exp

ENTITY qus

[zakjan]

( ) instead of [ ] in aggregate_initializer

169 aggregate_initializer = ’[’ [ element { ’,’ element } ] ’]’ .

data/resources/quantities_and_units_schema/quantities_and_units_schema.exp

qus(mass,(kilogram,tonne))

[zakjan]

missing EXTENSIBLE with GENERIC_ENTITY select_type

302 select_type = [ EXTENSIBLE [ GENERIC_ENTITY ] ] SELECT [ select_list | select_extension ] .

data/modules/ap239_ap242_combined_for_Domain_to_ARM_mapping_reference/ap210/concatenated_arm.exp

TYPE pu2ds_part_template_or_physical_unit_2d_shape_select =  GENERIC_ENTITY SELECT BASED_ON part_template_or_physical_unit_2d_shape_select WITH 
    (Physical_unit_planar_shape_model);
  END_TYPE;

[zakjan]

empty file, missing at least 1 schema

The primary syntax rule for EXPRESS is syntax.

324 syntax = schema_decl { schema_decl } .

ballots/ballots/ap210_wg12/express/mim/mim_resources_20050410v1.exp

[zakjan]

schema version as free-form text, should be in string literal in schema_head

296 schema_decl = SCHEMA schema_id [ schema_version_id ] ’;’ schema_body END_SCHEMA ’;’ .
298 schema_version_id = string_literal .

doc/express-check/express_e2_shtolo_sf.exp doc/express-check/ir53_from_PW.exp

{ iso standard       10303  part (11) version (4) }

[zakjan]

invalid remark tags

145 embedded_remark = ’(*’ ... ’*)’ .

data/modules/simplified_cataloguing/dvlp/catalogue_planning.exp

/*
   Catalogue planning model
   $Id: catalogue_planning.exp,v 1.3 2007/02/16 21:03:52 radack Exp $
*/

[zakjan]

files marked as invalid

data/modules/ap242_managed_model_based_3d_engineering/dvlp/pmi/concatenated_mim.exp

(* inject EXPRESS error to trigger awareness that
this is an obsolete file.
correct file is 
/stepmod/data/modules/ap242_managed_model_based_3d_engineering/dvlp/express/pmi/concatenated_arm.exp
*)
/////SCHEMA Activity_mim;////

data/modules/ap242_managed_model_based_3d_engineering/dvlp/pmi/concatenated_arm.exp

(* inject EXPRESS error to trigger awareness that
this is an obsolete file.
correct file is 
/stepmod/data/modules/ap242_managed_model_based_3d_engineering/dvlp/express/pmi/concatenated_arm.exp
*)
//////SCHEMA Activity_arm;////

[zakjan]

AND instead of | in query_expression

277 query_expression = QUERY ’(’ variable_id ’<*’ aggregate_source ’|’ logical_expression ’)’ .

data/modules/catalog_data_information/dvlp/express/mim_20060714v1.exp

    WR1 : SIZEOF(QUERY
                                    ( 
                                        aga <* USEDIN( SELF, 'MANAGEMENT_RESOURCES_SCHEMA.' + 'GROUP_ASSIGNMENT.ASSIGNED_GROUP' ) 
                                        AND
                                    (
                                        ( aga.role.name <> 'specification category member' ) 
                                        OR 
                                        ( SIZEOF(QUERY 
                                                                ( 
                                                                    i <* aga.items 
                                                                    | 
                                                                    ('PRODUCT_CONCEPT_SCHEMA.'+ 'PRODUCT_CONCEPT_FEATURE' IN TYPEOF ( i ) ) 
                                                                    AND 
                                                                    NOT ('PRODUCT_CONCEPT_SCHEMA.'+ 'CONDITIONAL_CONCEPT_FEATURE' IN TYPEOF (i))
                                                                )
                                                        ) <> SIZEOF (aga.items)
                                        )
                                    ) 
                            ) 
                    ) =0;

[zakjan]

missing , in the list

301 select_list = ’(’ named_types { ’,’ named_types } ’)’ .

data/modules/ap239_ap242_combined_for_Domain_to_ARM_mapping_reference/dvlp/arm_concatenated.exp

Uncertainty_qualifier,
Unit
Validation,

Validation,
Verification_relationship
Verification,

[zakjan]

extraneous ) in nested expression

ballots/ballots/annotated_text/mim_resources_20050407v1.exp

WR3: SIZEOF(QUERY(aeia <* applied_group_assignment |
(SELF.role.name = 'group membership')) AND
 (NOT  ((SIZEOF(SELF.items) = SIZEOF( QUERY( i <* SELF.items | 'CATALOG_DATA_INFORMATION_MIM' + '.GEOMETRIC_REPRESENTATION_ITEM' IN TYPEOF(i)))) OR 
  (SIZEOF(SELF.items) = SIZEOF( QUERY( i <* SELF.items | 'CATALOG_DATA_INFORMATION_MIM' + '.SHAPE_ASPECT' IN TYPEOF(i)))))) 
    ))=0; 

[zakjan]

embedded remark starting on the same line as tail remark

The tail remark is written at the end of a physical line. Two consecutive hyphens (--) start the tail remark and the following newline terminates it.

149 tail_remark = ’--’ [ remark_tag ] { \a | \s | \x9 | \xA | \xD } \n .

ballots/ballots/ap210_wg12/express/resources/resources.exp ballots/ballots/ap210_wg12/express/mim/mim_20050410v1_with_resources.exp ballots/ballots/ap210_wg12/express/mim/mim_20050410v1_with_resources2.exp ballots/ballots/annotated_text/mim_resources_20050407v1.exp

-- tail remark (*
embedded remark
*)

[ronaldtse]

Question raised at ISO/TC 184/SC 4/WG 12 meeting on 2020-09-03:

• .exp files in */dvlp/* are not final files and should be ignored.
• .exp files in ballots/* are historic files and should be ignored.
• the data/resources/quantities_and_units_schema/quantities_and_units_schema.exp schema is not often used. Not sure what we should do about it.
• @TRThurman says there is an index.xml that contains current schemas in use and only those need to be valid .exp files.

[TRThurman]

@TRThurman https://github.com/TRThurman says there is an index.xml that contains current schemas in use and only those need to be valid .exp files.

stepmod/repository_index.xml is the actual file referenced.

[zakjan]

Thanks! I'll use only files referenced in repository_index.xml for further testing.

[ronaldtse]

Thanks @TRThurman @zakjan !

[zakjan]

After running the grammar against files referenced in repository_index.xml, there are these errors left:

• missing schema_id https://github.com/lutaml/express-grammar/issues/3#issuecomment-678219643
• invalid usage of BY https://github.com/lutaml/express-grammar/issues/3#issuecomment-678175676
• AND instead of | in query_expression https://github.com/lutaml/express-grammar/issues/3#issuecomment-678984468
• INTYPEOF is not a recognized keyword https://github.com/lutaml/express-grammar/issues/3#issuecomment-678209755
• free-form text outside of remark https://github.com/lutaml/express-grammar/issues/3#issuecomment-678261181
• files marked as invalid https://github.com/lutaml/express-grammar/issues/3#issuecomment-678977105

[TRThurman]

what files?

[zakjan]

Specific files are referenced in the linked comments above

[ronaldtse]

The files listed as invalid in https://github.com/lutaml/express-grammar/issues/3#issuecomment-691432158 are all "inactive" files only kept for reference, such as prettified, modified, historic or concatenated_arm/mim files. This means that all of these files should not be included in the SMRL.

@TRThurman can we only keep the necessary files (i.e. remove all of these) in the new repo?

[TRThurman]

don't send me to a link. just list the explicit file in a comment that have syntax errors.

[TRThurman]

The SRL repo will need to support developer uploaded files that are not in the build process. Recall the build process only cares about the files included in build.xml so any repo replacement should be architected the same way. We can't prevent developers, stakeholders, and progrm manglers from changing their minds after something is uploaded but not published. Actually we can prevent that even after something is published. So trying to scrub STEPmod is a fools errand, IMHO.

We can and should be focused on repeatble process for building deliverables.

On Sep 13, 2020, at 9:27 AM, Ronald Tse notifications@github.com wrote:

The files listed as invalid in #3 (comment) https://github.com/lutaml/express-grammar/issues/3#issuecomment-691432158 are all "inactive" files only kept for reference, such as prettified, modified, historic or concatenated_arm/mim files. This means that all of these files should not be included in the SMRL.

@TRThurman https://github.com/TRThurman can we only keep the necessary files (i.e. remove all of these) in the new repo?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/lutaml/express-grammar/issues/3#issuecomment-691678372, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAMMKVEEL4HLM3CFOD4RNDTSFTJFDANCNFSM4QKESJ2A.

Thomas Thurman Principal Electrical Engineer, Rockwell Collins (Retired)

Confidentiality Notice: This e-mail (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is confidential and may be legally privileged. If you are not the intended recipient, you are hereby notified that any retention, dissemination, distribution, or copying of this communication is strictly prohibited. Please reply to the sender that you have received the message in error, then delete/destroy any and all copies of the original message. E-mail transmissions cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late, incomplete, or contain viruses. Therefore, the recipient should check this email and any attachments for the presence of viruses. The sender accepts no liability for any damage caused by any virus transmitted by this email.

[zakjan]

@TRThurman Please accept my apology, I wasn't familiar with the repo structure before. I have pushed a script which finds all valid Express files according to notes from @ronaldtse.

https://github.com/metanorma/stepmod-utils/blob/master/exe/stepmod-find-express-files

There are 1246 files total. All files are passing the parser. This issue can be closed.