search

antlr / grammars-v4

Grammars written for ANTLR v4; expectation that the grammars are free of actions.
MIT License
10.24k stars 3.72k forks source link

[all] Unreferenced parser rule symbols #3358

Open kaby76 opened 1 year ago

kaby76 commented 1 year ago

I wrote a little Trash script to see what I would find in grammars-v4 for unreferenced parser rule symbols, where there is a parser rule defined, but the LHS symbol of the rule is never reference anywhere in the grammar files. I was surprised to find a lot of parser rules that aren't used.

Perhaps the references to these rules were refactored out of the grammar purposefully. For example, to eliminate indirectly left recursion, unfolding is usually performed. But, afterward, unused rules should have been then been deleted.

Unused rules likely indicate an error in the grammar.

Here is the script and output:

Find unused parser symbols

#
for i in `find . -name desc.xml | grep -v Generated\*`
do
  echo $i
  d=`dirname $i`
  pushd $d > /dev/null 2>&1
  # Parse all grammar files so that any imports can also be checked.
  trparse *.g4 2> /dev/null | trxgrep ' //parserRuleSpec[not(doc("*")//ruleBlock//RULE_REF/text() = ./RULE_REF/text()) and not(./ruleBlock//TOKEN_REF/text() = "EOF")]/RULE_REF' | trtext
  popd > /dev/null 2>&1
done

Unused symbols in grammars-v4

$ bash ~/unused.sh
./abb/desc.xml
./abnf/desc.xml
./acme/desc.xml
acmePropertyElement
acmeViewDeclaration
acmeViewTypeDeclaration
terminatedDesignRuleExpression
./ada/ada2005/desc.xml
Ada2005Parser.g4:logical_operator
Ada2005Parser.g4:highest_precedence_operator
./ada/ada2012/desc.xml
AdaParser.g4:logical_operator
AdaParser.g4:highest_precedence_operator
AdaParser.g4:quantified_expression
./ada/ada83/desc.xml
Ada83Parser.g4:logical_operator
Ada83Parser.g4:highest_precedence_operator
./ada/ada95/desc.xml
Ada95Parser.g4:logical_operator
Ada95Parser.g4:highest_precedence_operator
Ada95Parser.g4:restriction
./agc/desc.xml
./alef/desc.xml
./algol60/desc.xml
integer
number
destination
./alloy/desc.xml
./alpaca/desc.xml
./angelscript/desc.xml
./antlr/antlr2/desc.xml
ANTLRv2Parser.g4:exceptionSpecNoLabel
ANTLRv2Parser.g4:action
./antlr/antlr3/desc.xml
./antlr/antlr4/desc.xml
./apex/desc.xml
enumConstantName
constantExpression
innerCreator
./apt/desc.xml
./aql/desc.xml
./argus/desc.xml
./arithmetic/desc.xml
./asl/desc.xml
./asm/asm6502/desc.xml
./asm/asm8080/desc.xml
./asm/asm8086/desc.xml
./asm/asmMASM/desc.xml
./asm/asmZ80/desc.xml
./asm/masm/desc.xml
segmentos
time
./asm/pdp7/desc.xml
./asm/ptx/ptx-isa-1.0/desc.xml
./asm/ptx/ptx-isa-2.1/desc.xml
id
opr_register_or_constant5
opr5
./asn/asn/desc.xml
usefulObjectClassReference
externalObjectClassReference
typeFieldSpec
fixedTypeValueFieldSpec
variableTypeValueFieldSpec
fixedTypeValueSetFieldSpec
objectSetElements
variableTypeValueSetFieldSpec
objectFieldSpec
objectSetFieldSpec
simpleTableConstraint
./asn/asn_3gpp/desc.xml
usefulObjectClassReference
externalObjectClassReference
typeFieldSpec
fixedTypeValueFieldSpec
variableTypeValueFieldSpec
fixedTypeValueSetFieldSpec
objectSetElements
variableTypeValueSetFieldSpec
objectFieldSpec
objectSetFieldSpec
simpleTableConstraint
./atl/desc.xml
./awk/desc.xml
./b/desc.xml
lvalue
./basic/desc.xml
./bcl/desc.xml
./bcpl/desc.xml
bcpl.g4:unless_command
bcpl.g4:program
origbcpl.g4:unless_command
origbcpl.g4:program
./bdf/desc.xml
./bencoding/desc.xml
./bibcode/desc.xml
./bibtex/desc.xml
./bicep/desc.xml
./bnf/desc.xml
./c/desc.xml
./calculator/desc.xml
./callable/desc.xml
./capnproto/desc.xml
./caql/desc.xml
CaQLParser.g4:expression
CaQLParser.g4:by
CaQLParser.g4:without
Metrics.g4:expr
./cayenne/desc.xml
./chip8/desc.xml
./clf/desc.xml
./clif/desc.xml
./clojure/desc.xml
./clu/desc.xml
./cmake/desc.xml
./cobol85/desc.xml
Cobol85.g4:basisName
Cobol85.g4:languageName
Cobol85.g4:routineName
./cookie/desc.xml
name
./cpp/desc.xml
./cql/desc.xml
./cql3/desc.xml
CqlParser.g4:password
./creole/desc.xml
./csharp/desc.xml
CSharpParser.g4:interface_body
CSharpPreprocessorParser.g4:preprocessor_directive
./css3/desc.xml
./csv/desc.xml
./ctl/desc.xml
./cto/desc.xml
./cypher/desc.xml
./dart2/desc.xml
Dart2Parser.g4:letExpression
Dart2Parser.g4:reserved_word
Dart2Parser.g4:typeNotVoidNotFunctionList
./databank/desc.xml
./datalog/desc.xml
./dcm/desc.xml
./dice/desc.xml
./dif/desc.xml
./doiurl/desc.xml
./dot/desc.xml
./edif300/desc.xml
./edn/desc.xml
./erlang/desc.xml
./esolang/brainflak/desc.xml
./esolang/brainfuck/desc.xml
./esolang/cool/desc.xml
./esolang/dgol/desc.xml
./esolang/lolcode/desc.xml
./esolang/loop/desc.xml
./esolang/nanofuck/desc.xml
./esolang/sickbay/desc.xml
./esolang/snowball/desc.xml
./esolang/wheel/desc.xml
./evm-bytecode/desc.xml
./fasta/desc.xml
./fdo91/desc.xml
./fen/desc.xml
./flatbuffers/desc.xml
./flowmatic/desc.xml
./focal/desc.xml
./fol/desc.xml
./fortran77/desc.xml
Fortran77Parser.g4:typenameLen
Fortran77Parser.g4:formatStatement
Fortran77Parser.g4:arithmeticExpression
Fortran77Parser.g4:arithmeticConstExpr
Fortran77Parser.g4:logicalConstExpr
Fortran77Parser.g4:arrayElementName
Fortran77Parser.g4:arrayName
Fortran77Parser.g4:subroutineName
Fortran77Parser.g4:functionName
./freedesktop/desktop-entry/desc.xml
./fusion-tables/desc.xml
keyword
error_message
column_alias
new_table_name
./gdscript/desc.xml
./gedcom/desc.xml
./gff3/desc.xml
./glsl/desc.xml
GLSLPreParser.g4:translation_unit
./gml/desc.xml
./golang/desc.xml
GoParser.g4:emptyStmt
GoParser.g4:receiverType
./graphql/desc.xml
./graphstream-dgs/desc.xml
./gtin/desc.xml
ean14_appid
./guido/desc.xml
./guitartab/desc.xml
./haskell/desc.xml
HaskellParser.g4:fpat
HaskellParser.g4:special
HaskellParser.g4:symbol
./html/desc.xml
HTMLParser.g4:htmlDocument
./http/desc.xml
./hypertalk/desc.xml
./icalendar/desc.xml
param_name
bool_
k_address
k_fr
k_mo
k_participant
k_process
k_relat
k_sa
k_su
k_th
k_tu
k_we
./icon/desc.xml
./idl/desc.xml
./inf/desc.xml
./informix/desc.xml
expressionSet
sqlPseudoColumn
fieldIdentifier
actualParameterList
./infosapient/desc.xml
./iri/desc.xml
iri_reference
absolute_iri
ipath
reserved
./iso8601/desc.xml
second
calendarDate
ordinalDate
weekDate
specificWeekWeekDate
datePrecise
localTimePrecise
specificMinuteLocalTime
localTimeAndTimeZone
datetimePrecise
interval
timeBeginEnd
timeBeginInterval
timeIntervalEnd
./istc/desc.xml
./itn/desc.xml
./jam/desc.xml
./janus/desc.xml
./java/java/desc.xml
./java/java8/desc.xml
Java8Parser.g4:unannInterfaceType
Java8Parser.g4:primaryNoNewArray
./java/java9/desc.xml
Java9Parser.g4:unannInterfaceType
Java9Parser.g4:primaryNoNewArray
./javadoc/desc.xml
./javascript/ecmascript/desc.xml
./javascript/javascript/desc.xml
./javascript/jsx/desc.xml
./javascript/typescript/desc.xml
TypeScriptParser.g4:arrayType
TypeScriptParser.g4:tupleType
TypeScriptParser.g4:requiredParameterList
./joss/desc.xml
./jpa/desc.xml
./json/desc.xml
./json5/desc.xml
./karel/desc.xml
./kirikiri-tjs/desc.xml
./kotlin/kotlin/desc.xml
./kotlin/kotlin-formal/desc.xml
./kquery/desc.xml
./kuka/desc.xml
./lambda/desc.xml
./lark/desc.xml
./lcc/desc.xml
./less/desc.xml
LessParser.g4:function_
./limbo/desc.xml
./lisa/desc.xml
./lisp/desc.xml
./llvm-ir/desc.xml
./logo/logo/desc.xml
./logo/ucb-logo/desc.xml
./lpc/desc.xml
./lrc/desc.xml
./ltl/desc.xml
./lua/desc.xml
./lucene/desc.xml
./matlab/desc.xml
translation_unit
./mckeeman-form/desc.xml
./mdx/desc.xml
./memcached_protocol/desc.xml
storage_response
retrieval_response
deletion_response
incr_or_decr_response
statistics_response
./metamath/desc.xml
./metric/desc.xml
./microc/desc.xml
./modelica/desc.xml
./modula2pim4/desc.xml
integer
real
scaleFactor
hexDigit
digit
octalDigit
./molecule/desc.xml
./moo/desc.xml
./morsecode/desc.xml
./mps/desc.xml
./muddb/desc.xml
./mumath/desc.xml
./mumps/desc.xml
program
./muparser/desc.xml
./newick/desc.xml
./oberon/desc.xml
./objc/desc.xml
./ocl/desc.xml
./oncrpc/desc.xml
./orwell/desc.xml
./p/desc.xml
./parkingsign/desc.xml
dayRangePlus
./pascal/desc.xml
empty_
./pbm/desc.xml
./pcre/desc.xml
./pddl/desc.xml
./pdn/desc.xml
./peoplecode/desc.xml
./pgn/desc.xml
./php/desc.xml
PhpParser.g4:magicMethod
./pii/desc.xml
./pike/desc.xml
./pl0/desc.xml
./plucid/desc.xml
where_clause
./ply/desc.xml
./pmmn/desc.xml
./postalcode/desc.xml
./powerbuilder/desc.xml
PowerBuilderParser.g4:atom_sub
./powerbuilderdw/desc.xml
./powerquery/desc.xml
PowerQueryParser.g4:document
./prolog/desc.xml
./promql/desc.xml
./propcalc/desc.xml
atoms
./properties/desc.xml
PropertiesParser.g4:propertiesFile
./protobuf2/desc.xml
./protobuf3/desc.xml
./prov-n/desc.xml
./python/python/desc.xml
./python/python2/desc.xml
single_input
file_input
eval_input
encoding_decl
./python/python2-js/desc.xml
single_input
file_input
eval_input
encoding_decl
./python/python3/desc.xml
Python3Parser.g4:single_input
Python3Parser.g4:encoding_decl
./python/python3-py/desc.xml
single_input
encoding_decl
./python/python3-ts/desc.xml
single_input
encoding_decl
./python/python3-without-actions/desc.xml
./python/python3alt/desc.xml
single_input
./python/tiny-python/desc.xml
./qif/desc.xml
./quakemap/desc.xml
./r/desc.xml
./racket-bsl/desc.xml
./racket-isl/desc.xml
./rcs/desc.xml
rcsheader
rcsrevisions
./recfile/desc.xml
./redcode/desc.xml
./refal/desc.xml
./rego/desc.xml
RegoParser.g4:ruleExt
RegoParser.g4:exprTermPairList
./restructuredtext/desc.xml
./rexx/desc.xml
./rfc1035/desc.xml
./rfc1960/desc.xml
./rfc3080/desc.xml
./rfc822/rfc822-datetime/desc.xml
./rfc822/rfc822-emailaddress/desc.xml
delimeters
./robotwars/desc.xml
./romannumerals/desc.xml
./ron/desc.xml
./rpn/desc.xml
./ruby/desc.xml
array_definition
./rust/desc.xml
./scala/desc.xml
compilationUnit
./scotty/desc.xml
./scss/desc.xml
./semver/desc.xml
./sexpression/desc.xml
./sgf/desc.xml
./sharc/desc.xml
SHARCParser.g4:mov_exp_6a
SHARCParser.g4:mov_exp_6b
SHARCParser.g4:mem_addr_int_
SHARCParser.g4:jump_addr_md_or_pc
SHARCParser.g4:r3_0
SHARCParser.g4:r7_4
SHARCParser.g4:r11_8
SHARCParser.g4:r15_12
SHARCParser.g4:f3_0
SHARCParser.g4:f7_4
SHARCParser.g4:f11_8
SHARCParser.g4:f15_12
SHARCParser.g4:addr
./sici/desc.xml
./sieve/desc.xml
comparator
./smalltalk/desc.xml
keywords
./smiles/desc.xml
./smtlibv2/desc.xml
generalReservedWord
./snobol/desc.xml
gt
convert
./solidity/desc.xml
./sparql/desc.xml
./spass/desc.xml
./sql/athena/desc.xml
AthenaParser.g4:field_name
./sql/derby/desc.xml
DerbyParser.g4:keyword
./sql/drill/desc.xml
DrillParser.g4:select_expression
./sql/hive/v2/desc.xml
HiveParser.g4:empty
HiveParser.g4:alterStatementSuffixStatsPart
./sql/hive/v3/desc.xml
HiveParser.g4:alterStatementSuffixStatsPart
ResourcePlanParser.g4:resourcePlanDdlStatements
ResourcePlanParser.g4:triggerOrExpression
./sql/hive/v4/desc.xml
HiveParser.g4:alterDatabaseSuffixSetManagedLocation
HiveParser.g4:alterStatementSuffixStatsPart
HiveParser.g4:disable
HiveParser.g4:triggerOrExpression
./sql/informix-sql/desc.xml
./sql/mariadb/desc.xml
./sql/mysql/Positive-Technologies/desc.xml
./sql/phoenix/desc.xml
./sql/plsql/desc.xml
PlSqlParser.g4:modify_diskgroup_file
PlSqlParser.g4:routine_clause
PlSqlParser.g4:compound_trigger_block
PlSqlParser.g4:alter_attribute_definition
PlSqlParser.g4:datetime_expr
PlSqlParser.g4:interval_expr
PlSqlParser.g4:synchronous_or_asynchronous
PlSqlParser.g4:including_or_excluding
PlSqlParser.g4:datatype_null_enable
PlSqlParser.g4:alter_varray_col_properties
PlSqlParser.g4:out_of_line_part_storage
PlSqlParser.g4:column_or_attribute
PlSqlParser.g4:add_constraint
PlSqlParser.g4:drop_constraint
PlSqlParser.g4:enable_constraint
PlSqlParser.g4:disable_constraint
PlSqlParser.g4:swallow_to_semi
PlSqlParser.g4:table_var_name
PlSqlParser.g4:string_function_name
PlSqlParser.g4:numeric_function_name
./sql/postgresql/desc.xml
PostgreSQLParser.g4:plsqlroot
./sql/snowflake/desc.xml
SnowflakeParser.g4:not_null
./sql/sqlite/desc.xml
SQLiteParser.g4:recursive_cte
SQLiteParser.g4:factored_select_stmt
SQLiteParser.g4:simple_select_stmt
SQLiteParser.g4:compound_select_stmt
SQLiteParser.g4:simple_function_invocation
SQLiteParser.g4:aggregate_function_invocation
SQLiteParser.g4:window_function_invocation
./sql/trino/desc.xml
./sql/tsql/desc.xml
TSqlParser.g4:empty_statement
./stacktrace/desc.xml
./star/desc.xml
./stellaris/desc.xml
./stl/desc.xml
./stringtemplate/desc.xml
STParser.g4:notConditionalExpr
./suokif/desc.xml
./swift/swift2/desc.xml
top_level_declaration
./swift/swift3/desc.xml
declarations
top_level_declaration
playground_literal
./swift/swift5/desc.xml
Swift5Parser.g4:switch_elseif_directive_clause
Swift5Parser.g4:declarations
Swift5Parser.g4:top_level_declaration
Swift5Parser.g4:expression_list
./swift-fin/desc.xml
./szf/desc.xml
./tcpheader/desc.xml
segmentheader
byte_
./teal/desc.xml
./telephone/desc.xml
./terraform/desc.xml
./thrift/desc.xml
./tiny/desc.xml
./tinybasic/desc.xml
./tinyc/desc.xml
./tinymud/desc.xml
./tinyos_nesc/desc.xml
TinyosParser.g4:chars_no_comma
./tl/desc.xml
./tnsnames/desc.xml
./tnt/desc.xml
./toml/desc.xml
./trac/desc.xml
./tsv/desc.xml
./ttm/desc.xml
./turing/desc.xml
./turtle/desc.xml
./turtle-doc/desc.xml
./unicode/graphemes/desc.xml
./unicode/unicode16/desc.xml
./unreal_angelscript/desc.xml
UnrealAngelscriptParser.g4:newPlacement
UnrealAngelscriptParser.g4:newInitializer_
UnrealAngelscriptParser.g4:trailingTypeSpecifierSeq
./upnp/desc.xml
./url/desc.xml
./useragent/desc.xml
./v/desc.xml
./vb6/desc.xml
VisualBasic6Parser.g4:cp_PropertyName
./vba/desc.xml
./velocity/desc.xml
./verilog/systemverilog/desc.xml
SystemVerilogPreParser.g4:source_text
./verilog/verilog/desc.xml
VerilogPreParser.g4:source_text
./vhdl/desc.xml
interface_signal_list
object_declaration
./vmf/desc.xml
./wat/desc.xml
WatParser.g4:value
./wavefront/desc.xml
./webidl/desc.xml
otherOrComma
./wkt/desc.xml
./wln/desc.xml
./wren/desc.xml
./xml/desc.xml
./xpath/xpath1/desc.xml
./xpath/xpath20/desc.xml
./xpath/xpath31/desc.xml
./xsd-regex/desc.xml
./xyz/desc.xml
./yara/desc.xml
./z/desc.xml
ZParser.g4:application
04/30-08:57:48 ~/l/grammars-v4
$
msagca commented 1 year ago

The following are the start symbols of the preprocessor grammars that I authored. Since the lexer processes both the source code and compiler directives at once, there is only one EOF token which appears in the start symbol(s) of the parser grammar and is omitted in the preprocessor parser grammar.

GLSLPreParser.g4:translation_unit
SystemVerilogPreParser.g4:source_text
VerilogPreParser.g4:source_text
kaby76 commented 1 year ago

As I thought I mentioned, the symbols in this list may be errors, or may not. The Verilog grammars are okay. We'll need to get trgen to be up to snuff for testing of it.

Each one of these symbols in this list is going to tell an interesting story. Let's look at a few.

kaby76 commented 1 year ago