elia-k
commented
7 years ago After a first review of the examples mentioned in the issue description I provide a sub-list of examples that in my opinion are not supporting due to inconsistencies of Reference and User Manual and VTL version 1.0 AND 1.1 . In our approach, the tokens and rules created/used were based on the reference manual for VTL 1.1.
More specific,
[1]
line 1737-1740*/define function compare_integer_descending(x as integer, y as integer) returns boolean as x > y
line 1784-1784/define function has_solution(a, b, c) as bb-4ac>0
line 1790-1792/define function has_solution(a as number, b as number, c as number) returns boolean > as bb-4ac>0
line 1802-1806: define function distance(d1 as dataset, d2 as dataset, n as integer := 2) as (d1^n - d2^n)^(1/n)
line 1813-1816: define function z_transform(x as number, mu as number := 0, sigma as number := 1) as (x-mu)/sigma
The examples above are not supported because in the reference manual of VTL 1.1 there is no definition for "define function". Instead, "create function is used". However, "define function" appears in the user manual.
[2]
line 1840-1845/define procedure quot_rem(in ds as dataset, in divisor as number, out quot as dataset, > out rem as dataset) as { quot := floor(ds / divisor) rem := ds - quotdivisor }
The example above is not supported because there is no definition for "define procedure" in the reference manual of VTL 1.1 . However, "define procedure" is described in the User manual
[3]
line 1852*/call quot_rem(PopPerCountry, AvgPop, Multiple, Remainder)
The example above is not supported because there is no "call" nor "quot_rem" defined in the reference manual of VTL 1.1 However, they both appear in the User manual.
[4]
line 2085-2086*/type population = dataset { identifier geo as string identifier year as integer measure population as float attribute status as string }
The example above is not supported because there is no "type" defined in the reference manual of VTL 1.1 However, it appears in the User manual.
[5]
line 951*/ds := check(ds_bop, ruleset_1, with measures , only failures)
The example above is not supported because there is no "with measures" nor "only failures" used in the reference manual of VTL 1.1. "Measures" is used instead of "with measures".
[6]
line 1130-1135*/define hierarchical ruleset AddBenelux (valuedomain=GeoArea) is Belgium = Belgium > Luxembourg = Luxembourg Netherlands = Netherlands Benelux = Belgium + Luxembourg + Netherlands end hierarchical ruleset
The example above is not supported because in "define hierarchical ruleset" the rules need to be separated by ';'as defined in the reference manual of VTL 1.1
[7]
line 1492*/alterDataset(ds_1 all)
The example above is not supported because in "alterDataset" a semicolon ';' is required at the end od the expression as defined in the reference manual of VTL 1.1
[8]
line 3587*/ds_3:= func_dep ( ds_1, (FISCAL_CODE ), (NAME, DATE_OF_BIRTH,PLACE_OF_BIRTH))
line 3590*/ds_2 := func_dep ( NAME), (FISCAL_CODE) )
The examples above are not supported because there is no "func_dep" defined in the reference manual of VTL 1.1
[9]
line 3834*/B := string_from_date (A, YYYY-MM)
line 3837*/ds_2:= string_from_date (ds_1, \"YYYY-MM\")
The examples above are not supported because YYYY-MM and "YYYY-MM" are not declared in the format in the reference manual of VTL 1.1
[10]
line 4271*/ds_agg := avg ( ds_bop.obs_value ) group by time
line 4279*/ds_agg := avg ( ds_bop.obs_value ) group by time, ref_area
The examples above are not supported because in "group by" the components that follow need to be in parenthesis as defined in the reference manual of VTL 1.1
[11]
line 945-947*/define datapoint ruleset ruleset_1 (FLOW as x, OBS_STATUS as y) (flow_dr : when x = \"CREDIT\" or x = \"AVERAGE\" then y <> \"C\" errorcode ( -XXXXX ))
line 4835-4843*/define datapoint ruleset dpr_ labour ( AGE as a) (rule_1 when a between 14 and 17 then edu <> 5 ;rule_2 when a between 16 and 19 then edu <> 6 ;rule_3 when a between 17 and 20 then edu <> 7 and edu <> 8 ;rule_4 when a between 18 and 21 then edu <> 10 ; rule_5 when a between 14 and 16 then edu <> 13 ;rule_6 when a between 16 and 18 then edu <> 14 ; rule_7 when a > between 17 and 20 then edu <> 15 ;) ;
The examples above are not supported because in "define datapoint ruleset" 'is' is required after the ruleset signature as defined in the reference manual of VTL 1.1
[12]
line 4978*/ds_outliers := check ( abs ( ds_bop.obs_value – ds_moving_average.obs_value ) <= 10 ) ;
The example above is not supported because in "check" a semicolon ';' should not be used at the end as defined in the reference manual of VTL 1.1
I will continue reviewing the rest of the examples and comment on them soon.
linardian
Issue Description
Examples from the user and the reference manual should be tested against the grammar. A list of examples of the reference manual that are not covered by the grammar can be found in this project: https://bitbucket.org/DominikLin/vtlparserusingofficialgrammar, class OfficialVtlParser (in VtlParserUsingOfficialGrammar/src/main/java/ecb/officialVtlParser/). Please note that this is not a complete list.
Proposed Solution