search

rsyslog / liblognorm

a fast samples-based log normalization library
http://www.liblognorm.com
GNU Lesser General Public License v2.1
99 stars 64 forks source link

test-suite fails on arm*, ppc64el and s390x #217

Closed mbiebl closed 8 years ago

mbiebl commented 8 years ago

Hi,

as you can see at [1], the liblognorm test-suite get's stuck on arm and is killed after 150 min of inactivity. The failing test is ./usrdef_simple.sh. It seems "../src/ln_test -r tmp.rulebase -e json" get's stuck in a read loop. stracing that process shows

read(0, "", 4096)                       = 0
read(0, "", 4096)                       = 0
read(0, "", 4096)                       = 0
read(0, "", 4096)                       = 0
read(0, "", 4096)                       = 0
read(0, "", 4096)                       = 0
read(0, "", 4096)                       = 0
...

[1] https://buildd.debian.org/status/package.php?p=liblognorm

mbiebl commented 8 years ago

Version is 2.0.1. I can reproduce the hanging test on a arm64 porterbox. So if you want me to run further diagnostics, please let me know what.

mbiebl commented 8 years ago

When running the test on i386 I get

$  echo 'a word w1 a byte 0xff another word w2' | ../src/ln_test -r tmp.rulebase -e json -v
liblognorm: loading rulebase file 'tmp.rulebase'
liblognorm: rulebase version is 2

liblognorm: read rulebase line[~2]: 'type=@hex-byte:%f1:hexnumber{"maxval": "255"}%'
liblognorm: type line to add: '@hex-byte:%f1:hexnumber{"maxval": "255"}%'
liblognorm: type name is '@hex-byte'
liblognorm: type line to add: '%f1:hexnumber{"maxval": "255"}%'
liblognorm: ln_pdagFindType, name '@hex-byte', bAdd: 1, nTypes 0
liblognorm: custom type '@hex-byte' does not yet exist, adding...
liblognorm: addSampToTree 0 of 31
liblognorm: parsed literal: ''
liblognorm: parsed field: 'f1'
liblognorm: field type 'hexnumber', i 13
liblognorm: ln_pdagAddParserInternal: { "name": "f1", "type": "hexnumber", "maxval": "255" }
liblognorm: ln_pdagAddParserInstance: { "name": "f1", "type": "hexnumber", "maxval": "255" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176a410, parser 0x176aa60
liblognorm: parsed literal: ''
liblognorm: end addSampToTree 31 of 31
liblognorm: read rulebase line[~3]: 'rule=:a word %w1:word% a byte % .:@hex-byte % another word %w2:word%'
liblognorm: rule line to add: ':a word %w1:word% a byte % .:@hex-byte % another word %w2:word%'
liblognorm: addSampToTree 0 of 62
liblognorm: parsed literal: 'a word '
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "a" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "a" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x17690a0, parser 0x176a390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176a4c0, parser 0x176a390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "w" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "w" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176a590, parser 0x176a390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "o" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "o" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176a7d0, parser 0x176a390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "r" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "r" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176ae00, parser 0x176a690
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "d" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "d" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176a880, parser 0x176aeb0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176af40, parser 0x176aeb0
liblognorm: parsed field: 'w1'
liblognorm: field type 'word', i 15
liblognorm: ln_pdagAddParserInternal: { "name": "w1", "type": "word" }
liblognorm: ln_pdagAddParserInstance: { "name": "w1", "type": "word" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b0f0, parser 0x176aeb0
liblognorm: addSampToTree 16 of 62
liblognorm: parsed literal: ' a byte '
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176afd0, parser 0x176b0b0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "a" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "a" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176ad90, parser 0x176b2f0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b370, parser 0x176b410
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "b" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "b" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b060, parser 0x176b410
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "y" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "y" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b520, parser 0x176b6a0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "t" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "t" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b620, parser 0x176b770
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "e" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "e" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b5b0, parser 0x176b770
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b880, parser 0x176ba00
liblognorm: parsed field: '.'
liblognorm: field type '@hex-byte', i 38
liblognorm: ln_pdagAddParserInternal: { "name": ".", "type": "@hex-byte" }
liblognorm: ln_pdagAddParserInstance: { "name": ".", "type": "@hex-byte" }
liblognorm: ln_pdagFindType, name '@hex-byte', bAdd: 0, nTypes 1
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b980, parser 0x176ba00
liblognorm: addSampToTree 39 of 62
liblognorm: parsed literal: ' another word '
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176bb00, parser 0x176bcd0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "a" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "a" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176bc00, parser 0x176bd30
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "n" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "n" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b910, parser 0x176be00
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "o" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "o" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176be90, parser 0x176be00
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "t" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "t" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c040, parser 0x176be00
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "h" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "h" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c1d0, parser 0x176bfd0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "e" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "e" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176b210, parser 0x176c280
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "r" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "r" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c310, parser 0x176c280
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c4c0, parser 0x176c280
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "w" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "w" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c650, parser 0x176c450
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "o" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "o" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c160, parser 0x176c700
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "r" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "r" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c790, parser 0x176c700
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "d" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "d" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c940, parser 0x176c700
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176cad0, parser 0x176c8d0
liblognorm: parsed field: 'w2'
liblognorm: field type 'word', i 61
liblognorm: ln_pdagAddParserInternal: { "name": "w2", "type": "word" }
liblognorm: ln_pdagAddParserInstance: { "name": "w2", "type": "word" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0x176c5e0, parser 0x176c8d0
liblognorm: parsed literal: ''
liblognorm: end addSampToTree 62 of 62
liblognorm: optimizing component @hex-byte

liblognorm: pre sort, parser 0:f1[7680016]
liblognorm: post sort, parser 0:f1[7680016]
liblognorm: optimizing 0x176aae0: field 0 type 'hexnumber', name 'f1': 'UNKNOWN':
liblognorm: optimizing main pdag component
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: optimizing 0x176a4c0: field 0 type 'literal', name '(null)': 'a':
liblognorm: opt path compact: add 0x176a770 to 0x176aa60
liblognorm: delete 0x176a4c0[1]: (null)
liblognorm: opt path compact: add 0x176a820 to 0x176aa60
liblognorm: delete 0x176a590[1]: (null)
liblognorm: opt path compact: add 0x176a650 to 0x176aa60
liblognorm: delete 0x176a7d0[1]: (null)
liblognorm: opt path compact: add 0x176ae70 to 0x176aa60
liblognorm: delete 0x176ae00[1]: (null)
liblognorm: opt path compact: add 0x176af90 to 0x176aa60
liblognorm: delete 0x176a880[1]: (null)
liblognorm: opt path compact: add 0x176ad50 to 0x176aa60
liblognorm: delete 0x176af40[1]: (null)
liblognorm: pre sort, parser 0:w1[7680032]
liblognorm: post sort, parser 0:w1[7680032]
liblognorm: optimizing 0x176afd0: field 0 type 'word', name 'w1': 'UNKNOWN':
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: optimizing 0x176ad90: field 0 type 'literal', name '(null)': ' ':
liblognorm: opt path compact: add 0x176b3c0 to 0x176aeb0
liblognorm: delete 0x176ad90[1]: (null)
liblognorm: opt path compact: add 0x176b4a0 to 0x176aeb0
liblognorm: delete 0x176b370[1]: (null)
liblognorm: opt path compact: add 0x176b570 to 0x176aeb0
liblognorm: delete 0x176b060[1]: (null)
liblognorm: opt path compact: add 0x176b730 to 0x176aeb0
liblognorm: delete 0x176b520[1]: (null)
liblognorm: opt path compact: add 0x176b800 to 0x176aeb0
liblognorm: delete 0x176b620[1]: (null)
liblognorm: opt path compact: add 0x176b8d0 to 0x176aeb0
liblognorm: delete 0x176b5b0[1]: (null)
liblognorm: opt path compact: add 0x176ba90 to 0x176aeb0
liblognorm: delete 0x176b880[1]: (null)
liblognorm: pre sort, parser 0:.[7680016]
liblognorm: post sort, parser 0:.[7680016]
liblognorm: optimizing 0x176bb00: field 0 type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: optimizing 0x176bc00: field 0 type 'literal', name '(null)': ' ':
liblognorm: opt path compact: add 0x176bdc0 to 0x176ba00
liblognorm: delete 0x176bc00[1]: (null)
liblognorm: opt path compact: add 0x176bee0 to 0x176ba00
liblognorm: delete 0x176b910[1]: (null)
liblognorm: opt path compact: add 0x176b1b0 to 0x176ba00
liblognorm: delete 0x176be90[1]: (null)
liblognorm: opt path compact: add 0x176bf90 to 0x176ba00
liblognorm: delete 0x176c040[1]: (null)
liblognorm: opt path compact: add 0x176c240 to 0x176ba00
liblognorm: delete 0x176c1d0[1]: (null)
liblognorm: opt path compact: add 0x176c360 to 0x176ba00
liblognorm: delete 0x176b210[1]: (null)
liblognorm: opt path compact: add 0x176c100 to 0x176ba00
liblognorm: delete 0x176c310[1]: (null)
liblognorm: opt path compact: add 0x176c410 to 0x176ba00
liblognorm: delete 0x176c4c0[1]: (null)
liblognorm: opt path compact: add 0x176c6c0 to 0x176ba00
liblognorm: delete 0x176c650[1]: (null)
liblognorm: opt path compact: add 0x176c7e0 to 0x176ba00
liblognorm: delete 0x176c160[1]: (null)
liblognorm: opt path compact: add 0x176c580 to 0x176ba00
liblognorm: delete 0x176c790[1]: (null)
liblognorm: opt path compact: add 0x176c890 to 0x176ba00
liblognorm: delete 0x176c940[1]: (null)
liblognorm: opt path compact: add 0x176cb40 to 0x176ba00
liblognorm: delete 0x176cad0[1]: (null)
liblognorm: pre sort, parser 0:w2[7680032]
liblognorm: post sort, parser 0:w2[7680032]
liblognorm: optimizing 0x176cbb0: field 0 type 'word', name 'w2': 'UNKNOWN':
liblognorm: finished optimizing main pdag component
liblognorm: ---AFTER OPTIMIZATION------------------
liblognorm: COMPONENT: @hex-byte
liblognorm: subDAG 0x176a410 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm: field type 'hexnumber', name 'f1': 'UNKNOWN': called 0
liblognorm: field type 'hexnumber', name 'f1': 'UNKNOWN':
liblognorm:   subDAG [TERM] 0x176aae0 (children: 0 parsers, ref 1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT:
liblognorm: subDAG 0x17690a0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm: field type 'literal', name '(null)': 'a word ': called 0
liblognorm: field type 'literal', name '(null)': 'a word ':
liblognorm:   subDAG 0x176b0f0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:   field type 'word', name 'w1': 'UNKNOWN': called 0
liblognorm:   field type 'word', name 'w1': 'UNKNOWN':
liblognorm:     subDAG 0x176afd0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:     field type 'literal', name '(null)': ' a byte ': called 0
liblognorm:     field type 'literal', name '(null)': ' a byte ':
liblognorm:       subDAG 0x176b980 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:       field type 'USER-DEFINED', name '.': 'UNKNOWN': called 0
liblognorm:       field type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm:         subDAG 0x176bb00 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:         field type 'literal', name '(null)': ' another word ': called 0
liblognorm:         field type 'literal', name '(null)': ' another word ':
liblognorm:           subDAG 0x176c5e0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:           field type 'word', name 'w2': 'UNKNOWN': called 0
liblognorm:           field type 'word', name 'w2': 'UNKNOWN':
liblognorm:             subDAG [TERM] 0x176cbb0 (children: 0 parsers, ref 1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT (alternative):
liblognorm: 0x17690a0[ref 1]: 
liblognorm:   0x176b0f0[ref 1]: a word 
liblognorm:     0x176afd0[ref 1]: a word %w1:word%
liblognorm:       0x176b980[ref 1]: a word %w1:word% a byte 
liblognorm:         0x176bb00[ref 1]: a word %w1:word% a byte %.:USER-DEFINED%
liblognorm:           0x176c5e0[ref 1]: a word %w1:word% a byte %.:USER-DEFINED% another word 
liblognorm:             0x176cbb0[ref 1]: a word %w1:word% a byte %.:USER-DEFINED% another word %w2:word%
liblognorm: =======================================
number of tree nodes: 35
To normalize: 'a word w1 a byte 0xff another word w2'
liblognorm: 0: enter parser, dag node 0x17690a0, json 0x176aba0
liblognorm: 0/0:trying 'literal' parser for field '(null)', data 'a word '
liblognorm: parser lookup returns 0, pParsed 7
liblognorm: 0: potential hit, trying subtree 0x176b0f0
liblognorm: 7: enter parser, dag node 0x176b0f0, json 0x176aba0
liblognorm: 7/0:trying 'word' parser for field 'w1', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 2
liblognorm: 7: potential hit, trying subtree 0x176afd0
liblognorm: 9: enter parser, dag node 0x176afd0, json 0x176aba0
liblognorm: 9/0:trying 'literal' parser for field '(null)', data ' a byte '
liblognorm: parser lookup returns 0, pParsed 8
liblognorm: 9: potential hit, trying subtree 0x176b980
liblognorm: 17: enter parser, dag node 0x176b980, json 0x176aba0
liblognorm: 17/0:trying 'USER-DEFINED' parser for field '.', data 'UNKNOWN'
liblognorm: calling custom parser '@hex-byte'
liblognorm: 17: enter parser, dag node 0x176a410, json 0x176cc40
liblognorm: 17/1:trying 'hexnumber' parser for field 'f1', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 4
liblognorm: 17: potential hit, trying subtree 0x176aae0
liblognorm: 21: enter parser, dag node 0x176aae0, json 0x176cc40
liblognorm: offs 21, strLen 37, isTerm 1
liblognorm: 21 returns 0, pParsedTo 0, parsedTo 0
liblognorm: 17: subtree returns 0, parsedTo 21
liblognorm: 17: parser matches at 17
liblognorm: parsedTo 21, *pParsedTo 21
liblognorm: offs 17, strLen 37, isTerm 0
liblognorm: 17 returns 0, pParsedTo 21, parsedTo 21
liblognorm: called CUSTOM PARSER '@hex-byte', result 0, offs 17, *pParsed 0
liblognorm: parser lookup returns 0, pParsed 4
liblognorm: 17: potential hit, trying subtree 0x176bb00
liblognorm: 21: enter parser, dag node 0x176bb00, json 0x176aba0
liblognorm: 21/0:trying 'literal' parser for field '(null)', data ' another word '
liblognorm: parser lookup returns 0, pParsed 14
liblognorm: 21: potential hit, trying subtree 0x176c5e0
liblognorm: 35: enter parser, dag node 0x176c5e0, json 0x176aba0
liblognorm: 35/0:trying 'word' parser for field 'w2', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 2
liblognorm: 35: potential hit, trying subtree 0x176cbb0
liblognorm: 37: enter parser, dag node 0x176cbb0, json 0x176aba0
liblognorm: offs 37, strLen 37, isTerm 1
liblognorm: 37 returns 0, pParsedTo 0, parsedTo 0
liblognorm: 35: subtree returns 0, parsedTo 37
liblognorm: 35: parser matches at 35
liblognorm: parsedTo 37, *pParsedTo 37
liblognorm: offs 35, strLen 37, isTerm 0
liblognorm: 35 returns 0, pParsedTo 37, parsedTo 37
liblognorm: 21: subtree returns 0, parsedTo 35
liblognorm: 21: parser matches at 21
liblognorm: parsedTo 35, *pParsedTo 37
liblognorm: offs 21, strLen 37, isTerm 0
liblognorm: 21 returns 0, pParsedTo 37, parsedTo 35
liblognorm: 17: subtree returns 0, parsedTo 21
liblognorm: 17: parser matches at 17
liblognorm: parsedTo 21, *pParsedTo 37
liblognorm: offs 17, strLen 37, isTerm 0
liblognorm: 17 returns 0, pParsedTo 37, parsedTo 21
liblognorm: 9: subtree returns 0, parsedTo 17
liblognorm: 9: parser matches at 9
liblognorm: parsedTo 17, *pParsedTo 37
liblognorm: offs 9, strLen 37, isTerm 0
liblognorm: 9 returns 0, pParsedTo 37, parsedTo 17
liblognorm: 7: subtree returns 0, parsedTo 9
liblognorm: 7: parser matches at 7
liblognorm: parsedTo 9, *pParsedTo 37
liblognorm: offs 7, strLen 37, isTerm 0
liblognorm: 7 returns 0, pParsedTo 37, parsedTo 9
liblognorm: 0: subtree returns 0, parsedTo 7
liblognorm: 0: parser matches at 0
liblognorm: parsedTo 7, *pParsedTo 37
liblognorm: offs 0, strLen 37, isTerm 0
liblognorm: 0 returns 0, pParsedTo 37, parsedTo 7
liblognorm: final result for normalizer: parsedTo 37, endNode 0x176cbb0, isTerminal 1, tagbucket (nil)
liblognorm: DONE, final return is 0
normalized: '{ "w2": "w2", "f1": "0xff", "w1": "w1" }'
{ "w2": "w2", "f1": "0xff", "w1": "w1" }
liblognorm: exitCtx 0x1769010
liblognorm: delete 0x17690a0[1]: 
liblognorm: delete 0x176b0f0[1]: a word 
liblognorm: delete 0x176afd0[1]: a word %w1:word%
liblognorm: delete 0x176b980[1]: a word %w1:word% a byte 
liblognorm: delete 0x176bb00[1]: a word %w1:word% a byte %.:USER-DEFINED%
liblognorm: delete 0x176c5e0[1]: a word %w1:word% a byte %.:USER-DEFINED% another word 
liblognorm: delete 0x176cbb0[1]: a word %w1:word% a byte %.:USER-DEFINED% another word %w2:word%
liblognorm: delete 0x176a410[1]: 
liblognorm: delete 0x176aae0[1]: %f1:hexnumber%
mbiebl commented 8 years ago

The same test on s390x

liblognorm: loading rulebase file 'tmp.rulebase'
liblognorm: rulebase version is 2

liblognorm: read rulebase line[~2]: 'type=@hex-byte:%f1:hexnumber{"maxval": "255"}%'
liblognorm: type line to add: '@hex-byte:%f1:hexnumber{"maxval": "255"}%'
liblognorm: type name is '@hex-byte'
liblognorm: type line to add: '%f1:hexnumber{"maxval": "255"}%'
liblognorm: ln_pdagFindType, name '@hex-byte', bAdd: 1, nTypes 0
liblognorm: custom type '@hex-byte' does not yet exist, adding...
liblognorm: addSampToTree 0 of 31
liblognorm: parsed literal: ''
liblognorm: parsed field: 'f1'
liblognorm: field type 'hexnumber', i 13
liblognorm: ln_pdagAddParserInternal: { "name": "f1", "type": "hexnumber", "maxval": "255" }
liblognorm: ln_pdagAddParserInstance: { "name": "f1", "type": "hexnumber", "maxval": "255" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e92410, parser 0xa5e92a60
liblognorm: parsed literal: ''
liblognorm: end addSampToTree 31 of 31
liblognorm: read rulebase line[~3]: 'rule=:a word %w1:word% a byte % .:@hex-byte % another word %w2:word%'
liblognorm: rule line to add: ':a word %w1:word% a byte % .:@hex-byte % another word %w2:word%'
liblognorm: addSampToTree 0 of 62
liblognorm: parsed literal: 'a word '
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "a" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "a" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e910a0, parser 0xa5e92390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e924c0, parser 0xa5e92390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "w" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "w" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e92590, parser 0xa5e92390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "o" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "o" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e927d0, parser 0xa5e92390
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "r" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "r" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e92e00, parser 0xa5e92690
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "d" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "d" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e92880, parser 0xa5e92eb0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e92f40, parser 0xa5e92eb0
liblognorm: parsed field: 'w1'
liblognorm: field type 'word', i 15
liblognorm: ln_pdagAddParserInternal: { "name": "w1", "type": "word" }
liblognorm: ln_pdagAddParserInstance: { "name": "w1", "type": "word" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e930f0, parser 0xa5e92eb0
liblognorm: addSampToTree 16 of 62
liblognorm: parsed literal: ' a byte '
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e92fd0, parser 0xa5e930b0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "a" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "a" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e92d90, parser 0xa5e932f0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93370, parser 0xa5e93410
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "b" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "b" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93060, parser 0xa5e93410
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "y" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "y" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93520, parser 0xa5e936a0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "t" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "t" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93620, parser 0xa5e93770
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "e" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "e" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e935b0, parser 0xa5e93770
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93880, parser 0xa5e93a00
liblognorm: parsed field: '.'
liblognorm: field type '@hex-byte', i 38
liblognorm: ln_pdagAddParserInternal: { "name": ".", "type": "@hex-byte" }
liblognorm: ln_pdagAddParserInstance: { "name": ".", "type": "@hex-byte" }
liblognorm: ln_pdagFindType, name '@hex-byte', bAdd: 0, nTypes 1
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93980, parser 0xa5e93a00
liblognorm: addSampToTree 39 of 62
liblognorm: parsed literal: ' another word '
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93b00, parser 0xa5e93cd0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "a" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "a" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93c00, parser 0xa5e93d30
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "n" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "n" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93910, parser 0xa5e93e00
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "o" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "o" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93e90, parser 0xa5e93e00
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "t" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "t" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e94040, parser 0xa5e93e00
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "h" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "h" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e941d0, parser 0xa5e93fd0
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "e" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "e" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e93210, parser 0xa5e94280
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "r" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "r" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e94310, parser 0xa5e94280
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e944c0, parser 0xa5e94280
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "w" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "w" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e94650, parser 0xa5e94450
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "o" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "o" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e94160, parser 0xa5e94700
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "r" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "r" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e94790, parser 0xa5e94700
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": "d" }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": "d" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e94940, parser 0xa5e94700
liblognorm: ln_pdagAddParserInternal: { "type": "literal", "text": " " }
liblognorm: ln_pdagAddParserInstance: { "type": "literal", "text": " " }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e94ad0, parser 0xa5e948d0
liblognorm: parsed field: 'w2'
liblognorm: field type 'word', i 61
liblognorm: ln_pdagAddParserInternal: { "name": "w2", "type": "word" }
liblognorm: ln_pdagAddParserInstance: { "name": "w2", "type": "word" }
liblognorm: assigned priority is 30000
liblognorm: pdag: 0xa5e945e0, parser 0xa5e948d0
liblognorm: parsed literal: ''
liblognorm: end addSampToTree 62 of 62
liblognorm: optimizing component @hex-byte

liblognorm: pre sort, parser 0:f1[7680016]
liblognorm: post sort, parser 0:f1[7680016]
liblognorm: optimizing 0xa5e92ae0: field 0 type 'hexnumber', name 'f1': 'UNKNOWN':
liblognorm: optimizing main pdag component
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: optimizing 0xa5e924c0: field 0 type 'literal', name '(null)': 'a':
liblognorm: opt path compact: add 0xa5e92770 to 0xa5e92a60
liblognorm: delete 0xa5e924c0[1]: (null)
liblognorm: opt path compact: add 0xa5e92820 to 0xa5e92a60
liblognorm: delete 0xa5e92590[1]: (null)
liblognorm: opt path compact: add 0xa5e92650 to 0xa5e92a60
liblognorm: delete 0xa5e927d0[1]: (null)
liblognorm: opt path compact: add 0xa5e92e70 to 0xa5e92a60
liblognorm: delete 0xa5e92e00[1]: (null)
liblognorm: opt path compact: add 0xa5e92f90 to 0xa5e92a60
liblognorm: delete 0xa5e92880[1]: (null)
liblognorm: opt path compact: add 0xa5e92d50 to 0xa5e92a60
liblognorm: delete 0xa5e92f40[1]: (null)
liblognorm: pre sort, parser 0:w1[7680032]
liblognorm: post sort, parser 0:w1[7680032]
liblognorm: optimizing 0xa5e92fd0: field 0 type 'word', name 'w1': 'UNKNOWN':
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: optimizing 0xa5e92d90: field 0 type 'literal', name '(null)': ' ':
liblognorm: opt path compact: add 0xa5e933c0 to 0xa5e92eb0
liblognorm: delete 0xa5e92d90[1]: (null)
liblognorm: opt path compact: add 0xa5e934a0 to 0xa5e92eb0
liblognorm: delete 0xa5e93370[1]: (null)
liblognorm: opt path compact: add 0xa5e93570 to 0xa5e92eb0
liblognorm: delete 0xa5e93060[1]: (null)
liblognorm: opt path compact: add 0xa5e93730 to 0xa5e92eb0
liblognorm: delete 0xa5e93520[1]: (null)
liblognorm: opt path compact: add 0xa5e93800 to 0xa5e92eb0
liblognorm: delete 0xa5e93620[1]: (null)
liblognorm: opt path compact: add 0xa5e938d0 to 0xa5e92eb0
liblognorm: delete 0xa5e935b0[1]: (null)
liblognorm: opt path compact: add 0xa5e93a90 to 0xa5e92eb0
liblognorm: delete 0xa5e93880[1]: (null)
liblognorm: pre sort, parser 0:.[7680016]
liblognorm: post sort, parser 0:.[7680016]
liblognorm: optimizing 0xa5e93b00: field 0 type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm: pre sort, parser 0:(null)[7680004]
liblognorm: post sort, parser 0:(null)[7680004]
liblognorm: optimizing 0xa5e93c00: field 0 type 'literal', name '(null)': ' ':
liblognorm: opt path compact: add 0xa5e93dc0 to 0xa5e93a00
liblognorm: delete 0xa5e93c00[1]: (null)
liblognorm: opt path compact: add 0xa5e93ee0 to 0xa5e93a00
liblognorm: delete 0xa5e93910[1]: (null)
liblognorm: opt path compact: add 0xa5e931b0 to 0xa5e93a00
liblognorm: delete 0xa5e93e90[1]: (null)
liblognorm: opt path compact: add 0xa5e93f90 to 0xa5e93a00
liblognorm: delete 0xa5e94040[1]: (null)
liblognorm: opt path compact: add 0xa5e94240 to 0xa5e93a00
liblognorm: delete 0xa5e941d0[1]: (null)
liblognorm: opt path compact: add 0xa5e94360 to 0xa5e93a00
liblognorm: delete 0xa5e93210[1]: (null)
liblognorm: opt path compact: add 0xa5e94100 to 0xa5e93a00
liblognorm: delete 0xa5e94310[1]: (null)
liblognorm: opt path compact: add 0xa5e94410 to 0xa5e93a00
liblognorm: delete 0xa5e944c0[1]: (null)
liblognorm: opt path compact: add 0xa5e946c0 to 0xa5e93a00
liblognorm: delete 0xa5e94650[1]: (null)
liblognorm: opt path compact: add 0xa5e947e0 to 0xa5e93a00
liblognorm: delete 0xa5e94160[1]: (null)
liblognorm: opt path compact: add 0xa5e94580 to 0xa5e93a00
liblognorm: delete 0xa5e94790[1]: (null)
liblognorm: opt path compact: add 0xa5e94890 to 0xa5e93a00
liblognorm: delete 0xa5e94940[1]: (null)
liblognorm: opt path compact: add 0xa5e94b40 to 0xa5e93a00
liblognorm: delete 0xa5e94ad0[1]: (null)
liblognorm: pre sort, parser 0:w2[7680032]
liblognorm: post sort, parser 0:w2[7680032]
liblognorm: optimizing 0xa5e94bb0: field 0 type 'word', name 'w2': 'UNKNOWN':
liblognorm: finished optimizing main pdag component
liblognorm: ---AFTER OPTIMIZATION------------------
liblognorm: COMPONENT: @hex-byte
liblognorm: subDAG 0xa5e92410 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm: field type 'hexnumber', name 'f1': 'UNKNOWN': called 0
liblognorm: field type 'hexnumber', name 'f1': 'UNKNOWN':
liblognorm:   subDAG [TERM] 0xa5e92ae0 (children: 0 parsers, ref 1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT:
liblognorm: subDAG 0xa5e910a0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm: field type 'literal', name '(null)': 'a word ': called 0
liblognorm: field type 'literal', name '(null)': 'a word ':
liblognorm:   subDAG 0xa5e930f0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:   field type 'word', name 'w1': 'UNKNOWN': called 0
liblognorm:   field type 'word', name 'w1': 'UNKNOWN':
liblognorm:     subDAG 0xa5e92fd0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:     field type 'literal', name '(null)': ' a byte ': called 0
liblognorm:     field type 'literal', name '(null)': ' a byte ':
liblognorm:       subDAG 0xa5e93980 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:       field type 'USER-DEFINED', name '.': 'UNKNOWN': called 0
liblognorm:       field type 'USER-DEFINED', name '.': 'UNKNOWN':
liblognorm:         subDAG 0xa5e93b00 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:         field type 'literal', name '(null)': ' another word ': called 0
liblognorm:         field type 'literal', name '(null)': ' another word ':
liblognorm:           subDAG 0xa5e945e0 (children: 1 parsers, ref 1) [called 0, backtracked 0]
liblognorm:           field type 'word', name 'w2': 'UNKNOWN': called 0
liblognorm:           field type 'word', name 'w2': 'UNKNOWN':
liblognorm:             subDAG [TERM] 0xa5e94bb0 (children: 0 parsers, ref 1) [called 0, backtracked 0]
liblognorm: MAIN COMPONENT (alternative):
liblognorm: 0xa5e910a0[ref 1]: 
liblognorm:   0xa5e930f0[ref 1]: a word 
liblognorm:     0xa5e92fd0[ref 1]: a word %w1:word%
liblognorm:       0xa5e93980[ref 1]: a word %w1:word% a byte 
liblognorm:         0xa5e93b00[ref 1]: a word %w1:word% a byte %.:USER-DEFINED%
liblognorm:           0xa5e945e0[ref 1]: a word %w1:word% a byte %.:USER-DEFINED% another word 
liblognorm:             0xa5e94bb0[ref 1]: a word %w1:word% a byte %.:USER-DEFINED% another word %w2:word%
liblognorm: =======================================
number of tree nodes: 35
To normalize: 'a word w1 a byte 0xff another word w2'
liblognorm: 0: enter parser, dag node 0xa5e910a0, json 0xa5e92ba0
liblognorm: 0/0:trying 'literal' parser for field '(null)', data 'a word '
liblognorm: parser lookup returns 0, pParsed 7
liblognorm: 0: potential hit, trying subtree 0xa5e930f0
liblognorm: 7: enter parser, dag node 0xa5e930f0, json 0xa5e92ba0
liblognorm: 7/0:trying 'word' parser for field 'w1', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 2
liblognorm: 7: potential hit, trying subtree 0xa5e92fd0
liblognorm: 9: enter parser, dag node 0xa5e92fd0, json 0xa5e92ba0
liblognorm: 9/0:trying 'literal' parser for field '(null)', data ' a byte '
liblognorm: parser lookup returns 0, pParsed 8
liblognorm: 9: potential hit, trying subtree 0xa5e93980
liblognorm: 17: enter parser, dag node 0xa5e93980, json 0xa5e92ba0
liblognorm: 17/0:trying 'USER-DEFINED' parser for field '.', data 'UNKNOWN'
liblognorm: calling custom parser '@hex-byte'
liblognorm: 17: enter parser, dag node 0xa5e92410, json 0xa5e94c40
liblognorm: 17/1:trying 'hexnumber' parser for field 'f1', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 4
liblognorm: 17: potential hit, trying subtree 0xa5e92ae0
liblognorm: 21: enter parser, dag node 0xa5e92ae0, json 0xa5e94c40
liblognorm: offs 21, strLen 37, isTerm 1
liblognorm: 21 returns 0, pParsedTo 0, parsedTo 0
liblognorm: 17: subtree returns 0, parsedTo 21
liblognorm: 17: parser matches at 17
liblognorm: parsedTo 21, *pParsedTo 21
liblognorm: offs 17, strLen 37, isTerm 0
liblognorm: 17 returns 0, pParsedTo 21, parsedTo 21
liblognorm: called CUSTOM PARSER '@hex-byte', result 0, offs 17, *pParsed 0
liblognorm: parser lookup returns 0, pParsed 4
liblognorm: 17: potential hit, trying subtree 0xa5e93b00
liblognorm: 21: enter parser, dag node 0xa5e93b00, json 0xa5e92ba0
liblognorm: 21/0:trying 'literal' parser for field '(null)', data ' another word '
liblognorm: parser lookup returns 0, pParsed 14
liblognorm: 21: potential hit, trying subtree 0xa5e945e0
liblognorm: 35: enter parser, dag node 0xa5e945e0, json 0xa5e92ba0
liblognorm: 35/0:trying 'word' parser for field 'w2', data 'UNKNOWN'
liblognorm: parser lookup returns 0, pParsed 2
liblognorm: 35: potential hit, trying subtree 0xa5e94bb0
liblognorm: 37: enter parser, dag node 0xa5e94bb0, json 0xa5e92ba0
liblognorm: offs 37, strLen 37, isTerm 1
liblognorm: 37 returns 0, pParsedTo 0, parsedTo 0
liblognorm: 35: subtree returns 0, parsedTo 37
liblognorm: 35: parser matches at 35
liblognorm: parsedTo 37, *pParsedTo 37
liblognorm: offs 35, strLen 37, isTerm 0
liblognorm: 35 returns 0, pParsedTo 37, parsedTo 37
liblognorm: 21: subtree returns 0, parsedTo 35
liblognorm: 21: parser matches at 21
liblognorm: parsedTo 35, *pParsedTo 37
liblognorm: offs 21, strLen 37, isTerm 0
liblognorm: 21 returns 0, pParsedTo 37, parsedTo 35
liblognorm: 17: subtree returns 0, parsedTo 21
liblognorm: 17: parser matches at 17
liblognorm: parsedTo 21, *pParsedTo 37
liblognorm: offs 17, strLen 37, isTerm 0
liblognorm: 17 returns 0, pParsedTo 37, parsedTo 21
liblognorm: 9: subtree returns 0, parsedTo 17
liblognorm: 9: parser matches at 9
liblognorm: parsedTo 17, *pParsedTo 37
liblognorm: offs 9, strLen 37, isTerm 0
liblognorm: 9 returns 0, pParsedTo 37, parsedTo 17
liblognorm: 7: subtree returns 0, parsedTo 9
liblognorm: 7: parser matches at 7
liblognorm: parsedTo 9, *pParsedTo 37
liblognorm: offs 7, strLen 37, isTerm 0
liblognorm: 7 returns 0, pParsedTo 37, parsedTo 9
liblognorm: 0: subtree returns 0, parsedTo 7
liblognorm: 0: parser matches at 0
liblognorm: parsedTo 7, *pParsedTo 37
liblognorm: offs 0, strLen 37, isTerm 0
liblognorm: 0 returns 0, pParsedTo 37, parsedTo 7
liblognorm: final result for normalizer: parsedTo 37, endNode 0xa5e94bb0, isTerminal 1, tagbucket (nil)
liblognorm: DONE, final return is 0
normalized: '{ "w2": "w2", "f1": "0xff", "w1": "w1" }'
mbiebl commented 8 years ago

The test itself seems to be successful, but it get's stuck in the cleanup routine

liblognorm: exitCtx 0x1769010
liblognorm: delete 0x17690a0[1]: 
liblognorm: delete 0x176b0f0[1]: a word 
liblognorm: delete 0x176afd0[1]: a word %w1:word%
liblognorm: delete 0x176b980[1]: a word %w1:word% a byte 
liblognorm: delete 0x176bb00[1]: a word %w1:word% a byte %.:USER-DEFINED%
liblognorm: delete 0x176c5e0[1]: a word %w1:word% a byte %.:USER-DEFINED% another word 
liblognorm: delete 0x176cbb0[1]: a word %w1:word% a byte %.:USER-DEFINED% another word %w2:word%
liblognorm: delete 0x176a410[1]: 
liblognorm: delete 0x176aae0[1]: %f1:hexnumber%
mbiebl commented 8 years ago

Ok, I guess I've found the issue.

https://github.com/rsyslog/liblognorm/blob/master/src/lognormalizer.c#L177 contains

    char ch = 0;
    do {
        ch = fgetc(fp);
        if (ch == EOF) break;

This doesn't work. fgetc returns an int which is cast to an unsigned char.

The compiler complains

lognormalizer.c: In function 'read_line':
lognormalizer.c:185:10: warning: comparison is always false due to limited range of data type [-Wtype-limits]
   if (ch == EOF) break;
          ^~
  CCLD     lognormalizer

This means we never exit the loop and the test spins forever in that loop.

The fgetc() man page says:

fgetc() reads the next character from stream and returns it as an unsigned char cast to an int, or EOF on end of file or error.

So instead of a 'char' we should use and 'int'. Then the test suite completes successfully.