search

kubkon / zig-yaml

YAML parser for Zig
MIT License
139 stars 37 forks source link

parser: token `map_value_ind` handled incorrectly inside a scalar (str) #33

Open perillo opened 11 months ago

perillo commented 11 months ago

Here is a simple example:

test:
  time: 04:30
$ zig build run -- test.yaml
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.literal, .start = 0, .end = 4 }
debug(tokenizer):     | test
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.map_value_ind, .start = 4, .end = 5 }
debug(tokenizer):     | :
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.new_line, .start = 5, .end = 6 }
debug(tokenizer):     | 

debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.space, .start = 6, .end = 8 }
debug(tokenizer):     |   
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.literal, .start = 8, .end = 12 }
debug(tokenizer):     | time
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.map_value_ind, .start = 12, .end = 13 }
debug(tokenizer):     | :
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.space, .start = 13, .end = 14 }
debug(tokenizer):     |  
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.literal, .start = 14, .end = 16 }
debug(tokenizer):     | 04
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.map_value_ind, .start = 16, .end = 17 }
debug(tokenizer):     | :
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.literal, .start = 17, .end = 19 }
debug(tokenizer):     | 30
debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.new_line, .start = 19, .end = 20 }
debug(tokenizer):     | 

debug(tokenizer): Tokenizer.Token{ .id = Tokenizer.Token.Id.eof, .start = 20, .end = 20 }
debug(tokenizer):     | 
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse): (main) next literal@0
debug(parse): (doc) begin literal@0
debug(parse): eatToken('doc_start')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse):   (not found)
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse):   next literal@0
debug(parse): eatToken('map_value_ind')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'map_value_ind')
debug(parse):   (found at 1)
debug(parse): (map) begin literal@0
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse): (map) key test@0
debug(parse): expectToken('map_value_ind')
debug(parse): eatToken('map_value_ind')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'map_value_ind')
debug(parse):   (found at 1)
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'new_line')
debug(parse):   (token 'space')
debug(parse):   (token 'literal')
debug(parse):   next literal@4
debug(parse): eatToken('map_value_ind')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'map_value_ind')
debug(parse):   (found at 5)
debug(parse): (map) begin literal@4
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse): (map) key time@4
debug(parse): expectToken('map_value_ind')
debug(parse): eatToken('map_value_ind')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'map_value_ind')
debug(parse):   (found at 5)
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'space')
debug(parse):   (token 'literal')
debug(parse):   next literal@7
debug(parse): eatToken('map_value_ind')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'map_value_ind')
debug(parse):   (found at 8)
debug(parse): (map) begin literal@7
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse): (map) key 04@7
debug(parse): expectToken('map_value_ind')
debug(parse): eatToken('map_value_ind')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'map_value_ind')
debug(parse):   (found at 8)
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'literal')
debug(parse):   next literal@9
debug(parse): eatToken('map_value_ind')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'new_line')
debug(parse):   (not found)
debug(parse): (leaf) 30
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'new_line')
debug(parse):   (token 'eof')
debug(parse): (map) end new_line@10
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'eof')
debug(parse): (map) end new_line@10
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'eof')
debug(parse): (map) end new_line@10
debug(parse): eatToken('doc_end')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'eof')
debug(parse):   (not found)
debug(parse): eatToken('doc_start')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'eof')
debug(parse):   (not found)
debug(parse): eatToken('eof')
debug(parse): eatCommentsAndSpace
debug(parse):   (token 'eof')
debug(parse):   (found at 11)
debug(parse): (doc) end new_line@10
debug(parse): eatCommentsAndSpace
---
test: 
    time: 
        04: 30
...