search

dlang-community / D-YAML

YAML parser and emitter for the D programming language
https://dlang-community.github.io/D-YAML/
Boost Software License 1.0
119 stars 37 forks source link

core.exception.AssertError: prefixBytes out of bounds #257

Closed FigmentEngine closed 4 years ago

FigmentEngine commented 4 years ago 
core.exception.AssertError@/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/reader.d(235): prefixBytes out of bounds
----------------
??:? _d_assert_msg [0x55c931ab9ab2]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/reader.d:235 pure nothrow @nogc @safe void dyaml.reader.Reader.prefixBytes(const(ulong)).__require(ref const(ulong)) [0x55c931a6a66c]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/reader.d:235 pure nothrow @nogc @safe char[] dyaml.reader.Reader.prefixBytes(const(ulong)) [0x55c931a6a598]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:1585 pure @safe void dyaml.scanner.Scanner.scanPlainSpacesToSlice() [0x55c931a710b3]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:1559 pure @safe dyaml.token.Token dyaml.scanner.Scanner.scanPlain() [0x55c931a70f55]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:655 @safe void dyaml.scanner.Scanner.fetchPlain() [0x55c931a6e174]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:244 @safe void dyaml.scanner.Scanner.fetchToken() [0x55c931a6d036]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:181 @safe bool dyaml.scanner.Scanner.empty() [0x55c931a6cdb5]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:172 @safe const(dyaml.token.Token) dyaml.scanner.Scanner.front() [0x55c931a6cd24]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/parser.d:753 @safe dyaml.event.Event dyaml.parser.Parser.parseBlockMappingValue() [0x55c931a69814]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/parser.d:182 @safe void dyaml.parser.Parser.ensureState() [0x55c931a66be4]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/parser.d:173 @safe void dyaml.parser.Parser.popFront() [0x55c931a66b8e]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:207 @safe dyaml.node.Node dyaml.composer.Composer.composeScalarNode() [0x55c931a746e5]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:185 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a743c2]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:341 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x55c931a7506f]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a74455]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:342 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x55c931a750bb]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a74455]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:342 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x55c931a750bb]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a74455]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:342 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x55c931a750bb]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a74455]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:342 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x55c931a750bb]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a74455]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:342 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x55c931a750bb]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a74455]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:342 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x55c931a750bb]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x55c931a74455]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:133 @safe dyaml.node.Node dyaml.composer.Composer.composeDocument() [0x55c931a73db1]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:100 @safe dyaml.node.Node dyaml.composer.Composer.getNode() [0x55c931a73c2b]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/loader.d:229 @safe void dyaml.loader.Loader.popFront() [0x55c931a649e5]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/loader.d:208 @safe bool dyaml.loader.Loader.empty() [0x55c931a64863]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/loader.d:190 @safe dyaml.node.Node dyaml.loader.Loader.load() [0x55c931a64784]

caused when loading this: https://github.com/awslabs/serverless-application-model/blob/master/tests/translator/input/error_cognito_userpool_duplicate_trigger.yaml

File Contents:

Resources:
  UserPool:
    Type: AWS::Cognito::UserPool

  ImplicitApiFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: s3://sam-demo-bucket/member_portal.zip
      Handler: index.gethtml
      Runtime: nodejs12.x
      Events:
        OneTrigger:
          Type: Cognito
          Properties:
            UserPool: 
              Ref: UserPool
            Trigger: PreSignUp
        TwoTrigger:
          Type: Cognito
          Properties:
            UserPool: 
              Ref: UserPool
            Trigger: PreSignUp
Herringway commented 4 years ago

I cannot reproduce this. can you provide your code as well?

FigmentEngine commented 4 years ago

let me produce a harness with minimal code that does this, will post tomorrow

FigmentEngine commented 4 years ago

Ok, was able to recreate the problem. passing a file with a last line that has trailing spaces (0x20) but NO final linefeed causes the problem.

code

import std.stdio;
import dyaml;
import std.file;

int main(string[] args)
{
 auto yamlFilename = args[1];

 auto yaml = readText(yamlFilename);

 auto root = Loader.fromString(yaml).load();

 return root.isValid;
}

created a script to show this:

echo "hello: world" > test.yml
hd test.yml
echo "Running with \w\n"
dub run -- ./test.yml
echo "Running with \w\W\n"
echo "hello: world " > test.yml
dub run -- ./test.yml
hd test.yml
truncate -s-1 test.yml
hd test.yml
echo "Running with \w\W"
dub run -- ./test.yml

results:

$ ./test.sh
00000000  68 65 6c 6c 6f 3a 20 77  6f 72 6c 64 0a           |hello: world.|
0000000d
Running with \w\n
Performing "debug" build using /home/fitz/dlang/dmd-2.092.1/linux/bin64/dmd for x86_64.
tinyendian 0.2.0: target for configuration "library" is up to date.
dyaml 0.8.1: target for configuration "library" is up to date.
dyaml-harness ~master: target for configuration "application" is up to date.
To force a rebuild of up-to-date targets, run again with --force.
Running ./dyaml-harness ./test.yml
Program exited with code 1
Running with \w\W\n
Performing "debug" build using /home/fitz/dlang/dmd-2.092.1/linux/bin64/dmd for x86_64.
tinyendian 0.2.0: target for configuration "library" is up to date.
dyaml 0.8.1: target for configuration "library" is up to date.
dyaml-harness ~master: target for configuration "application" is up to date.
To force a rebuild of up-to-date targets, run again with --force.
Running ./dyaml-harness ./test.yml
Program exited with code 1
00000000  68 65 6c 6c 6f 3a 20 77  6f 72 6c 64 20 0a        |hello: world .|
0000000e
00000000  68 65 6c 6c 6f 3a 20 77  6f 72 6c 64 20           |hello: world |
0000000d
Running with \w\W
Performing "debug" build using /home/fitz/dlang/dmd-2.092.1/linux/bin64/dmd for x86_64.
tinyendian 0.2.0: target for configuration "library" is up to date.
dyaml 0.8.1: target for configuration "library" is up to date.
dyaml-harness ~master: target for configuration "application" is up to date.
To force a rebuild of up-to-date targets, run again with --force.
Running ./dyaml-harness ./test.yml
core.exception.AssertError@/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/reader.d(235): prefixBytes out of bounds
----------------
??:? _d_assert_msg [0x5583d977fd92]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/reader.d:235 pure nothrow @nogc @safe void dyaml.reader.Reader.prefixBytes(const(ulong)).__require(ref const(ulong)) [0x5583d96fab80]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/reader.d:235 pure nothrow @nogc @safe char[] dyaml.reader.Reader.prefixBytes(const(ulong)) [0x5583d96faaac]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:1585 pure @safe void dyaml.scanner.Scanner.scanPlainSpacesToSlice() [0x5583d97015c7]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:1559 pure @safe dyaml.token.Token dyaml.scanner.Scanner.scanPlain() [0x5583d9701469]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:655 @safe void dyaml.scanner.Scanner.fetchPlain() [0x5583d96fe688]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:244 @safe void dyaml.scanner.Scanner.fetchToken() [0x5583d96fd54a]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:181 @safe bool dyaml.scanner.Scanner.empty() [0x5583d96fd2c9]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/scanner.d:172 @safe const(dyaml.token.Token) dyaml.scanner.Scanner.front() [0x5583d96fd238]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/parser.d:753 @safe dyaml.event.Event dyaml.parser.Parser.parseBlockMappingValue() [0x5583d96f9d28]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/parser.d:182 @safe void dyaml.parser.Parser.ensureState() [0x5583d96f70f8]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/parser.d:173 @safe void dyaml.parser.Parser.popFront() [0x5583d96f70a2]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:207 @safe dyaml.node.Node dyaml.composer.Composer.composeScalarNode() [0x5583d9704bf9]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:185 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x5583d97048d6]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:341 @safe dyaml.node.Node dyaml.composer.Composer.composeMappingNode(const(uint), const(uint)) [0x5583d9705583]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:191 @safe dyaml.node.Node dyaml.composer.Composer.composeNode(const(uint), const(uint)) [0x5583d9704969]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:133 @safe dyaml.node.Node dyaml.composer.Composer.composeDocument() [0x5583d97042c5]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/composer.d:100 @safe dyaml.node.Node dyaml.composer.Composer.getNode() [0x5583d970413f]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/loader.d:229 @safe void dyaml.loader.Loader.popFront() [0x5583d96f4add]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/loader.d:208 @safe bool dyaml.loader.Loader.empty() [0x5583d96f495b]
/home/fitz/.dub/packages/dyaml-0.8.1/dyaml/source/dyaml/loader.d:190 @safe dyaml.node.Node dyaml.loader.Loader.load() [0x5583d96f487c]
source/app.d:11 _Dmain [0x5583d96e2f4a]
Program exited with code 1
(dmd-2.092.1)10:34:09(563) ~/Creo/projects/misc/test/dyaml-harness