goerwin / directory-validator

Tool to validate directory structures
MIT License
15 stars 3 forks source link

Maximum call stack size exceeded #19

Open lxnagel opened 1 month ago

lxnagel commented 1 month ago

Running in a bookworm-tls node container, and this is what I got after adding "all the folders" I want to check. Installed the npm package from npmjs. Output

Performing Check

    Error:
     - Maximum call stack size exceeded

And this is the folder/file struct I have drafted and does exist. Somehow the last folder here has a larger set of files and sub-folders. Any recommendation?

{
    "ignoreFiles": [],
    "ignoreDirs": [],
    "commonRules": {},
    "rules": [
        {
            "type": "file",
            "name": "CHANGELOG.md"
        },
        {
            "type": "file",
            "name": "README.html"
        },
        {
            "type": "directory",
            "name": "Binaries",
            "rules": [
                {
                    "type": "file",
                    "name": "Prod.bin"
                },
                {
                    "type": "file",
                    "name": "Prod.elf"
                },
                {
                    "type": "file",
                    "name": "Prod.hex"
                },
                {
                    "type": "file",
                    "name": "Prod.list"
                },
                {
                    "type": "file",
                    "name": "Prod.map"
                }
            ]
        },
        {
            "type": "directory",
            "name": "3rdParty",
            "rules": [
                {
                    "type": "file",
                    "name": "*.md",
                    "isOptional": true
                },
                {
                    "type": "file",
                    "name": "/Thing_v(.*)/",
                    "extension": "bin"
                }
            ]
        },
        {
            "type": "directory",
            "name": "Doc",
            "rules": [
                {
                    "type": "file",
                    "name": "Documentation.html"
                },
                {
                    "type": "directory",
                    "name": "html",
                    "isRecursive": true
                }
            ]
        },
        {
            "type": "directory",
            "name": "Tool2"
        },
        {
            "type": "directory",
            "name": "APIDoc"
        },
        {
            "type": "directory",
            "name": "Metrics"
        },
        {
            "type": "directory",
            "name": "Scripts"
        },
        {
            "type": "directory",
            "name": "TestReports"
        }
    ]
}
goerwin commented 1 month ago

@lxnagel can you provide the folders/files structure you are checking against? or the minimum folder setup where you can reproduce this bug?

lxnagel commented 1 month ago

Working on it without providing the real data. While doing this I found something where I'm not sure if bug or feature:

{
            "type": "directory",
            "name": "TestReports",
            "rules": [
                {
                    "type": "directory",
                    "name": "NUnit",
                    "rules": []
                },
                {
                    "type": "directory",
                    "name": "StaticCode",
                    "rules": []
                },
                {
                    "type": "directory",
                    "name": "UnitTests",
                    "rules": []
                }
            ]
        }

empty_folder.zip this leads to no successfukl validation if the test.log file in TestReports/NUnit is created. Is an emtpy folder tree not allowed? Somehow at least one file seems to be needed.

lxnagel commented 1 month ago

In my real-world thing, there are 600 files in deeper folders (TestReports is 3 layers deep), yet it also crashes with dummy files: max_stack_folder.zip

Parallel I would now to try debugging it as well.

lxnagel commented 1 month ago

While debugging I found something: the code does build the ascii-tree regardless required or not. Yet in the helper file module, the string is not being built successfully. I was able to get the rules to be verified when I limit the file-list to e.g. size 1. Then the script was executing. Proceeding to find a solution about building the ascii-string.

edit: I did not yet "fix" the helper function to be not recursive as I noticed in my example there are 13k files