bonede / tree-sitter-ng

Next generation Tree Sitter Java binding.
MIT License
50 stars 4 forks source link

Periodic Crashes in Production Environment #25

Open csophys opened 1 week ago

csophys commented 1 week ago

I am using TreeSitter in a production environment. However, during actual use, periodic crashes occur. Each crash file is about 5GB, I hope to find the cause and thoroughly resolve this issue. Currently, I have not pinpointed the specific request logs that lead to the crashes.

Could you please help me understand how to resolve this problem? Or what can I do to provide more information to help identify the issue?

image

csophys commented 1 week ago

Here are two additional scenarios where clashes occur:

Coredump occurring during the Cleaner process. image

Coredump occurring when accessing the child of a node. image

jingyuan4ever commented 1 week ago

I have encountered same problem.

bonede commented 4 days ago

Could you share the source code and code snippets that crashed the program?

jingyuan4ever commented 2 days ago

Here is my code


    private static String tryRecoverFromTheError(Language language, String before, String after, String firstLine) {
        // try to recover from error
        List<TSNode> errorNodes = null;
        try {
            TSTree cst = TreeSitterUtilsByLanguage.parse(language, before + firstLine + after);
            TSTreeCursor unit = getUnit(language, cst, before.getBytes(StandardCharsets.UTF_8).length);
            TSTreeCursor cursor = new TSTreeCursor(unit.currentNode());
            errorNodes = new ArrayList<>();
            List<TSNode> missingNodes = new ArrayList<>();
            boolean loop = true;
            while (loop) {
                if (cursor.currentNode().hasError()) {
                    if (cursor.currentNode().isMissing()) {
                        missingNodes.add(cursor.currentNode());
                    } else {
                        errorNodes.add(cursor.currentNode());
                    }
                }
                if (cursor.gotoFirstChild() || cursor.gotoNextSibling()) {
                    continue;
                }
                do {
                    if (!cursor.gotoParent() || cursor.currentNode().equals(unit.currentNode())) {
                        loop = false;
                        break;
                    }
                } while (!cursor.gotoNextSibling());
            }
            for (TSNode missing : missingNodes) {
                List<TSNode> notRecovered = new ArrayList<>();
                for (TSNode error : errorNodes) {
                    if (!(error.getStartByte() <= missing.getStartByte() && error.getEndByte() >= missing.getEndByte() && error.getStartByte() != missing.getStartByte())) {
                        notRecovered.add(error);
                    }
                }
                errorNodes = notRecovered;
            }
        } catch (Exception e) {
            throw new RuntimeException(e);
        }
        if (errorNodes.isEmpty()) {
            return firstLine;
        } else {
            return "";
        }
    }