schrodinger / coordgenlibs

Schrodinger-developed 2D Coordinate Generation
BSD 3-Clause "New" or "Revised" License
42 stars 28 forks source link

NaN coordinates crash coordgen #124

Open bjonnh-work opened 1 year ago

bjonnh-work commented 1 year ago

Related to : https://github.com/rdkit/rdkit/issues/4845 (@greglandrum) and https://github.com/schrodinger/coordgenlibs/issues/106

On arm64 only (this doesn't happen on amd64), we see failures in sketcherMinimizerAtom::clockwiseOrderedNeighbors (in sketcherMinimizerAtom) where sketcherMinimizerMaths::signedAngle returns NaN but this method can't handle it. What happens is that the loop doesn't trigger the if that sets lastPoppedIndex so it keeps the value to its previous one.

when the value was 1, and neighs only has one element left, it tries to delete an element out of the vector. And when the glibc destructor hits at the exit of the function we get a "double free or corruption (out)".

By resetting lastPoppedIndex to 0 before that loop, we avoid the issue and the tests pass (at least the rdkit ones).

while (neighs.size()) {
        float smallestAngle = 361;
        lastPoppedIndex = 0;
        for (unsigned int i = 0; i < neighs.size(); i++) {
            float newAngle = sketcherMinimizerMaths::signedAngle(
                lastPoppedAtom->coordinates, coordinates,
                neighs[i]->coordinates);
            if (newAngle < 0) {
                newAngle += 360;
            }
            if (newAngle < smallestAngle) {
                smallestAngle = newAngle;
                lastPoppedIndex = i;
            }
        }
        lastPoppedAtom = neighs[lastPoppedIndex];
        orderedNeighs.push_back(lastPoppedAtom);
        neighs.erase(neighs.begin() + lastPoppedIndex);
    }

We could also check for NaN and set to 0 in that specific case.

I can make a PR if you think that's a viable solution.