search

antchfx / xpath

XPath package for Golang, supports HTML, XML, JSON document query.
MIT License
692 stars 88 forks source link

memory leak or endless loop in v1.3.0 #98

Closed rny closed 3 months ago

rny commented 4 months ago

after upgrade xpath v1.2.5 => v1.3.0 ( not change anything else) our app memory keep increasing until OOM and CPU keeps 100%

I found a new loop was added in func (t *NodeIterator) MoveNext() bool is it cause endless loop in some case?

zhengchun commented 4 months ago

v1.3.0 changes a lot of code. https://github.com/antchfx/xpath/commits/master/

The best way is give your xpath expression to cause CPU high, let me test it.

zhengchun commented 4 months ago

can you remove this code to disable caches and test again? https://github.com/antchfx/xpath/blob/4b4638b370e898a9d10709464b8cd460b7dcfd0c/xpath.go#L96

rny commented 4 months ago

I noticed 1.3.1 removed that loop. I have tested these upgrades

go: upgraded github.com/antchfx/xmlquery v1.3.18 => v1.4.1
go: upgraded github.com/antchfx/xpath v1.2.5 => v1.3.1

the v1.3.1 CPU and memory is ok

zhengchun commented 4 months ago

I guess the high CPU is caused by the thread deadlock when application run in the multiple goroutines that reused cached expr object. so I removed those loop code. https://github.com/antchfx/htmlquery/issues/58

rny commented 4 months ago

agree. our app use xpath inside go { } a lot.