JuliaGeo / LibGEOS.jl

Julia package for manipulation and analysis of planar geometric objects
MIT License
72 stars 23 forks source link

Out of memory after intersects(s1, s2) #206

Open Ronneesley opened 5 months ago

Ronneesley commented 5 months ago

Hello,

First of all, congrats for your work. It's owesome!

I'm working with your module, and need check 100 thousands of intersections. After doing my loop, my memory is empty, it's about 32 GB. Although the shapes are small. Here a sample of my code that the problem can be observed:

for i = 1:100000
    LibGEOS.intersects(shape2, shape1)
end

I observed that the problem happends even when I check the intersection of two small shapes many times.

I don't know why the problem happens, but I suspect of some global data. Inspecting the code at: https://github.com/JuliaGeo/LibGEOS.jl/blob/master/src/geos_functions.jl, I saw the context variable that could be linked at shape, that is a global variable at my code, because it is at a dataframe (Dataframes.jl), I read like this:

using Shapefile, DataFrames
table = Shapefile.Table(path)
df = DataFrame(table)

If it's the problem, how could I clear this context? If it isn't the problem, how can I check the interssection without lack of memory?

Ronneesley commented 5 months ago

Note: There is a difference os memory usage of:

for i = 1:100000
    LibGEOS.intersects(shape1, shape2)
end

and

for i = 1:100000
    LibGEOS.intersects(shape2, shape1)
end
asinghvi17 commented 5 months ago

What's the average number of vertices for your shapes? Have you tried other geometry libraries like ArchGDAL.jl?

Ronneesley commented 5 months ago

Hi @asinghvi17 ,

shape2 has 17 points and shape1 4262.

No I didn't know ArchGDAL, I'll try now.

Ronneesley commented 5 months ago

Hello @asinghvi17 ,

I tried use ArchGDAL.jl and memory remains stable. My code was:

shape1n = ArchGDAL.createmultipolygon(GeoInterface.coordinates(shape1))
shape2n = ArchGDAL.createmultipolygon(GeoInterface.coordinates(shape2))

for i = 1:100000
    ArchGDAL.intersects(shape1n, shape2n)
end

and it runned very stable.

asinghvi17 commented 5 months ago

Great! In this case, you should use GeoInterface.convert(ArchGDAL, shape1) - it's more efficient.

Looks like there is a memory leak in the Julia wrapper of LibGEOS (this package), since ArchGDAL depends on the underlying C library GEOS.

evetion commented 5 months ago

Relates to #184