search

statsig-io / java-server-sdk

An SDK for Java or Kotlin intended for multi-user/server environments
ISC License
7 stars 3 forks source link

Possible memory leak #27

Closed Natalia588 closed 11 minutes ago

Natalia588 commented 1 week ago

Hello,

We've recently enabled the statsig in our microservice. And we see that our pod's memory usage is constantly increasing. We've updated the statsig SDK to the latest version, but still the same. It appears to be a classloader leak and is probably related to the ip3country.CountryLookup:

image

Could you please share your opinion about it? Why can it happen?

Our service performs statsig checks very often. And for every check we create a new StatsigUser with the appropriate tenantId - our feature gates perform checks based on the tenant. Maybe, it's better to store the StatsigUser instance and reuse it instead of instantiation for every statsig call. What would you recommend?

Natalia588 commented 1 week ago

Most likely the classloader leak is related to https://github.com/statsig-io/ip3country-kotlin/blob/d74b35b52de8d26997c783eded96b702ca75a625/src/main/kotlin/CountryLookup.kt#L17

Natalia588 commented 1 week ago

In our service we are using the UnitID criteria, so we do not need IP lookup (ConfigCondition.IP_BASED) at all. How can we set the ConfigCondition.UNIT_ID to the StatsigUser object to avoid calls to CountryLookup for every statsig check?

tore-statsig commented 1 week ago

We've been debugging a memory leak in the java server sdk so this is very timely. We're continuing to look today.

In terms of your gate, can you link to it? I agree that it shouldnt be hitting the ip lookup if its just an id check. Is it possible any gate in your project hits this?

tore-statsig commented 1 week ago

Does your service or webserver shutdown/restart, and is that when you are observing the leak? Or is it while you are running the service for an extended period of time?

Natalia588 commented 1 week ago

Our service uses only the feature gates with UnitID criteria (based on tenantID).

Memory usage constantly increases during the service run, unrelated to service shutdown/restart.

tore-statsig commented 1 week ago

We havent been able to reproduce this - do you have any more details for how you got this? When did you capture this snapshot? What sort of QPS did you have for gate checks while the server was running?

weihao-statsig commented 6 days ago

Hi @Natalia588 Please check out our newest release, https://github.com/statsig-io/java-server-sdk/releases/tag/1.21.0 We've provided a new statsigOption to disable ip3 country library.

Let me know if after disabling ip3 country library the pod's memory usage is still increasing

Natalia588 commented 6 days ago

Thank you. We will test the new version and let you know about the results.

weihao-statsig commented 5 days ago

@Natalia588 Thanks, Is it getting better now?

anmannsb commented 5 days ago

Thanks @weihao-statsig Seems like it's better now, but I would like to check it on distance. At least I don't see Country Lockup in heap dumps anymore.

We will keep you updated!