search

oracle / oci-hdfs-connector

HDFS Connector for Oracle Cloud Infrastructure
https://cloud.oracle.com/cloud-infrastructure
Other
27 stars 26 forks source link

Better Handle BmcFileSystemImpl Close & Cache Invalidate to prevent thread leak #110

Open eaugene opened 1 week ago

eaugene commented 1 week ago

Fixes : https://github.com/oracle/oci-hdfs-connector/issues/109

When debugging the thread leak we were able to see the executorService which creates the threads are not shutdown when closing, which can lead to this issue.

Good Read about this : MEMORY LEAK IN JAVA EXECUTOR! - GC easy

However attempting to close the executorService Instances in BmcFileSystemImpl close, had another problem with cache invalidation logic.

Problem With Cache invalidation

The current cache size is set as 0 with the cache disabled and in the removal listener we are closing the BmcFileSystemImpl , so the ExecutorService also gets closed even at the creation, leading to not being able to make operations.

Will this issue ( ExecService close at creation ) exist if the cache is enabled?

With cache enabled, we’ll hit this issue once evicting the BmcFileSystemImpl & in the process of closing if we close the ExecService. So enabling caching is not a solution to avoid this issue

How this did not affect till now?

We are using the closed BmcFileSystemImpl , as the Executor service is open we are able to make I/O calls using the underlying threads ( this only has null check & not isClosed() check - This PR fixes this as well )

Anti Pattern of child Object clearing all the holding ( parent) objects

At BmcFileSystemImpl.close() we can see it closes all the Objects BmcFileSystem's which are holding the implementations . This causes problems to the clients which hold that .

cc : @yanhaizhongyu

oracle-contributor-agreement[bot] commented 1 week ago

Thank you for your pull request and welcome to our community! To contribute, please sign the Oracle Contributor Agreement (OCA). The following contributors of this PR have not signed the OCA:

To sign the OCA, please create an Oracle account and sign the OCA in Oracle's Contributor Agreement Application.

When signing the OCA, please provide your GitHub username. After signing the OCA and getting an OCA approval from Oracle, this PR will be automatically updated.

If you are an Oracle employee, please make sure that you are a member of the main Oracle GitHub organization, and your membership in this organization is public.