search

globus / globus-compute

Globus Compute: High Performance Function Serving for Science
https://www.globus.org/compute
Apache License 2.0
143 stars 47 forks source link

"leaked semaphore objects" error prevents endpoint start #362

Open mattebaughman opened 3 years ago

mattebaughman commented 3 years ago

Following the readme at: https://github.com/funcx-faas/funcX/tree/forwarder_rearch_1/funcx_sdk/funcx/tests

Only change to the readme code was editing the config.py for the "test_local" endpoint I created rather than the one mentioned (as it doesn't exist in my clean install).

MacOS 11.1, fresh conda (4.9.2) environment and funcx install with Python 3.8

When attempting to start the endpoint, I get the following error and it prevents startup.

(funcx_testing_py3.8) mattebaughmans-MacBook-Pro:funcX matt$ funcx-endpoint start test_local 2021-02-05 16:22:52 funcx.sdk.client.FuncXClient:79 [INFO] [instance:140382332256112] Creating client of type <class 'funcx.sdk.client.FuncXClient'> for service "funcX" 2021-02-05 16:22:53 funcx.sdk.client.FuncXClient:79 [INFO] [instance:140382334765040] Creating client of type <class 'funcx.sdk.client.FuncXClient'> for service "funcX" 2021-02-05 16:22:53 funcx:241 [INFO] Starting endpoint with uuid: d091a1a5-1d71-4ec2-8a7e-8739fbe73e4c (funcx_testing_py3.8) mattebaughmans-MacBook-Pro:funcX matt$ /Users/matt/opt/anaconda3/envs/funcx_testing_py3.8/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 6 leaked semaphore objects to clean up at shutdown

yongyanrao commented 3 years ago

I believe when this ticket was created funcx did not support funcx-endpoint on mac yet. Just worked on a PR to support mac endpoint. The leaked semaphore issue still exists.

2021-04-08 19:02:36 funcx.sdk.client.FuncXClient:79 [INFO]  [instance:140336758332192] Creating client of type <class 'funcx.sdk.client.FuncXClient'> for service "funcX"
2021-04-08 19:02:36 funcx:162 [INFO]  Starting endpoint with uuid: fe2febb4-8576-461c-af04-7162cdac2026
2021-04-08 19:02:37 funcx:227 [INFO]  Launching endpoint daemon process
(funcx-mac) willrmac:funcx_endpoint willr$ /Users/willr/miniconda3/envs/funcx-mac/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: Ther
e appear to be 6 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

After the PR merge, it should work for python=3.7

yadudoc commented 3 years ago

@mattebaughman, Do you think you might be able to give @yongyanrao 's fixes in PR: https://github.com/funcx-faas/funcX/pull/423 a try?

mattebaughman commented 3 years ago

Sure, will do that and let you know.

On Wed, May 5, 2021 at 11:34 AM Yadu Nand Babuji @.***> wrote:

@mattebaughman https://github.com/mattebaughman, Do you think you might be able to give @yongyanrao https://github.com/yongyanrao 's fixes in PR: #423 https://github.com/funcx-faas/funcX/pull/423 a try?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/funcx-faas/funcX/issues/362#issuecomment-832837050, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFOQIQ2EFYZOPGQ6NV3UBHLTMFXR3ANCNFSM4XFRUSHA .

yadudoc commented 3 years ago

423 closes this.

yadudoc commented 3 years ago

Reopening this issue since @yongyanrao's fix does not address the leaked semaphore issue with Python3.8+