search

globus / globus-compute

Globus Compute: High Performance Function Serving for Science
https://www.globus.org/compute
Apache License 2.0
148 stars 47 forks source link

Run tests on linux [Ubuntu] #348

Open yadudoc opened 3 years ago

yadudoc commented 3 years ago

Launch funcx-endpoint following instructions here: https://github.com/funcx-faas/funcX/tree/forwarder_rearch_1/funcx_sdk/funcx/tests

Part of the test campaign: #318

Linux is the standard across clusters/grids/HPC. So testing across flavors or Linux as well on machines with different default shells (bash/csh/zsh) would also be useful.

ZhuozhaoLi commented 3 years ago

First issue for this with Parsl 1.0.0: the endpoint is scaling indefinitely because we need to make some fixes to the funcx strategy.

ZhuozhaoLi commented 3 years ago

it scales to 

2021-02-06 00:05:16.736 interchange.strategy.simple:48 [INFO]  Task breakdown [('interchange', 0, True), ('2745d327b194', 0, True), ('e477166fc686', 0, True), ('e3789484fb3f', 0, True), ('307d2a73d4e1', 0, True), ('36ebfae4626d', 0, True), ('01e5c4cb89b3', 0, True), ('1749f251c0ff', 0, True), ('d96411b12659', 0, True), ('a2587953b0ff', 0, True), ('5a76406baa32', 0, True), ('1f6eb7e32346', 0, True)]
2021-02-06 00:05:17.737 interchange.strategy.simple:48 [INFO]  Task breakdown [('interchange', 0, True), ('2745d327b194', 0, True), ('e477166fc686', 0, True), ('e3789484fb3f', 0, True), ('307d2a73d4e1', 0, True), ('36ebfae4626d', 0, True), ('01e5c4cb89b3', 0, True), ('1749f251c0ff', 0, True), ('d96411b12659', 0, True), ('a2587953b0ff', 0, True), ('5a76406baa32', 0, True), ('1f6eb7e32346', 0, True)]
2021-02-06 00:05:18.737 interchange.strategy.simple:48 [INFO]  Task breakdown [('interchange', 0, True), ('2745d327b194', 0, True), ('e477166fc686', 0, True), ('e3789484fb3f', 0, True), ('307d2a73d4e1', 0, True), ('36ebfae4626d', 0, True), ('01e5c4cb89b3', 0, True), ('1749f251c0ff', 0, True), ('d96411b12659', 0, True), ('a2587953b0ff', 0, True), ('5a76406baa32', 0, True), ('1f6eb7e32346', 0, True)]

when the config is 

config = Config(
    executors=[HighThroughputExecutor(
        provider=LocalProvider(
            init_blocks=1,
            min_blocks=0,
            max_blocks=1,
        ),
    )],
    funcx_service_address='http://k8s-dev.funcx.org/api/v1'
)
ZhuozhaoLi commented 3 years ago

it seemsfuncx-endpoint stop does not work properly

yongyanrao commented 3 years ago

Agree with Zhuozhao. We might need to entirely delete the endpoint and re-configure once to make it work for the second time.

yadudoc commented 3 years ago

@ZhuozhaoLi This is going to be an issue until we implement a solution that has the web-service push a command to the endpoint to terminate. Please make an issue. At this point, we can document the steps to terminate an EP.