fyoorer / ShadowClone

Unleash the power of cloud
Apache License 2.0
672 stars 86 forks source link

Error while running the task #31

Closed sumgr0 closed 1 year ago

sumgr0 commented 1 year ago

Screenshot 2023-05-29 at 12 56 41 PM

Python 3.9.16 The test input file contained only 100000 lines.

lc commented 1 year ago

Change the runtime_timeout: in ~/.lithops/config

I believe lambdas can run for up to 15 minutes.

I have mine set as following:

runtime_memory: 512
runtime_timeout: 500
sumgr0 commented 1 year ago

I tried doing the change in the ~/.lithops/config

Then ran: lithops runtime deploy shadowclone --memory 512 --timeout 500

but still getting this message...

image

What am I missing?

fyoorer commented 1 year ago

can you post the logs from /tmp/lithops-root/logs/620191-8-M000.log

sumgr0 commented 1 year ago

Here is the snippet as it contains the data, which I'm not sure if it should be shared here:

Activation: 'shadowclone' (49f5d5bc-e837-4689-b3f3-caf731ff0dd9) [ 2023-05-30 13:21:59,354 [INFO] handler.py:152 -- Lithops v2.9.0 - Starting AWS Lambda execution 2023-05-30 13:21:59,354 [INFO] handler.py:153 -- Execution ID: 620191-0-M000/00068 2023-05-30 13:21:59,373 [INFO] aws_s3.py:67 -- S3 client created - Region: us-east-1 2023-05-30 13:21:59,471 [INFO] jobrunner.py:134 -- Getting dataset from aws_s3://shasumgr0-storage/b1598ac2-3801-433e-b027-ae037cace627 2023-05-30 13:21:59,595 [INFO] jobrunner.py:180 -- Chunk: 1/1 - Size: 43818 - Range: 0-43817 2023-05-30 13:21:59,595 [INFO] jobrunner.py:233 -- Going to execute 'execute_command()' ---------------------- FUNCTION LOG ---------------------- <--- targets in the list ---->

2023-05-30 13:27:37,085 [INFO] jobrunner.py:239 -- Success function execution
2023-05-30 13:27:37,085 [INFO] jobrunner.py:306 -- Storing function result - Size: 33.4KiB
2023-05-30 13:27:37,208 [INFO] jobrunner.py:311 -- Process finished

]

Vanderlansantos commented 1 year ago

I had this same problem today, I increased the --timeout 500 and redeployed. In this case I did it twice with timeout 350 and timeout 500 to work.

sumgr0 commented 1 year ago

I was finally able to run the tasks using the timeout to 500.

Thanks for your help @Vanderlansantos @lc