search

aws-samples / sagemaker-run-notebook

Tools to run Jupyter notebooks as jobs in Amazon SageMaker - ad hoc, on a schedule, or in response to events
Apache License 2.0
142 stars 70 forks source link

Add Support for Additional Files #5

Open jimmycfa opened 4 years ago

jimmycfa commented 4 years ago

If the Notebook to be run/scheduled requires additional *.py files there is no way to pass those files in. The ask would be to be able to pass in either a notebook to run-notebook {schedule, run} notebook OR a notebook and path run-notebook {schedule,run} [--input_path folder/s3_path] notebook. I think the fix would be relatively easy looking at https://github.com/aws-samples/sagemaker-run-notebook/blob/master/sagemaker_run_notebook/cli.py#L61.

adrianmeltom commented 3 years ago

any update about this? I have the same problem.

jimmycfa commented 3 years ago

any update about this? I have the same problem.

We ended up modifying the lambda and creating the scheduled job to include the root prefix in S3 vs just the notebook in S3 so that everything was included.

See this: https://github.com/jimmycfa/sagemaker-run-notebook/commit/fdd697abdbaa03f18aeba61876c8c4ea924b5dc7

adrianmeltom commented 3 years ago

any update about this? I have the same problem.

We ended up modifying the lambda and creating the scheduled job to include the root prefix in S3 vs just the notebook in S3 so that everything was included.

See this: jimmycfa@fdd697a

And whats the modification on the lambda? I can see the changes in the run_notebook.py but in the fork you are refering to download a cloudformation template from the original repo without the lambda modified.

JagritiJ commented 2 years ago

I have similar question, unable to import a config.py file which is needed in the code (inside the jupyter notebook), I am using 0.19.0 version of the script because 0.20 failed, because of the compatibility with Jupyter version <3.8.

Could you kindly let me know what can I manually add in lambda function for this ?

fjpa121197 commented 2 years ago

Hi @jimmycfa,

Can you please share your workaround for this? I'm having a similar situation where I have an external .py file that I need to use within my notebook that I want to run.

Were you able to find the solution @JagritiJ @adrianmeltom ?

adrianmeltom commented 2 years ago

I finally used papermill for that.

lasdem commented 2 years ago

@adrianmeltom could you share how you did it? I am still trying to figure out how to solve it.