search

awslabs / mlmax

Example templates for the delivery of custom ML solutions to production so you can get started quickly without having to make too many design choices.
https://mlmax.readthedocs.io/en/latest/
Apache License 2.0
66 stars 19 forks source link

Self-contained screening of train and batch-transform permissions #17

Closed verdimrc closed 3 years ago

verdimrc commented 3 years ago

Issue #, if available: N/A

Description of changes: Self-contained notebook to validate that notebook's role has the necessary permissions to run processing, train, create model, and batch transform.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

josiahdavis commented 3 years ago

Love this idea @verdimrc.

Two questions:

verdimrc commented 3 years ago

What are your thougths on including the smallmatter package? I'm just wondering if it is something that some folks might not have access to and how much we could make this fully self-contained.

That's a valid concern.

How about we document it clearly about the dependencies, and remind users with restricted access to bring in this smallmatter repo with the exact same way they did to mlmax repo? As for the rest users, they can still follow the same steps, or just install directly from github.

Other alternatives I can think off:

  1. git submodule => to signal to users with restricted access that they need to bring in an additional repo, but bear in mind we'll be asking users to use different mechanics: one to bring in mlmax (however they want to do it), and then git submodule steps for the dependencies.

  2. snapshot just the relevan portion of smallmatter (for now only two functions are used). In the long term, there may be risk of code divergence with the upstream.

Do you think we could put all of this great content in its own subfolder (in notebooks) e.g., notebooks/screening

+1 on this.

josiahdavis commented 3 years ago

How about we document it clearly about the dependencies, and remind users with restricted access to bring in this smallmatter repo with the exact same way they did to mlmax repo?

Sounds good! Thanks @verdimrc.

yapweiyih commented 3 years ago

Suggest to have a readme to describe the objective of this folder to encourage people to look into it.

yapweiyih commented 3 years ago

Tested to work with private VPC configuration.