Open waytrue17 opened 2 years ago
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
have you tested below inference use cases with the DLC container
1) customer provides an inference script with context
(new way) ?
2) customer provides an inference script without the context
(old way) ?
could you please attach the test details on the description ?
I think this change will also be required for the MXNET DLC containers with MMS, instead of adding a new transformer and adapting the handler service in the pytorch toolkit, could we consider adapting the transformer and handler service in the inference toolkit to work with the context
so that this change could be applied to both ? It would also help make it less error prone in the future since the change would be at a single place.
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
I think this change will also be required for the MXNET DLC containers with MMS, instead of adding a new transformer and adapting the handler service in the pytorch toolkit, could we consider adapting the transformer and handler service in the inference toolkit to work with the
context
so that this change could be applied to both ? It would also help make it less error prone in the future since the change would be at a single place.
Makes sense. I will split the code and re-run some tests. Will post the test results afterward
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Powered by github-codebuild-logs, available on the AWS Serverless Application Repository
Description of changes: Enabling multi gpu support. It passes context information to hander functions so that model/data can be assigned to multiple gpu devices.
input_fn(input_data, content_type)
toinput_fn(input_data, content_type, context)
.input_fn(input_data, content_type)
will still work.By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.