AWS Lambda operator does not honor Docker template

[giacomov]

Describe the bug Building bentos with the AWS Lambda operator with custom Dockerfile templates does not work. It seems that the AWS lambda operator uses its own Dockerfile template without regards on whether the model to build has its own Dockerfile template.

To Reproduce Steps to reproduce the issue:

1. Create a bentoml with a custom Dockerfile template, for example:

   
   {% extends bento_base_template %}
   {% block SETUP_BENTO_BASE_IMAGE %}
   {{ super() }}
   ARG GITHUB_TOKEN

{% endblock %} {% block SETUP_BENTO_COMPONENTS %} {{ super() }} {% endblock %}

2. Try to build the model with 

bentoctl build -b [model name] -f deployment_config.yaml --build-arg GITHUB_TOKEN=testing --verbose

3. Look for the Dockerfile produced by the command (under `/tmp/*` as printed by the command). It won't contain the added instructions (in this case, the GITHUB_TOKEN arg)

This happens because of this line for the AWS lambda operator:

~bentoctl/operators/aws-lambda/bentoctl_lambda/create_deployable.py(61)create_deployable() -> options["dockerfile_template"] = TEMPLATE_PATH

where `TEMPLATE_PATH` points to a template within the operator itself (in my case, `PosixPath('~/bentoctl/operators/aws-lambda/bentoctl_lambda/aws_lambda/template.j2')`

**Expected behavior**
<!--- A clear and concise description of what you expected to happen. -->

The model is built honoring the custom Dockerfile template included in the bento (or the dockerfile included in the bento under `env/docker/Dockerfile`).

<!--- 
If applicable, add screenshots, logs or error outputs to help explain your problem.

To give us more information for diagnosing the issue, make sure to enable debug logging for both bentoctl and BentoML:

Enable via environment variable, e.g.:

$ git clone git@github.com:bentoml/BentoML.git && cd bentoml $ BENTOCTL_DEBUG=True BENTOML_DEBUG=TRUE python guides/quick-start/main.py

Or set debug logging in your Python code:
```python
from bentoml.configuration import set_debug_mode
set_debug_mode(True)

For bentoctl CLI commands, simply add the --verbose flag, e.g.:

bentoctl init --verbose

-->

Environment:

• OS: Ubuntu 20.04
• Python Version 3.8.12
• BentoML Version 1.0.10
• Bentoctl Version 0.3.4 (same problem also installing the HEAD of the repo)

[jjmachan]

Yes, this is a limitation with the lambda operator since we are using the custom template for modifying the bento to make it ready for deployment to AWS Lambda. I would love to figure out a workaround for you, what would be the best way to setup some time for that?

[giacomov]

I found a workaround already :-) I modified the template within the operator itself. Not very sustainable, but works for now.

I think a rather simple fix for this would be to merge the two templates (the one from the operator and the one from the bento) before generating the Dockerfile.

[jjmachan]

Glad to hear that you fixed it! Yes, what you suggested should work in most cases—moving this issue to the lambda repo so we can take it up