search

trendmicro / cloudone-filestorage-plugins

Trend Micro Cloud One File Storage Security plugins reference code.
https://cloudone.trendmicro.com
Apache License 2.0
42 stars 51 forks source link

feat(deployment): FSS Deployment tool for Azure #89

Open GeorgeDavis-TM opened 2 years ago

GeorgeDavis-TM commented 2 years ago

FSS Deployment tool for Azure

Change Summary

PR Checklist

Other Notes

:warning: Recommend to merge PR #87 #88 before merging this PR.

GeorgeDavis-TM commented 2 years ago

@jack-c-tang Thank you for your review. Please see my comments inline.

Hi @GeorgeDavis-TM, Thank you for this PR. This can help customers to protect lots of storage accounts at a time. I have few questions about the solution:

  • How long does the function take to deploy stacks for, like 10, storage accounts? Consumption function app has 10 minutes as maximal timeout, and deploying one set of stacks would take that long time. Can the deployment run in parallel? Or can we use ARM to deploy that many stacks?

This tool was designed to run in a terminal or Azure Cloud Shell, with versions for Azure functions planned for future releases when we have achieved parallelity.

Please consider this tool, an MVP attempt at implementing deployment automation for FSS Stacks in Azure. I am looking for validation of this approach. Once validated from the product team, we can design for parallel runs.

  • One scanner stack for serving 50 storage stacks looks a bit aggressive to me. And some storage accounts that have higher uploading frequency would need dedicated scanner for each.

The value is configurable in the config.json file, under cloudone.max_storage_stack_per_scanner_stack. The default is set to 50 as a maximum guideline. We can add a warning at the start of execution on rightsizing for the workload.

  • How to monitor the failure and recover the process from failure when deploying multiple stacks?

At the moment, failure of a deployment, throws an exception and exits the script. Complex handling logic can be introduced in later stages, where we are deploying multiple stacks in parallel.

  • The way to configure credentials are not documented. Managed identity with a proper role would be another choice for you to refer to. By using it you won't need to configure credentials.

At the moment, I am using an Azure registered app and secret from the same, similar to tm-fss-management.

trend-jack-c-tang commented 2 years ago

Hi @GeorgeDavis-TM , Sorry that I've been busy and cannot fully review this huge and rich-functional PR. In prod team Q3 OKR we have this item in the secondary priority, and this PR could be a start point for building this. Thank you for submitting this PR.

Due to resources and priority, the prod team might not be able to fully validate this solution in a short time. I tried to understand how the plugin works, but some points are still unclear to me. The deploy.sh script looks like just to deploy a function app and doesn't trigger it, so I don't get how to trigger the function. And there's no serverless template in the PR, why is there serverless deploy -s dev command in the README? It looks like it's the function app who deploys the stacks not the script, so the timeout hard limit needs to be considered. Maybe you can use premium tier which has unlimited timeout.

Do you think a short demo video clip can help us understand it quickly? If you have run some tests, like the number of storage accounts you've protected by this plugin, that would also be helpful. Thanks.

GeorgeDavis-TM commented 2 years ago

Hi @GeorgeDavis-TM , Sorry that I've been busy and cannot fully review this huge and rich-functional PR. In prod team Q3 OKR we have this item in the secondary priority, and this PR could be a start point for building this. Thank you for submitting this PR.

Due to resources and priority, the prod team might not be able to fully validate this solution in a short time. I tried to understand how the plugin works, but some points are still unclear to me. The deploy.sh script looks like just to deploy a function app and doesn't trigger it, so I don't get how to trigger the function. And there's no serverless template in the PR, why is there serverless deploy -s dev command in the README? It looks like it's the function app who deploys the stacks not the script, so the timeout hard limit needs to be considered. Maybe you can use premium tier which has unlimited timeout.

Do you think a short demo video clip can help us understand it quickly? If you have run some tests, like the number of storage accounts you've protected by this plugin, that would also be helpful. Thanks.

I will work on updating the README and get back to you for review. Thank you for your feedback.