Closed yyyttthan99 closed 6 months ago
Thanks Vijit for bringing up the Sensitive field here regarding the issue with password field. I tried that one but unfortunately even with Sensitive to true, it doesn't function properly and we will still have the same issue. Existing fields like secret_key in S3 and private_key in GCP metrics are all using plain text, so I think this behavior is acceptable for share_access_policy_key as well.
We had discussions about cutoffTimestamp in slack and weekly sync. To summarize, with current changes, user can only update cutoffTimestamp to a non-zero value with tf, this is a discrepancy in tf that we have to live with. If users do want to update cutoffTimestamp to 0, they can do it via API, which we believe is a very rare case and to achieve "Collection from Now", user can alternatively just give the exact epoch timestamp of Now. And we have confirmed ingestion won't be affected.
Can I get one more approval for this PR? Thanks! cc @samjsong @vsinghal13 @yuting-liu
Any need to modify the polling source's terraform documentation while we change scan_interval to optional?
Thanks Yuting for reviewing. Even though we make the scan_interval optional, I am inclined to not exposing this change to customer. The reason is that this change is only for us to handle scan_interval properly, customers don't need to do any changes for azure sources or their existing sources (s3 etc). Besides, I feel it's good practice for the customer to specify scan_interval explicitly. Wdyt?
Addressed comments in #590. Closing the last pr since the base branch is too old.
I was able to create the source with the dev build and acceptance tests can pass now.