fix: Bigquery dataset create table disposition

[danbaron63]

What this PR does / why we need it:

• Updates get_historical_features to not create a BigQuery dataset when create_table_disposition is set to CREATE_NEVER.
• Cleans up or ignores some linting issues preventing pushing due to pre-commit checks.

Which issue(s) this PR fixes:

Fixes #4648

Misc

This PR brings BigQuery dataset creation in line with table creation - this will hopefully be useful to orgs (such as mine!) that have stricter requirements over what can/cannot make infrastructure/warehouse changes in production environments. In our case get_historical_features is expected to be read only operation.

[tokoko]

@danbaron63 I get the issue, but how would this change help? with or without this additional config, get_historical_features will still fail unless you prepopulate the dataset, right?

[danbaron63]

@danbaron63 I get the issue, but how would this change help? with or without this additional config, get_historical_features will still fail unless you prepopulate the dataset, right?

Hey @tokoko, this addresses the issue of creating cloud resources during what should be a read only operation when create_table_disposition is appropriately configured, effectively widening the safety net offered by create_table_disposition. This would be useful when a user or service is misconfigured with the wrong dataset and also happens to be over-permissioned - instead of just creating the dataset it can fail gracefully.

In our org this is a serious issue (and is actually the reason one of my colleagues authored create_table_disposition 😄) as our dataset resources are strictly managed by IaC and are not expected to be created/amended by Feast.

[tokoko]

got it, thanks. over-permissioned is a great word 😆