debug/ developer tooling accomodations

[MACKAT05]

Developer tooling. I wrote a simple PowerShell script to run the render command over all the files and redirect them into a subfolder that is listed in the .gitignore Can we add Block quotes to the outputs like so?:

This would allow Vs code users to execute and debug the rendered code interactively as captured from the output with the new snowflake extension.

[MACKAT05]

Optionally this loop over command could be incorporated (and refined to match the deployment gathering Regular expressions) into an expanded render command syntax... as per #156

[sfc-gh-tmathew]

Interesting @MACKAT05. Using the Render to verify the sql that will be executed appears to be useful application of the render functionality. It is as if we need a --dry-run option for the render command without incrementing the version of the SQL files to during the development lifecycle.

How do you think this pattern will fare for CI/CD based deployment workflows in Development environment? Are you envisioning the iterative development is done locally, say on a clone and then pushed for actual deployment in Development and higher environments?

[MACKAT05]

@sfc-gh-tmathew i started working on a fork to deal with this and other issues...

• parallel execution for the jinja interpreter, full debug reporting
• new macro from_csv --> create a jinja python table to iterate over for use in higher level pipeline macro development.
• state detection of deployment --> eg what is deployed where among multiple targets
• strict option --> fail deployment if a skipped v script is detected or a regression of an R script is detected.
• rework snowflake deployment to use the underlying file iterator(https://github.com/snowflakedb/snowflake-connector-python/blob/ca87fd8412efebfbdeb34ff9fabe20b929da9f27/src/snowflake/connector/util_text.py#L35) so comments and empty statement can be ignored... refine activity detection so state normalization only occurs when needed. which kicks open the door for more improvements.

Getting to the point of this query while I was working on it I split cli.py into individual class files fore each level of functionality. I needed to do this to help isolate the levels of responsibility.

I recall seeing somewhere that there was a technical reason for keeping it as one file... does that still need to happen? or would a split of files like this ( with some naming oversight be a maintainable option)

[sfc-gh-tmathew]

This appears to be a major refactor. Appears to be a 4.0 release candidate. Will get back to you @MACKAT05 once I have a chance to sync up with @jeremiahhansen.

[zanebclark]

@MACKAT05 , in #275 I've split the cli.py file out into multiple files / directories to make it easier to write unit tests and such. I poked around your fork and couldn't find a branch with the folder structure above. Is that work in a local branch that you haven't committed? Do you think you could build your feature off of #275?

[MACKAT05]

yeah sorry about that it is still on my local... i'll try to get that visible and up later tonight. edit added experimental branch and published the current state https://github.com/MACKAT05/schemachange/tree/experimental/schemachange