Ensure that exception is not swallowed by finally block

[adityapatwardhan]

PR Summary

Some cases the exception was not caught but swallowed by finally block. Ensure we rethrow the exception after writing error.

PR Context

PR Checklist

• [x] PR has a meaningful title
  • Use the present tense and imperative mood when describing your changes
• [x] Summarized changes
• [x] Make sure all .h, .cpp, .cs, .ps1 and .psm1 files have the correct copyright header
• [ ] This PR is ready to merge and is not Work in Progress.
  • If the PR is work in progress, please add the prefix WIP: or [ WIP ] to the beginning of the title (the WIP bot will keep its status check at Pending while the prefix is present) and remove the prefix when the PR is ready.
• Breaking changes
  • [x] None
  • OR
  • [ ] [Documentation needed]()
    • [ ] Issue filed:
• User-facing changes
  • [x] Not Applicable
  • OR
  • [ ] [Documentation needed]()
    • [ ] Issue filed:
• Testing - New and feature
  • [x] N/A or can only be tested interactively
  • OR
  • [ ] Make sure you've added a new test if existing tests do not effectively test the code changed
• Tooling
  • [x] I have considered the user experience from a tooling perspective and don't believe tooling will be impacted.
  • OR
  • [ ] I have considered the user experience from a tooling perspective and enumerated concerns in the summary.