"parking-sensors-synapse": bug fixes and improvements

[promisinganuj]

Type of PR

• Documentation changes
• Code changes
• Test changes
• CI-CD changes

Purpose

This PR is primarily intended to fix some of the issues related to the deployment of "parking-sensors-synapse" on Mac. There are additional improvements and documentation update to provide more clarity around the solution.

Here is a summary of the changes:

• 471

  
  # Removed "multi-line" option to comply with the incoming schema changes.
  e2e_samples/parking_sensors_synapse/synapse/notebook/02_standardize.ipynb [M]

Updated integration test assertion to check for record count of more than 1.

e2e_samples/parking_sensors_synapse/tests/integrationtests/tests/test_synapse_pipeline_ingest_melbparkingdata.py [M]

General clean-up, removed extra markdown cell

e2e_samples/parking_sensors_synapse/synapse/notebook/00_setup.ipynb [M]

Removed the old "raw_data" folder which is no longer used and/or required.

e2e_samples/parking_sensors_synapse/data/raw_data/On-street_Parking_Bay_Sensors/On-street_Parking_Bay_Sensors.csv [D] e2e_samples/parking_sensors_synapse/data/raw_data/On-street_Parking_Bay_Sensors/On-street_Parking_Bay_Sensors.json [D] e2e_samples/parking_sensors_synapse/data/raw_data/On-street_Parking_Bay_Sensors/On-street_Parking_Bay_Sensors_baylist.csv [D] e2e_samples/parking_sensors_synapse/data/raw_data/README.md [D]

- #472 

Updated "Setup and Deployment" section

e2e_samples/parking_sensors_synapse/README.md [M] e2e_samples/parking_sensors_synapse/docs/images/SynapseGithubIntegration.png [A]

- #470

Removed the "s" from "sleep", increased the sleep time during wheel package deployment, suppressed a warning message and replaced "sed" with "tr" to make it work for Mac. Also rearranged the code a bit to improve readability.

e2e_samples/parking_sensors_synapse/scripts/deploy_synapse_artifacts.sh [M]

Removed the "s" from "sleep"

e2e_samples/parking_sensors_synapse/scripts/common.sh [M]

Added condition for "sed" command for Mac

e2e_samples/parking_sensors_synapse/deploy.sh

Suppressing prerequisite to check for "makepasswd" as it's not available for Mac and not really required in the dev-container and/or if the password is passed as a a parameter.

e2e_samples/parking_sensors_synapse/scripts/verify_prerequisites.sh [M]

Removed the custom query from the log analytics as it's a legacy way and not adding value (it also throws validation warnings)

e2e_samples/parking_sensors_synapse/infrastructure/modules/log_analytics.bicep [M]

- #469 

Publishing right "requirements.txt" file for Synapse configuration

e2e_samples/parking_sensors_synapse/devops/azure-pipelines-ci-artifacts.yml [M]

## Does this introduce a breaking change? If yes, details on what can break
No

## Author pre-publish checklist
<!-- Please check check before publishing PR using "x". Remove a column if it's not applicable. -->
- [ ] Added test to prove my fix is effective or new feature works
- [x] No PII in logs
- [x] Made corresponding changes to the documentation

## Validation steps
<!-- Optional. -->
- Deploy the solution, run the main pipeline and check the record count loaded.

## Issues Closed or Referenced
<!-- This will automatically close the issue when the PR closes. -->
- Closes #469 #472 #471 #470 
<!-- this references the issue but does not close with PR. -->
- References #issue_number

[devlace]

Retested this with a fresh deploy, looking good! Output fact table in SQL Dedicated Pool: