Feat: transitFeedSyncProcessing implementation

[AlfredNwolisa]

Summary:

The pull request addresses feed sync processing, ensuring proper handling and consistency of feed data using Pub/Sub messages. It includes necessary configuration files, comprehensive tests, and documentation.

Key implementations include:

• Created FeedProcessor class with comprehensive database interaction capabilities
• Implemented idempotent feed processing logic that handles both new and existing feeds
• Added support for feed URL change detection and deprecation workflow
• Integrated with Google Cloud Pub/Sub for dataset batch processing
• Implemented stable ID generation for feed tracking across updates
• Added comprehensive logging and error handling at all processing stages
• Implemented database transaction management with rollback support

Added support for:

• Authentication type handling for feeds
• External ID mapping and management
• Feed redirection tracking
• URL duplication checking
• Feed status management (active/deprecated)

This pull request addresses the functionality described in issue https://github.com/MobilityData/product-tasks/issues/102, which is part of the https://github.com/MobilityData/product-tasks/issues/95 epic.

Expected behavior:

Feed Processing Flow:

1. The Cloud Function receives a Pub/Sub event containing feed information:

• Decodes base64 encoded message
• Validates and parses the payload into a FeedPayload object

2. For each feed processing request:

• Checks if feed exists using external ID and source
• Validates feed URL for duplicates across the system

3. New Feed Processing:

• If feed doesn't exist:

• Generates new UUID and stable ID

• Creates feed record with active status

• Creates external ID mapping

• Publishes to dataset batch topic if not authenticated

4. Feed Update Processing:

• If feed exists with different URL:

• Creates new feed record with updated URL

• Deprecates old feed record

• Updates external ID mapping

• Creates redirect mapping between old and new feed IDs

• Publishes update to dataset batch topic if not authenticated

5. Database Transaction Handling:

• Commits successful operations
• Rolls back on any errors
• Maintains data consistency across all operations

7. Error Handling:

• Provides detailed logging at each step

Testing tips:

Provide tips, procedures and sample files on how to test the feature. Testers are invited to follow the tips AND to try anything they deem relevant outside the bounds of the testing tips.

Please make sure these boxes are checked before submitting your pull request - thanks!

• [x] Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
• [x] Add or update any needed documentation to the repo
• [x] Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
• [x] Linked all relevant issues
• [ ] Include screenshot(s) showing how this pull request works and fixes the issue(s)

[cka-y]

New feeds and feed updates should have operational_status="wip" so they can be manually validated before becoming public.