Fix/socket io design changes - Githubissues

Zipstack / unstract

No-code LLM Platform to launch APIs and ETL Pipelines to structure unstructured documents

https://unstract.com

GNU Affero General Public License v3.0

354 stars 30 forks source link

Fix/socket io design changes #385

Closed muhammad-ali-e closed 1 month ago

muhammad-ali-e commented 1 month ago

What

Updated SocketIO logging from pubsub to celery queue
Implemented socket rooms
Partial implementation of session authentication in socket connection
Implemented Connection Manager

Why

Addressing drawbacks in the current WebSocket logs:
- No SessionId check, allowing SocketIO connections even from the landing page before login.
- All emitted messages from the server are received by all connected clients, causing data leaks.
- Each message from publisher is received by every backend worker due to pubsub
The Connection Manager enables using the same socket across different services or sessions.

How

Changed log consumer to Celery tasks. -Replaced pubsub with Kombu Publisher.
Implemented SocketIO rooms for each user session to prevent data leaks.
Added Session ID null check

Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)

No, the main changes are in the log publisher. We need to verify that no logs are missing.

Database Migrations
No

Env Config

Backend:
- NEW
- CELERY_BROKER_URL
Worker
- NEW
- CELERY_BROKER_URL
- TO REMOVE
- REDIS_HOST
- REDIS_PORT
- REDIS_PASSWORD
- REDIS_USER

Relevant Docs

-

Related Issues or PRs

-

Dependencies Versions

-

Notes on Testing

-

Screenshots

Screenshot from 2024-06-05 14-19-26

Checklist

I have read and understood the [Contribution Guidelines]().

sonarcloud[bot] commented 1 month ago

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.0% Duplication on New Code

See analysis details on SonarCloud