fatchat commented 10 months ago

Description

This project will build a notification system for Dalgo. Although this ticket is part of the DDP_backend repo, the selected contributor will work on the frontend webapp repo as well, and possibly on the prefect-proxy repo if required

Goals

[ ] An interface for a Dalgo admin to post a notification to a user or to a set of users
[ ] Support for multiple notification channels
[ ] Email should be one channel
[ ] Discord should be one channel
[ ] Users should be able to choose their preferred channel
[ ] Users should be able to turn off notifications to all channels
[ ] Notifications should appear in the React frontend
[ ] Indicator for unread notifications for a user in the frontend
[ ] "Urgent" notifications should appear as a horizontal bar in the frontend e.g. for "System is going down in 10 minutes!" type of messages

Expected Outcome

In the backend:

a Python interface to create new notifications, fetch unread notifications for a user, fetch the notification history and more
an API to create new notifications (this API will be invoked by the dalgo-admin tool)
a pub-sub model in which handlers for the various notification channels can subscribe and receive new notifications to be sent out
handlers for Email and Discord to start off with

In the frontend

Notifications in the toolbar (e.g. a "bell icon") showing number of unread
User should be able to view their unread notifications by clicking on the icon
A notification bar for "urgent" notifications, possibly with a red background or something similarly eye-catching
A UX where a user can set and manage their notification preferences

Acceptance Criteria

[ ] Text-only notifications are acceptable, but extra points for rich text (html, markdown)
[ ] Admin should be able to send notifications to individual users
[ ] Admin should be able to send notifications to a set of users (including all users)
[ ] Admin should be able to send "urgent" notifications
[ ] Users should receive or not receive channel notifications based on their preferences
[ ] Users should see notifications in the frontend as described above
[ ] Notification history should be stored in the database

Implementation Details

Libraries may be used but must be open-source Our backend is Django and our frontend is NextJS

Mockups / Wireframes

None but we will decide on designs before the frontend work begins

Product Name

Dalgo

Project Name

Notification System for Dalgo

Organization Name:

Project Tech4Dev

Domain

Other

Tech Skills Needed:

Python, Django, JavaScript, NextJS

Mentor(s)

@fatchat

Complexity

High

Sub Category

API, Frontend, Backend

Shruti3004 commented 9 months ago

Linking Installation guide here: https://github.com/DalgoT4D/DDP_backend?tab=readme-ov-file#setup-instructions

venkatramanm commented 9 months ago

@SaumyaSamagra , The requirements seem quite clear and it seems to be doable in the timeframe of the internship. What is the pub-sub mechanism to be used, is not mentioned. Some clarity may be added or may need to be figured out earlier on on the project. That may be added to the skill set requirement as a preference.

Snehil-Shah commented 8 months ago

@fatchat is the plan to use something like RabbitMQ/Kafka for the pub-sub model?

fatchat commented 8 months ago

@Snehil-Shah I'm looking for a solution which uses an existing library. Having said that, I'd want to avoid a library which uses Kafka since that would be overkill. Rabbit/Zero/something small

The contributor would need to know how the notifications library is using its messaging components, but only for testing and debugging and not for extending them

amansgith commented 8 months ago

Hii @fatchat can "FastAPI Websocket Pub/Sub" package be used for it.. as its lightweight and maybe still effective for the pub sub model Link

ricky08sirus commented 7 months ago

I want to contribute for this project as a part of DSP 2024. please guide me with the procedure @fatchat .

srihari77 commented 7 months ago

I want to contribute for this project as a part of DSP 2024. please guide me with the procedure @fatchat .

AkanshuAich commented 7 months ago

Hii @fatchat,

I am writing to express my interest in contributing to this project as a part of DSP 2024. Having thoroughly reviewed the project, I am impressed by its objectives and it seeks the potential for great impact in industries.

With my background in Backend using Django , MERN with practicing hands on Machine learning, I believe I can make valuable contributions to both backend and frontend part. My experience includes several projects like Society-Expenditure Manager using Django and Real Estate using MERN, which I believe align well with the goals of your project.

I am particularly interested in fulfilling the requirements of the project and have some ideas on how to approach it effectively. I am committed to adhering to best practices, contributing high-quality code, and actively collaborating with the project maintainers and community.

I am excited about the opportunity to contribute to Notification System for Dalgo and help further its mission. I look forward to discussing potential contributions and how I can best support the project.

Please guide me with procedure and with all your knowledge and experience.

fatchat commented 7 months ago

hello all, thank you for your interest

please refer to the c4gt website / discord community on how to apply

and as much as i would love to, i won't be mentoring anyone during the application process... there are just too many people. whoever is selected will get two hours a week from me

thank you again

SaumyaSamagra commented 7 months ago

Hi Snehil,

Please create a duplicate file and upload your proposal document https://docs.google.com/document/d/1D2pLc_eP-8-waBnRuj57Tc_jQT7_-ZWRQCDjv6qtPTY/edit?userstoinvite=mohitdto0311%40gmail.com&sharingaction=manageaccess&role=writer with the specified nomenclature as specified on Unstop https://unstop.com/competitions/dedicated-mentoring-program-dmp-2024-code-for-govtech-932803?lb=7gWXVfKv .

For any questions please feel free to direct your queries at @.*** and join the Discord server https://bit.ly/C4GTJoinDiscordServer

Thank you

Best, Saumya

On Sat, Mar 9, 2024 at 10:54 PM Snehil Shah @.***> wrote:

@fatchat https://github.com/fatchat is the plan to use something like RabbitMQ/Kafka for the pub-sub model?

— Reply to this email directly, view it on GitHub https://github.com/DalgoT4D/DDP_backend/issues/442#issuecomment-1986923163, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7TNBGFBMMONBHM2M6RFR3LYXNATPAVCNFSM6AAAAABCTLBAG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBWHEZDGMJWGM . You are receiving this because you were mentioned.Message ID: @.***>

--

Saumya Gupta

Consultant | +91 9629770622

Samagra | Transforming Governance

Address: 9/5 Sarvapriya Vihar, New Delhi-16 @.***,77.1994501,17z/data=!3m1!4b1!4m5!3m4!1s0x390ce273981e5dfb:0x7bfceb198af17562!8m2!3d28.5566128!4d77.2016388?shorturl=1>

Website: www.samagragovernance.in

Follow Us: Linkedin https://www.linkedin.com/company/samagra-transforming-governance/ | Youtube https://www.youtube.com/channel/UCfkXErS-f87xUQkmSKSC8bg | Instagram https://www.instagram.com/samagragovernance/?hl=en

SaumyaSamagra commented 7 months ago

Hi Aman,

Please create a duplicate file and upload your proposal document https://docs.google.com/document/d/1D2pLc_eP-8-waBnRuj57Tc_jQT7_-ZWRQCDjv6qtPTY/edit?userstoinvite=mohitdto0311%40gmail.com&sharingaction=manageaccess&role=writer with the specified nomenclature as specified on Unstop https://unstop.com/competitions/dedicated-mentoring-program-dmp-2024-code-for-govtech-932803?lb=7gWXVfKv .

For any questions please feel free to direct your queries at @.*** and join the Discord server https://bit.ly/C4GTJoinDiscordServer

Thank you

Best, Saumya

On Sun, Mar 31, 2024 at 1:34 PM Aman Khan @.***> wrote:

Hii @fatchat https://github.com/fatchat can "FastAPI Websocket Pub/Sub" package be used for it.. as its lightweight and maybe still effective for the pub sub model Link https://pypi.org/project/fastapi-websocket-pubsub/

— Reply to this email directly, view it on GitHub https://github.com/DalgoT4D/DDP_backend/issues/442#issuecomment-2028593915, or unsubscribe https://github.com/notifications/unsubscribe-auth/A7TNBGHH3FZF3X42DW5Y2VLY267RJAVCNFSM6AAAAABCTLBAG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRYGU4TGOJRGU . You are receiving this because you were mentioned.Message ID: @.***>

--

Saumya Gupta

Consultant | +91 9629770622

Samagra | Transforming Governance

Address: 9/5 Sarvapriya Vihar, New Delhi-16 @.***,77.1994501,17z/data=!3m1!4b1!4m5!3m4!1s0x390ce273981e5dfb:0x7bfceb198af17562!8m2!3d28.5566128!4d77.2016388?shorturl=1>

Website: www.samagragovernance.in

Follow Us: Linkedin https://www.linkedin.com/company/samagra-transforming-governance/ | Youtube https://www.youtube.com/channel/UCfkXErS-f87xUQkmSKSC8bg | Instagram https://www.instagram.com/samagragovernance/?hl=en

AbhimanyuSamagra commented 7 months ago

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries.

AkanshuAich commented 7 months ago

@fatchat for the notification system using Django framework it will be more efficient to go with the third-party applications like Pusher, Firebase Cloud Messaging (FCM), Amazon Simple Notification Service (SNS), and Twilio. Celery with redis/RabbitMQ is also a good option as the background function and the notification system both can work simultaneously.

RSN601KRI commented 7 months ago

Hello, @fatchat , I would like to contribute to this project. I have the required skillset for it. Kindly guide me so that I can contribute to this project.

amansgith commented 7 months ago

Hi @fatchat can you help me with these things regarding the project of C4GT:

What problem exists in Dalgo or what will be enhanced by this system for which the notification system is to be developed?
Is any previous work related to the notification system already started or done?
As different channels like Email or discord have different methods for API usage and invocation method.. for handling these do we need to build a common platform so that new handles can be added in future?

Thanks

fatchat commented 7 months ago

@amansgith

We currently have notifications only for failures of flow-runs, and these only go to our Discord #ops channel. We would like to be able to send notifications to a user's email address, and let them configure this. More generally we would like to a. notify users when their connection schemas change b. notify users when the platform is going down for upgrades c. any other messages we would like to send our users
No
Yes, with limits. Right now we only want to be able to send text notifications, not html or rich-text or whatever else

AbhimanyuSamagra commented 7 months ago

Do not ask process related questions about how to apply and who to contact in the above ticket. The only questions allowed are about technical aspects of the project itself. If you want help with the process, you can refer instructions listed on Unstop and any further queries can be taken up on our Discord channel titled DMP queries. Here's a Video Tutorial on how to submit a proposal for a project.

SachinSangaal commented 7 months ago

Hello @fatchat I want to contribute for this project of C4GT 2024. please guide me with the procedure.

techiAlok commented 6 months ago

Hello @fatchat i too want to contribute to this project of C4GT 2024. please guide me to move further in the process.

Rishikesh63 commented 6 months ago

@fatchat is DMP selection of DalgoT4D announced?

fatchat commented 6 months ago

Yes, announcements were made on Tuesday I believe

On Thu, May 30, 2024 at 5:10 PM Rishikesh Maddhesiya < @.***> wrote:

@fatchat https://github.com/fatchat is DMP selection of DalgoT4D announced?

— Reply to this email directly, view it on GitHub https://github.com/DalgoT4D/DDP_backend/issues/442#issuecomment-2139369000, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAQPOIAJDSNTL76723GNAELZE4F3NAVCNFSM6AAAAABCTLBAG6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMZZGM3DSMBQGA . You are receiving this because you were mentioned.Message ID: @.***>

huzaifmalik786 commented 5 months ago

Weekly Learnings & Updates

Week 1

Set up the DDP_backend project locally along with prefect-proxy and airbyte server.
Week 2
Make a documentation file. #718
Learn how to make migrations and revert them. python manage.py makemigrations ddpui python manage.py migrate python manage.py migrate ddpui 0077
Week 3
Create notification, recipients, and user preference table models and make migrations.
Create functions for managing user preferences.
- Create APIs for calling functions for managing user preferences
  Issues:
674
675
676
684
683
682
688

PR:
730

Week 4

Create functions for creating notification, deleting notification, fetching notification history, fetching notifications for a user, mark notification as read or unread.
Create APIs for managing notification related functions.
Create email and discord handlers.
Create a celery worker for scheduling notification.
Issues:
689
690
677
678
679
680
681

PR:

744

Week 5

Write pytests for notification APIs.
Write pytests for notification preferences APIs.
Write pytests for notification core functions.

Issues:

746
747
745

PR:
744

Week 6

Create UI for user panel to see notifications.
Integrate user panel with backend APIs.

Week 7

Add an API for fetching unread notification count.
Add tests for this API.
Integrate it with user panel frontend.

Issues

767

PR:
766

Week 8

Add a pop up to update user's notification preferences.
Intergrate user preferences with backend APIs.

Week 9

Planning and discussion for writing scripts to create and delete notifications
Assigned to 3 different tickets other than this ticket.
Issues
https://github.com/DalgoT4D/webapp/issues/1068

Week 9

Make the save button floating when scrolling throughout the table.
Issues
https://github.com/DalgoT4D/webapp/issues/1068
PR:
https://github.com/DalgoT4D/webapp/pull/1119

Week 10

Add column search in all the forms.
Issues
https://github.com/DalgoT4D/webapp/issues/1065
https://github.com/DalgoT4D/webapp/issues/1040
PR:
https://github.com/DalgoT4D/webapp/pull/1119

Week 11

Explored the prefect-proxy repository and planned the changes required for the #789
Made changes in the prefect-proxy repo to add pagination to logs.
PR:
https://github.com/DalgoT4D/prefect-proxy/pull/154

Week 12

Created a new API to fetch graph for a flow run.
Added limit, offset and task_run_id to the logs API to fetch logs for a task run with pagination.
Issues:
789

PR:
832

Week 13

Changes in the webapp application to apply pagination to the logs.
PR
https://github.com/DalgoT4D/webapp/pull/1143

Week 14

Added scripts for creating and deleting notifications.
Tested all the use cases.

PR

847

fatchat commented 5 months ago

Weekly Goals

Week 1

[x] Talked about the project, architecture, approach

Week 2

[x] Set up the codebase locally.
[x] Make a documentation file for the ticket.
[x] Learn how to make migrations and revert them.

Week 3

[x] Create notification, recipients, and user preference table models. (https://github.com/DalgoT4D/DDP_backend/pull/730)
[x] Create functions for managing user preference. (https://github.com/DalgoT4D/DDP_backend/pull/730)

Week 4

[x] Create functions for handling fetch, create, delete, update read_status of notifications and sending notification through email and discord.

Week 5

[x] Manage scheduling of notifications.
[x] Wrote pytests for all backend functions

Week 6

[x] UI to view notifications

Week 7

[x] API to get unread count

Week 8

[x] Notification Preferences panel

Week 9

[x] Worked on https://github.com/DalgoT4D/webapp/pull/1119

Week 10

[x] Completed https://github.com/DalgoT4D/webapp/pull/1119

Week 11

[x] https://github.com/DalgoT4D/prefect-proxy/pull/154

Week 12

[x] https://github.com/DalgoT4D/DDP_backend/pull/832

Week 13

[x] https://github.com/DalgoT4D/webapp/pull/1143

Week 14

[x] Django management command to create a notification

DalgoT4D / DDP_backend

Notification System for Dalgo #442

Description

Goals

Expected Outcome

Acceptance Criteria

Implementation Details

Mockups / Wireframes

Product Name

Project Name

Organization Name:

Domain

Tech Skills Needed:

Mentor(s)

Complexity

Category

Sub Category

Weekly Learnings & Updates

Week 1

Week 2

Week 3

Issues:

674

675

676

684

683

682

688

PR:

730

Week 4

Issues:

689

690

677

678

679

680

681

PR:

744

Week 5

Issues:

746

747

745

PR:

744

Week 6

Week 7

Issues

767

PR:

766

Week 8

Week 9

Issues

Week 9

Issues

PR:

Week 10

Issues

PR:

Week 11

PR:

Week 12

Issues:

789

PR:

832

Week 13

PR

Week 14

PR

847

Weekly Goals

Week 1

Week 2

Week 3