Update the Sample Data With More Meaningful Entries

[palisadoes]

We need to have more realistic data in the Talawa-API sample data importation process. The data also needs to be more secular so that the application will more likely appeal to a wider range of Admins testing Talawa for their communities.

The organization names also need to reflect the most common use case of related organizations being managed by Talawa. They also need to be more global to have broader appeal

Therefore the following changes are needed.

Possible Approach

You will need to investigate whether this approach is viable and suggest modifications where applicable.

1. Load the current sample data into Talawa Admin
2. Use Talawa Admin to make the modifications below
3. Save the resulting database as the new sample data that setup.ts uses

New Organization Names

Rename the organizations in the sample data to the following, with updated geographic location data. These cities are well known and geographically close together. This would be a typical scenario for a community based organization

1. Unity Foundation – Bronx
2. Unity Foundation – Queens
3. Unity Foundation – Staten Island
4. Unity Foundation – Brooklyn

Users

We will need more users who can be used in later tasks

1. Add 8 new users
2. They must be registered members for all 4 organizations

Events

We need recurring events created so that the calendar is populated no matter when the data is loaded.

Weekly Recurring Events – Non Registration

Add the following public events to each organization. Registration is not required:

1. Monday:
   1. "Aerobics for Everyone": 6:00am – 7:00am, Location: Studio A
   2. "Early Morning Pilates": 6:00am – 7:00am, Location: Studio B
   3. "Yoga Class for Seniors": 10:00am – 11:00am, Location: Studio B
   4. "Lap Swimming": 6:00am – 3:00pm, Location: Pool
   5. "Book of the Week Club": 2:00pm – 3:00pm, Location: Atlantic Conference Room
2. Tuesday:
   1. "Aerobics for Everyone": 6:00am – 7:00am, Location: Studio A
   2. "Early Morning Pilates": 6:00am – 7:00am, Location: Studio B
   3. "Travel Club": 6:00pm – 7:00pm, Location: Atlantic Conference Room
   4. "Lap Swimming": 6:00am – 3:00pm, Location: Pool
3. Wednesday:
   1. "Aerobics for Everyone": 6:00am – 7:00am, Location: Studio A
   2. "Early Morning Pilates": 6:00am – 7:00am, Location: Studio B
   3. "Travel Club": 6:00pm – 7:00pm, Location: Atlantic Conference Room
   4. "Art Class - Advanced": 10:00am – 11:00am, Location:
   5. "Lap Swimming": 6:00am – 3:00pm, Location: Pool
4. Thursday:
   1. "Aerobics for Everyone": 6:00am – 7:00am, Location: Studio A
   2. "Early Morning Pilates": 6:00am – 7:00am, Location: Studio B
   3. "Kids Karate": 6:00pm – 7:00pm, Location: Studio B
   4. "Art Class - Beginners": 3:00pm – 4:00pm, Location: Studio A
   5. "Lap Swimming": 6:00am – 3:00pm, Location: Pool
5. Friday:
   1. "Aerobics for Everyone": 6:00am – 7:00am, Location: Studio A
   2. "Early Morning Pilates": 6:00am – 7:00am, Location: Studio B
   3. "Neighborhood Watch Meeting": 6:00pm – 7:00pm, Location: Atlantic Conference Room
   4. "Lap Swimming": 6:00am – 3:00pm, Location: Pool
6. Saturday:
   1. "So Bad They’re Good Movie Night": 7:00pm – 9:00pm, Location: Main Theatre
7. Sunday:
   1. "Travel Club": 6:00pm – 7:00pm, Location: Atlantic Conference Room

Weekly Recurring Events – With Registration

Add the following public events to each organization. Registration is required. Make at least 4 of the newly added users be pre-registered in each of the events with no timing conflicts.

1. Monday:
   1. "Youth Swimming Lessons": 3:00pm – 5:00pm, Location: Pool
   2. "Finance Committee Meeting": 5:00pm – 6:30pm, Location: Pacific Conference Room
   3. "Risk Committee Meeting": 6:00pm – 7:30pm, Location: Arctic Conference Room
2. Tuesday:
   1. "Adult Swimming Lessons": 3:00pm – 5:00pm, Location: Pool
   2. "Conference Planning Committee": 5:00pm – 6:30pm, Location: Pacific Conference Room
   3. "Organization Strategy Workshop": 6:00pm – 7:30pm, Location: Arctic Conference Room
3. Wednesday:
   1. "Youth Swimming Lessons": 3:00pm – 5:00pm, Location: Pool
   2. "Families With Addiction": 5:00pm – 6:30pm, Location: Pacific Conference Room
   3. "Investment Strategy Committee": 6:00pm – 7:30pm, Location: Arctic Conference Room
4. Thursday:
   1. "Adult Swimming Lessons": 3:00pm – 5:00pm, Location: Pool
   2. "Programming for Everyone": 3:00pm – 5:00pm, Location: Computer Lab
   3. "University Preparation Tips": 5:00pm – 8:00pm, Location: Arctic Conference Room
5. Friday:
   1. "Career Counseling": 3:00pm – 5:00pm, Location: Pacific Conference Room
6. Saturday:
   1. "Saturday Serenade Concert": 5:00pm – 7:00pm, Location: Garden Patio
   2. "Youth Soccer Lessons": 9:00am – 10:00am, Location: Sports Field
7. Sunday:
   1. "Travel Club": 6:00pm – 7:00pm, Location:

Posts

Make all 8 new users have posts created at the time of data import for these organizations. The posts must have meaningful titles and text that matches the title (at least 20 words). Half the posts must have images, the rest must be text only. The images must be single color royalty-free and a part of the repo in an area dedicated for the sample data importation.

1. Unity Foundation – Bronx
2. Unity Foundation – Queens

Other

1. All tests must pass
2. No functionality of the app must be affected
3. The suggested entries above may have errors. Wherever possible there must not be scheduling conflicts for:
   1. Locations
   2. Users
   3. The sample data must be importable by the setup script without error and fully functional no matter how many times the setup script is run.

[DecodeAndCode]

I have already started working on this issue, please assign this it to me.

[indiansamaltman]

is this issue still open can i work on it ?

[DecodeAndCode]

@palisadoes There is no option in admin-panel for making the event recurring like weekly, monthly etc. and even after I have modified "recurrance": "WEEKLY" its treating as "recurrance": "ONCE" why so??

[palisadoes]

1. You may have found a bug.
2. It could be related to Talawa-Admin or the API.
3. Investigate whether it's the API, Admin or both and report your findings here

If it is with the API, we'll need to create a very detailed issue

It should have been working before. It's hard to believe that this is the first time anyone has noticed this.

[DecodeAndCode]

Sure , I am working on it

[aabbi15]

Does this issue still need fixing, if yes can i work on it?

[Cioppolo14]

@aabbi15 If the issue has already been assigned, please don't ask to be assigned. We want everyone to get a chance.

[Veer0x1]

@DecodeAndCode are you still working on this issue?

[DecodeAndCode]

@Veer0x1 actually I have raised a PR but that PR depends on other functionality so my PR will be merged after that functionality is included in codebase.

[Farhanpathan0007]

Is this issue still unresolved, and if so, am I able to take on the task?

[github-actions[bot]]

This issue did not get any activity in the past 10 days and will be closed in 180 days if no update occurs. Please check if the develop branch has fixed it and report again or close the issue.

[Cioppolo14]

@Farhanpathan0007 Please do not ask to be assigned to an issue that is already assigned. We want everyone to get a chance.

[palisadoes]

@DecodeAndCode Are you still working on this?

[DecodeAndCode]

Yes I'm actually i got into some db error just resolving them.

[github-actions[bot]]

This issue did not get any activity in the past 10 days and will be closed in 180 days if no update occurs. Please check if the develop branch has fixed it and report again or close the issue.

[Cioppolo14]

@DecodeAndCode Are you still working on this?

[DecodeAndCode]

yes @Cioppolo14

[github-actions[bot]]

This issue did not get any activity in the past 10 days and will be closed in 180 days if no update occurs. Please check if the develop branch has fixed it and report again or close the issue.

[Technmad]

is the issue fixed?

[Sahi1l-Kumar]

Please assign this issue to me

[Sahi1l-Kumar]

@palisadoes making the events recurring is increasing the event documents count by a lot [165 --> 2312]. Is this ok?

[palisadoes]

1. Please verify with @meetulr to make sure there isn't an error. He implemented recurring events and we need to make sure that it's working OK.
2. The feature generates a single recurring entry (or very few initial entries), and will auto-generate events as you scroll from day to day, week to week, month to month or year to year. If you did that to test, then that may be the cause. Reset the DB, create the events via the monthly view without scrolling to the following month. See whether the count declines.

[Sahi1l-Kumar]

The issue might be with the startDate of events. Currently the events are starting in the month of January. Due to this all the following months are filled with the same events. Or the 2nd point you mentioned. I will try to see what is causing it.

[Sahi1l-Kumar]

I have thought of some solutions.

1. When importing sample data, we can change the month of the events in sample data to the current month of the user. This ensures that users, regardless of when they import the data, will see events displayed in their local time zone.
2. Changing the events to Monthly Recurring Events. OR
3. Adjusting the endDate to an earlier month.

What are your thoughts? @palisadoes

[palisadoes]

1. By creating a recurring event now, you will automatically create events from this day forward on demand. This is described detailed below. You don't have to change the month, it is unnecessary.
   1. https://docs.talawa.io/docs/functionalities/recurring-events
2. We need weekly recurring events to that the calendar always appears full.
3. There must be no end date for the recurring events. We need the data to be present at any time in the future that the user decides to import the sample data

[meetulr]

@Sahi1l-Kumar @palisadoes

1) The current dynamic generation process of recurring is such that it will generate recurring event instances till some limit date (based on the recurrence frequency link) in the future. This limit date would be calculated from the current Date (link), and not the calendar date the user selected on the frontend. This assumes normal rational user behavior that they won't move too far from the current date in the calendar. i.e.-

• i.e. If the current date is X, and the limit date calculated is Y, the instances would be generated up till Y. If the user whose current date is X moves to a date > Y in the calendar, they won't see new instances there, as the new instances would be generated when they satisfy the criteria in the query (i.e. if the date of the latest instance generated is less than the queryUptoDate (link)).
• This is because we are currently returning all the events from the query, and if we were to base the new instances generation on the calendar date, it would make a query every time the user switches months, this would slow the app down. Also, the app would be slowed down anyway if there were too many events.
• We would eventually need to add pagination to the query, and then we could base the dynamic generation on the calendar date instead of the current date. i.e. in that scenario, if the user moves to a date > Y in the calendar, new instances would then be generated ahead based on that calendar date, and we would be returning a chunk of events close to the calendar date. 2) Sahil, You just need to create a few weekly recurring events, with the endDate of RecurrenceRule and BaseRecurringEvent as null. Then the query will keep generating new instances, and the calendar will always be populated. (This will work even if you add just one instance in the sample data, the rest would be generated automatically, we just need the endDate to be far away in the future).

[palisadoes]

Therefore for this purpose we should be creating recurring events with a null end date.

Can this be done in the Admin UI to make things simpler?

[meetulr]

Therefore for this purpose we should be creating recurring events with a null end date.

Can this be done in the Admin UI to make things simpler?

I don't understand😅, in the admin?

[Sahi1l-Kumar]

Therefore for this purpose we should be creating recurring events with a null end date.

Can this be done in the Admin UI to make things simpler?

No, the end date cannot be null in the Admin UI.

We need weekly recurring events to that the calendar always appears full.

Is it necessary to display events in the previous months when users open the event page, considering they are already directed to their current day?

Thank you for the explanation @meetulr

[meetulr]

No, the end date cannot be null in the Admin UI.

What do you mean by that? The individual instances would have their endDate same as the startDate, the RecurrenceRule and the BaseRecurringEvent would have the endDate as null.

[meetulr]

Is it necessary to display events in the previous months when users open the event page, considering they are already directed to their current day?

1) The point is we need to always have some events visible on any date. e.g. suppose you added some recurring events in the sample data with endDate as the end of this month. Now if someone starts the app next month, they won't see any events. If the endDate of the recurring event was null however, the query would keep generating new instances and the calendar would always show some events.

2) And it would not make sense to not show the previous events. As I said, the query currently just returns all the events, that obviously needs to change as it would just bloat the app with so many events, pagination would be required so that we can query events in chunks based on the calendar date the user is at, then your point of not fetching all the previous events wouldn't be a problem either.

[Sahi1l-Kumar]

And it would not make sense to not show the previous events. As I said, the query currently just returns all the events, that obviously needs to change as it would just bloat the app with so many events, pagination would be required so that we can query events in chunks based on the calendar date the user is at, then your point of not fetching previous events wouldn't be a problem either.

@palisadoes Pagination would be needed to not bloat the app as @meetulr mentioned.

[palisadoes]

1. Please create an issue to implement pagination in the calendar query.
2. You'll also need to create issues in Admin and Mobile to work with this new methodology

What do you both think the default behaviors should be?

1. Fetch X days at time?
2. If you fetch more than 1 week, but less than a year should it just return the 5 weeks of the month calendar?
3. Do you have other suggestions? The typical graphql first, last etc. filters don't cleanly apply

[meetulr]

Yes, the standard gaphql pagination won't apply here, not directly.

I was thinking a simple query that fetches all the events for that year. i.e. based on the calendar date the user is at, we can fix a eventsQueryStartDate & a eventsQueryEndDate, and pass these to the query:

• eventsQueryStartDate: It would be the 1st day of that year.
• eventsQueryEndDate: It would be the last day of that year. and then we would fetch all the events that occur during that year (i.e. events whose startDate lies between these two / i.e. events that occur during that year).

These dates will change only If the user moves to another year through the calendar, then only another query would be made. Basically, fetching the events year by year. This would be simple & efficient. What do you think?

[Sahi1l-Kumar]

Since the default view of the calendar is Month View, showing the events for the current month plus the adjacent weeks from the previous and next months should be the default behavior.

[Sahi1l-Kumar]

fetching the events year by year.

This should be easy to implement but there are a lot of recurring events in the db. I created Weekly Recurring Events – Non Registration as mentioned here and the total count of the events documents is 2630

[meetulr]

fetching the events year by year.

This should be easy to implement but there are a lot of recurring events in the db. I created Weekly Recurring Events – Non Registration as mentioned here and the total count of the documents is 2630

Yes but we'd only fetch for one year. You're saying there are 2630 recurring events during that one year? We probably don't need that many for our sample data😅, just a couple weekly recurring events to show on the calendar would be enough.

[Sahi1l-Kumar]

fetching the events year by year.

This should be easy to implement but there are a lot of recurring events in the db. I created Weekly Recurring Events – Non Registration as mentioned here and the total count of the documents is 2630

Yes but we'd only fetch for one year. You're saying there are 2630 recurring events during that one year? We probably don't need that many for our sample data😅, just a couple weekly recurring events to show on the calendar would be enough.

This is the number I got from setting endDate to null. It's for 2 years. Yes probably should create less events and which are not recurring too much in a single week.

[meetulr]

Yeah so what I'm saying is that the query would fetch the events for that year only. Still, couple hundred won't be a problem (small communities won't have many), the problem is fetching all the events in the database, which would be rectified by fetching in chunks.

Yes probably should create less events and which are not recurring too much in a single week.

Yeah we wouldn't want to bloat the calendar view either right?😅 a few weekly recurring events would be enough @palisadoes ?

[palisadoes]

Since the default view of the calendar is Month View, showing the events for the current month plus the adjacent weeks from the previous and next months should be the default behavior.

1. This should be the default, monthly
2. A year of data only for the annual view.
3. We need a lot of events as stated in the original issue comment so that the calendar is full and better emulates a typical scenario
4. @meetulr Why does null generate 2 years of data and not the most likely default of a month, or week? It's going to auto increment anyway with queries beyond the currently generated number of events.

[meetulr]

As I mentioned, the limit date for instances generation is based on the recurrence frequency: limits, limit date. For weekly it generates up to 2 years ahead, for daily it's 1 year. We can change these limits anytime.

This is because the dynamic generation is currently based on the current date (not the calendar date the user is at), as I mentioned here. A user navigating through months would be pretty common. If we only generated up to couple weeks or couple months ahead (say, 2 months), then the user won't see the events beyond that short limit date, i.e. if they navigated to a date more than 2 months ahead, they won't see the instances, because they wouldn't have been generated yet.

But this would change when we fetch events based on the calendar date the user is at. The method I proposed would fetch the events for the year the calendar date lies in (by default, the current year). The yearly view can just use this data too.

The reason behind setting the year start and year end date as query parameters instead of the calendar date is: 1) If we set the calendar date as the query parameter, then a new query would be made every time the user switches months, i.e. every time the calendar date changes. We don't want to show the loading state that many times. 2) Making the year start and end as query parameters would only make one query per year, and a new query would only be made if the user navigates to next or previous year. And we'd have all the data for the year we're on in the calendar. 3) We could then base the dynamic generation on these dates too, instead of the current local date. i.e. it'll keep generating new instances if the user keeps moving ahead in the calendar. This should be the default behavior. (I had intended to modify the dynamic generation and base it on the query parameters i.e. year start and year end date, after we modified our current query to fetch in chunks)

I'm assuming small communities here, i.e. not incredibly many events.

To make it even more efficient, we could fetch in quarters of a year for the monthly view, but we'd have to make a separate query for fetching the data for the yearly view.

[palisadoes]

1. Thanks for the explanation @meetulr. Let's not change the code logic for the purposes of this issue.
2. @Sahi1l-Kumar please create a PR with the data you generated

We can create new issues to optimize performance if the data loads slowly. There will always be lots of events to view

[Sahi1l-Kumar]

Ok will create it soon. @palisadoes Just wanted to ask if there is a way to export the data in the same format as the sample_data. The file I get from mongoexport is formatted differently from the original sample data.

The current format :-

The format after exporting as JSON :-

[meetulr]

1. Thanks for the explanation @meetulr. Let's not change the code logic for the purposes of this issue. We can create new issues to optimize performance if the data loads slowly. There will always be lots of events to view

Yes, I was saying we will eventually need to modify the query, not for this issue😅

[Sahi1l-Kumar]

There are total 17280 event documents across all 4 organiztations. [4295 each]

[palisadoes]

Add them. We can clean it up later

It's a lot!

[Sahi1l-Kumar]

Ok will create it soon. @palisadoes Just wanted to ask if there is a way to export the data in the format same as the sample_data. The file I get from mongoexport is formatted differently from the sample data.

The current format :-

The format after exporting as JSON :-

Any information on this would be helpful.

[palisadoes]

Ask the contributors who last generated the data files in PRs. They should be able to help.