Implement City-Based Shipping Conditions via Database Seeding

[John-zaviago]

Issue: Implement City-Based Shipping Conditions via Database Seeding

Description: In a recent discussion with Umer, we explored the option of implementing city-based shipping conditions in the @Muzammal Rasool Frappe app. The consensus favors the practical choice of seeding city databases into all user site databases rather than integrating with an external API. This approach is driven by concerns around potential latency and scalability issues if extensive scaling occurs.

Now, it's important to clarify that the city databases will become part of all user site databases. This aligns with our goal of efficiency and minimal latency. The suggestion is to integrate this worldwide data into our Frappe version, not specifically in Muzzalmmal Shipping app.

The rationale behind this decision is rooted in foresight, anticipating future applications or doctypes that may require the same city data. This approach aligns with efficiency, avoiding redundant efforts and centralizing the data for broader use.

Implementation Steps:

1. Modify the Frappe version to include a city database structure.
2. Develop a seeding mechanism to populate city data into user site databases during setup.
3. Ensure proper indexing and optimization for efficient city-based queries.
4. Update the Muzammal Shipping app to utilize the city data from the Frappe version.

Expected Outcome: Users of the Muzammal Shipping app will benefit from city-based shipping conditions without relying on external APIs, ensuring minimal latency and scalable performance. The centralized city data in the Frappe version sets the foundation for potential future applications or doctypes requiring the same data.

Additional Information: This enhancement aligns with our efficiency and foresight-driven development approach, providing a robust solution for city-based shipping conditions in the Frappe ecosystem.

Labels:

• enhancement
• database-seeding
• scalability
• efficiency

[John-zaviago]

Seem to be done

[chomchuen]

City in Thailand have only 48 cities. Must have 929 cities. @Muzmal

[John-zaviago]

@Muzmal @chomchuen @deviljutt

Update: Optimizing Data Architecture for City-Based Shipping Conditions

Following our recent discussion regarding the implementation of city-based shipping conditions in the @muzammal Rasool Frappe app, it has become apparent that the current data architecture for seeding city databases needs optimization. The consensus is to restructure the data into a hierarchical format, organized by country, state, city, and sub-districts.

Implementation Steps:

1. Review the current data architecture and identify areas for improvement.
2. Restructure the data into a tree-level hierarchy, with countries as master categories, states as sub-categories, cities as sub-sub-categories, and sub-districts as subsequent levels.**
3. Ensure proper indexing and optimization for efficient querying and retrieval of city-based shipping conditions. when we gonna use API call from storefront

Expected Outcome: The optimized data architecture will provide a more organized and scalable framework for managing city-based shipping conditions. Users will benefit from improved data organization and streamlined access to shipping-related information.

Additional Information: Optimizing the data architecture aligns with our goal of efficiency and scalability, ensuring that the @muzammal Rasool Frappe app can effectively handle extensive scaling and future growth.

[Muzmal]

Missing cities are added now. @John-zaviago you can consult with @umer2001 about structure. I had a meeting with him and he recommended current structure with a little modifications. You can close this issue

[chomchuen]

• [x] 1. Get all state and city list.
• [x] 2. Confirm structure of the data.
• [x] 3.1 Remove duplicated state
• [x] 3.2 Confirm which pattern is correct and remove the duplicated one.

---

1. Completeness

Current State: 5,152 | Target: 5,084

Current Cites: 125,967 | Target: 150,573

2. Structure

State: No link with higher level.

City: Linked with State and Country.

3. State Doctype 3.1 State was duplicated 3.2 State name and ID was switched

(OLD-left | NEW-right)

[chomchuen]

Have problem with data pattern on city doctype.

---

I already confirmed this solution with @Muzmal .

1. Change current 'State name' field to 'State ID'
2. Add 'State name' field to store 'state name'.

City Doctype will look like this⤵️

City doctype City ID: Chak Jhumra - PB - PK City Name: Chak Jhumra State ID: Punjab - PB - PK State Name: Punjab Country: Pakistan Country Code: PK

State ID field will use for link City doctype with State doctype. State Name will use for front-end for other purpose.

[umer2001]

To expedite the incorporation of comprehensive geographical data, I have addressed the initial setup of countries, and now the focus shifts to seeding States and cities. The proposed structure is outlined below:

State:

name: {State-name}-{Country-Code}
state_name: string
state_code: nullable string
country: link

City:

name: {City-name}-{Country-Code}
city_name: string
country: link
state: link

The data seeding process will be facilitated through the utilization of the py-countries-states-cities-database pip package. This package, maintained by me, undergoes a weekly auto-sync with its upstream repository at dr5hn/countries-states-cities-database. This ensures that our system is consistently updated with the latest information on cities, countries, and states.

For implementation assistance, a reference point is provided in the Frappe framework, specifically in the country.py file. The bulk_insert function at line 32 showcases an effective pattern for handling similar operations. @Muzmal Feel free to leverage this as a guide while integrating the seeding process for states and cities.

Let me know if you have any questions or need further clarification!