search

appirio-tech / connect-app

Build your next project on Connect with the power of crowdsourcing
https://connect.topcoder.com
44 stars 139 forks source link

Steps needed to push Dev to Release #1236

Closed vic-tian closed 7 years ago

vic-tian commented 7 years ago

We changed the generic "chatbot" project category to "all-chatbots", and "generic_chatbot" project type to "chatbot"; "watson_chatbot" remains the same.

In order to update the Elastic Search (ES) cache for the changes on the existing chatbot projects, we had to recreate the whole cache. @mtwomey wrote a tool to do so.

For the push to production, we have to:

  1. Deploy the project service;
  2. Update the Database records of all existing chatbot projects to match the new all-chatbots;
  3. Update the ES cache so projects will be listed on main page;
  4. Update CloudFront servers (reload cache).

@vikasrohit - is there a way to partially update existing ES cache to reflect the updates on the old DB records? If so - how do we do it? We have only 2 projects that are watson_chatbot. The DB update is super quick, but the update of ES cache is slow. Please ping @mtwomey if we have a better solution, because he wrote a tool to update the cache.

vikasrohit commented 7 years ago

I got better idea @vic-topcoder @mtwomey. I am never in favour of direct updating database (updating ES is okay as it it not a source of truth and can be updated to facilitate new use cases). And as far as tool for updating index in ES for project is concerned, I think we already have one here https://github.com/topcoder-platform/tc-project-service/tree/dev/migrations.

The new idea to avoid all these database updates is to have a static mapping at front end between actual project category id and its possible aliases. all-software or all-chatbots might make sense for URLs but not for the database model. Database model is not supposed to be SEO friendly, rather it should be query friendly. Further, we might come up with new name in near future because some other stakeholder might not like the all-software terminology. So, we can not do database and ES update every time. I am strictly not in favor of this kind of updates unless they serve a very critical use case or fixes a critical bug. The idea is to add new field aliases in projectWizard/index.js mappings. And when I can update the code to look into this field as well to determine the project category or product from the URL and once a project category is located I would still use the current id of the project to persist it into database. That way we can have more SEO friendly URLs but still have existing database friendly values for project category field.

vikasrohit commented 7 years ago

@vic-topcoder @mtwomey if you still think the database update worth it, I think we can use https://github.com/parthshah/aws-es-cli to update records selectively instead of rebuilding whole index. I have done one member index update in past using this utility.

vikasrohit commented 7 years ago

@vic-topcoder as per slack discussion I am doing changes to allows us achieving SEO friendly URLs without updating database here #1240