Docker-deploy as the name suggests is a docker-based deployment script that can be used to set up the UCI for usage.
We are using Docker to enable a smooth, single command experience for all the adopters of UCI to get it up and running. Docker is an open-source containerization platform. It enables developers to package applications into containers—standardized executable components combining application source code with the operating system (OS) libraries and dependencies required to run that code in any environment.
Any user/ organization that wants to use setup UCI on their own server.
Install Docker if not already installed. Click Here
Infra:
Basic familiarity required with:
Please make sure all of the ports mentioned used in the file are open & are not being used by any other service on server.
Add the desired version of uci-apis which you want to use to .env
. Run setup-manual.sh
and follow the prompts.
Once all the containers are up, you will notice that there will be an minio-fusionauth setup out of the box. Go to minio console at port 9001, navigate to Identity -> Open ID, click on edit and turn Claim User Info
on.
Some of the components are required to be created manually, they will be automated once once-click-deployment is completed
Hashicorp Vault
has to initiated for the first time and relevant keys should be setup.
A kv v1
engine has to be created with firebase secrets in hashicorp vault.
Org and ownerIDs to should be created
Manual adapters are need to be created - https://uci.sunbird.org/use/developer/uci-basics
Transformer ( broadcast & generic) and ODK has to created
On the first run cassandra migrations are applied. If you see any migration error then set it to 0
to run all migrations if not already applied, else set to 6
if all migrations are applied. In most cases after the first run, setting the migrations count
in .env
to 6
will work. Migrations get applied from 0
to 6
in a sequential manner.
NOTE:
MINIO_IDENTITY_OPENID_CONFIG_URL
configuration setting by replacing fusionauth
with either the server's IP address along with the designated port or the domain name you have directed it to. For running it on your local machine, the provided configurations should suffice.
Take clone of this repository.
git clone https://github.com/samagra-comms/docker-deploy.git
Go to folder
cd docker-deploy
Run below command to download & start the services using docker.
bash installv2.sh
If script is fail on runtime then you should run below commands for continue run this script
Stop the all services
docker-compose down
Start the all services
docker-compose up -d
While the script is running, You will be asked to enter encryption key, contact the administrator and get this.
If you are asked to enter Netcore Whatsapp Auth Token, Source, URI, enter the details if you have any else press enter. Click here to check the whatsapp configuration for netcore.
Note: Please note this installation is just the first step. If your needs are not fulfilled with the current installation, please start scaling the individual services by using them in docker stack.
Script execution time - Around 1 hours (its depends on your server)
On a successful run of the script you will get below services:
Note: If above mentioned url's are not working then restart the all docker services or check the docker container logs.
If you want to check logs for a specific service, follow below flow.
Show all docker containers
docker ps -a
Copy the container id from the list for the service you want to see the logs for, and use it in below command.
docker --follow --tail 10 container_id
Note: If the services are updated on any server, use server's ip instead of localhost. Eg. http://143.112.x.x:9080
Tracking Tables/Views : When the script is executed, tables will be created for bot schema via migrations. These tables can be assessed in Hasura. We need to track these tables to expose these tables to Hasura GraphQL. Click to view tracking overview.
After the tables are created, some default data for adapter, transformers etc should be added. These will be used later while creating a bot. Please follow below step:
INSERT INTO service ("id", "type", "config")
VALUES ('94b7c56a-6537-49e3-88e5-4ea548b2f075', 'odk', '{"cadence": { "retries": 0, "timeout": 60, "concurrent": true, "retries-interval": 10 }, "credentials": { "vault": "samagra", "variable": "samagraMainODK" } }');
INSERT INTO adapter ("id", "provider", "channel", "config", "name")
VALUES ('44a9df72-3d7a-4ece-94c5-98cf26307324', 'gupshup', 'WhatsApp', '{ "2WAY": "2000193033", "phone": "9876543210", "HSM_ID": "2000193031", "credentials": { "vault": "samagra", "variable": "gupshupSamagraProd" } }', 'SamagraProd');
INSERT INTO adapter ("id", "provider", "channel", "config", "name")
VALUES ('44a9df72-3d7a-4ece-94c5-98cf26307323', 'Netcore', 'WhatsApp', '{ "phone": "912249757677", "credentials": { "vault": "samagra", "variable": "netcoreUAT" } }', 'SamagraNetcoreUAT');
INSERT INTO adapter ("id", "provider", "channel", "config", "name")
VALUES ('64036edb-e763-44b1-99b8-37b6c7b292c5', 'gupshup', 'sms', '{"credentials": {"vault": "samagra","variable": "gupshupSamagraSMS"}}', 'SamagraGupshupSms');
INSERT INTO adapter ("id", "provider", "channel", "config", "name")
VALUES ('4e0c568c-7c42-4f88-b1d6-392ad16b8546', 'cdac', 'sms', '{"2WAY":"2000193033","phone":"9876543210","HSM_ID":"2000193031","credentials":{"vault":"samagra","variable":"gupshupSamagraProd"}}', 'SamagraCdacSms');
INSERT INTO adapter ("id", "provider", "channel", "config", "name")
VALUES ('2a704e82-132e-41f2-9746-83e74550d2ea', 'firebase', 'web', '{ "credentials": { "vault": "samagra", "variable": "uci-firebase-notification" } }', 'SamagraFirebaseWeb');
INSERT INTO transformer ("name", "tags", "config", "id", "service_id")
VALUES ('SamagraODKAgg', array['ODK'], '{}', 'bbf56981-b8c9-40e9-8067-468c2c753659', '94b7c56a-6537-49e3-88e5-4ea548b2f075');
INSERT INTO transformer ("name", "tags", "config", "id", "service_id")
VALUES ('SamagraBroadcast', array['broadcast'], '{}', '774cd134-6657-4688-85f6-6338e2323dde', '94b7c56a-6537-49e3-88e5-4ea548b2f075');
INSERT INTO transformer ("name", "tags", "config", "id", "service_id")
VALUES ('SamagraGeneric', array['generic'], '{}', '0832ca13-c698-4234-8070-b5f708bc0b1a', '94b7c56a-6537-49e3-88e5-4ea548b2f075');
Now we can start Sent/Receive messages using uci web channel http://localhost:9098/ but first you should create a bot for conversation after that you will send starting message.
You can start using FusionAuth Console using http://localhost:9011/ and create an Account, for managing users and what resources they are authorized to access.
For managing all the assesment data go on URL : http://localhost:15002/ and track all the tables and relation using token.
Gitpod continuously builds your git branches like a CI server so that you can start coding right away - no more waiting for dependencies to download and builds to finish.
Gitpod = server-side-dev-envs + dev-env-as-code + prebuilds + IDE + collaboration.
Click below button for gitpod workspace
After click above button we will show below screen
While the script is running, You will be asked to enter encryption key, contact the administrator and get this.
If you are asked to enter Netcore Whatsapp Auth Token, Source, URI, enter the details if you have any else press enter. Click here to check the whatsapp configuration for netcore.
After configuration, it will take time for cloning and container creation Around 45 Minutes
If you see below screen it means your all services are up
Note: This script may fail during runtime, in this case run the below commands to restart services.
Stop all the services
docker-compose down
Start all the services
docker-compose up -d
How to get list of open ports
How to get gitpod url for individual service
List of running services port. Click the browser icon
After click, It will open in new tab
Gitpod Sample URL : https://15003-samagracomm-dockerdeplo-93qdi1tvb7x.ws-us47.gitpod.io/console/login
On a successful run of the script you will get below services:
If you want to check logs for a specific service, follow below flow.
Show all docker containers
docker ps -a
Copy the container id from the list for the service you want to see the logs for, and use it in below command.
docker --follow --tail 10 container_id
On Sending the starting message to UCI front , if you do not receive any reply in 1 minute. Follow below steps
Check the logs of inbound service using below command
Check the container id
docker ps -a
Check the logs for orchestrator container
docker logs --follow --tail 100 container_id
One of the service must not be receiving data from topic, restart that service using below commands
Check the container id
docker ps -a
Stop the service
docker stop container_id
Start the service
docker-compose up -d service_name
When we run script on gitpod sometimes we are facing below error
Stop the all services
docker-compose down
Start the all services
docker-compose up -d
APIs
Convert a ODK Excel form to XML form using Link.
Upload this XML from using this api.
curl --location --request POST 'http://localhost:9999/admin/v1/forms/upload' \
--header 'admin-token: EXnYOvDx4KFqcQkdXqI38MHgFvnJcxMS' \
--form 'form=@"{PATH_OF_ODK_FORM}"'
Response: The api will return a form id, use this form id in create conversation logic api. Form id Eg. List-Button-test-v1
{
"ts": "2022-05-24T13:46:06.640Z",
"params": {
"resmsgid": "dc586de0-db67-11ec-ae84-fbd67a9c1174",
"msgid": null,
"status": "successful",
"err": null,
"errmsg": null
},
"responseCode": "OK",
"result": {
"data": "List-Button-test-v1"
}
}
Create a Conversation Logic
curl --location --request POST 'http://localhost:9999/admin/v1/conversationLogic/create' \
--header 'admin-token: EXnYOvDx4KFqcQkdXqI38MHgFvnJcxMS' \
--header 'Content-Type: application/json' \
--data-raw '{
"data": {
"name": "UCI demo bot logic",
"transformers": [
{
"id": "bbf56981-b8c9-40e9-8067-468c2c753659",
"meta": {
"form": "https://hosted.my.form.here.com",
"formID": "List-Button-test-v1"
}
}
],
"adapter": "44a9df72-3d7a-4ece-94c5-98cf26307323"
}
}'
Response: It will return a conversation logic id, use it in create bot api. Eg. 92f7b965-4118-4ddc-9c7d-0bc0f77092db
{
"ts": "2022-05-24T13:48:06.407Z",
"params": {
"resmsgid": "23b94970-db68-11ec-ae84-fbd67a9c1174",
"msgid": null,
"status": "successful",
"err": null,
"errmsg": null
},
"responseCode": "OK",
"result": {
"data": {
"transformers": "[{"id":"bbf56981-b8c9-40e9-8067-468c2c753659","meta":{"form":"https://hosted.my.form.here.com/%22,/%22formID/%22:/%22List-Button-test-v1/%22%7D%7D]",
"adapter": "44a9df72-3d7a-4ece-94c5-98cf26307323",
"name": "UCI demo bot logic",
"id": "92f7b965-4118-4ddc-9c7d-0bc0f77092db"
}
}
}
Create a bot
curl --location --request POST 'http://localhost:9999/admin/v1/bot/create' \
--header 'admin-token: EXnYOvDx4KFqcQkdXqI38MHgFvnJcxMS' \
--header 'Content-Type: application/json' \
--data-raw '{
"data": {
"startingMessage": "Hi Test Bot",
"name": "Test Bot",
"users": [],
"logic": [
"c556dfc8-5dd3-477c-83bb-65d234c4d223" // Get this id from Create a conversation logic api.
],
"status": "enabled",
"startDate": "2022-05-24",
"endDate": "2023-05-24"
}
}'
Response: This api will return a bot id & other bot information. Use the starting message (Eg. Hi Test Bot) from here to start conversation with a bot.
{
"ts": "2022-05-24T13:49:15.292Z",
"params": {
"resmsgid": "4cc874d0-db68-11ec-ae84-fbd67a9c1174",
"msgid": null,
"status": "successful",
"err": null,
"errmsg": null
},
"responseCode": "OK",
"result": {
"data": {
"startingMessage": "Hi Test Bot",
"name": "Test Bot",
"users": [],
"status": "enabled",
"startDate": "2022-05-24",
"endDate": "2023-05-24",
"logicIDs": [
"92f7b965-4118-4ddc-9c7d-0bc0f77092db"
],
"id": "9f0b1401-44d2-46be-83bd-7cbd5014f899"
}
}
}
Note: If you want to use the gupshup adapter, please contact the administrator for credentials.
Once the bot is created, we can start using it. Send the starting message added in the Create a bot api.
Example : Hi Test Bot
Once the bot is created, we can start using it. If you have set up gupshup/netcore provider for whatsapp, Send the starting message added in the Create a bot api to the whatsapp number.
To get the messages from any service provider say netcore/gupshup, contact their support team, and ask them to add your ip with netcore/gupshup adapter url For Netcore: ip:inbound_extrenal_port/netcore/whatsApp (Eg. - 143.112.x.x:9080/netcore/whatsApp) For Gupshup: ip:inbound_external_port/gupshup/whatsApp (Eg. - 143.112.x.x:9080/gupshup/whatsApp)
To sent messages via outbound, we'll use API from service provider (Netcore/Gupshup). For Netcore, Contact their support team to grant below credentials :
NETCORE_WHATSAPP_AUTH_TOKEN = # Authentication Token
NETCORE_WHATSAPP_SOURCE = # Source ID for sending messages to Netcore
NETCORE_WHATSAPP_URI = # Netcore API Base URL
Update these details in .env file and restart inbound & outbound services to reflect the changes.
Please write to the Maintainer - Chakshu (chakshu@samagragovernance.in), and cc - Saket (saket@samagragovernance.in), Sukhpreet (sukhpreet@samagragovernance.in)
If asked Would you like to share anonymous usage data about this project with the Angular Team at Google under Google’s Privacy Policy at https://policies.google.com/privacy? For more details and how to change this setting, see http://angular.io/analytics., Press y.
If you change anything in .env file, you will have to stop the services, then restart them.
Stop all services:
docker-compose -f docker-compose.yml down
Start all services:
docker-compose -f docker-compose.yml up -d