UTokyo-PBL/pbl2022-2022-team6

TranslanGo

TranslanGo is an image based translation app that allows anyone with a smart device to learn about their environment. Users can capture an object with their cameras, learn how to say it in the local language and play games around those objects.
Explore the docs »

View Demo · Report Bug · Request Feature

Table of Contents

About The Project
- Built With
Contributors
Getting Started
Roadmap

About The Project

As international students and avid travelers, we know that communication is one big concern while going overseas. We started TranslanGo with the mission to help people relax and have fun with new languages instead. TranslanGo is currently available in 7 languages!

Here is what you can do with TranslanGo:

Identify objects using your camera / camera roll and translate them into your favourite languages!
Learn how to say or write some text in your favourite languages!
Keep a map diary of all the things you learnt about with our map feature!
Play quizzes and practice your language skills to learn as you go

This repository contains the source code that makes the application possible! Follow along to learn more about TranslanGo.

(back to top)

Built With

TranslanGo is a testament to the advancement of framework and libraries available. It is made with:

(back to top)

Contributors

This project would not be possible without the contribution of the five active team members. Any contributions made are greatly appreciated. The list of teammates with their expertise are listed in alphabetical order.

Rodrigo Curiel - Middle End | Project Manager
Gaurish Gangwar - Front End | Machine Learning
Kunsheng Li - Back End | Database Integration
Yu Nakai - Back End | Cloud Developer
Megha Sharma - Front End | Project Management

(back to top)

Getting Started

The project code has been divided into three branches, which all refer to the following aspect of the project:

Front End

The branch frontend tracks the progress of the frontend code for the app. The initial code for this app was generated by create-react-app using this command:

npx create-react-app translango --template=typescript

To get started with the project front end, follow the steps below

Installation

Install all the plug ins required to run the application using the command below

Clone the repo

https://github.com/UTokyo-PBL/pbl2022-2022-team6.git

Install NPM packages
```
npm install
```
Go to localhost:3000/ to view the application locally

Please note that you need to run your own Google Maps and Places API to run the application in your local host.

Machine Learning

This branch deals with the cloud APIs used for machine learning tasks, such as Google Vision and Google Translate. It is recommended to run this code using your own APIs to prevent unwanted behavior.

Back End

The branch develop tracks the progress of the back end code for the app. This deals with the server handling the database and requests from the front end. The usage is highlighted in the subsections below.

Usage

view languages

$ curl https://translango.y-nakai.com/user/signup

$ curl -XPOST \
  -d '{"id":"210e4cee-ba0b-4cb7-848c-2901e3310cee","email":"example@translango.com","password":"passw0rd","first_name":"nakai","last_name":"yu","username":"nakai-yu","language":"ja"}' \
  -H "content-type: application/json" \
  https://translango.y-nakai.com/user/signup

$ curl -XPOST -s \
  -d '{"email":"example@translango.com","password":"passw0rd"}' \
  -H "content-type: application/json" \
  --dump-header - \
  https://translango.y-nakai.com/user/login

$ SESSION=$(curl -XPOST -s \
  -d '{"email":"example@translango.com","password":"passw0rd"}' \
  -H "content-type: application/json" \
  --dump-header - \
  https://translango.y-nakai.com/user/login | \
  grep -E "[s|S]et-[c|C]ookie" | \
  sed -e "s/^.\{12\}//")

logout

$ curl -XPOST \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  https://translango.y-nakai.com/user/logout

profile

$ curl \
  -H "Cookie: ${SESSION}" \
  https://translango.y-nakai.com/user/profile

edit profile

$ curl -XPOST \
  -d '{"id":"210e4cee-ba0b-4cb7-848c-2901e3310cee","email":"example@translango.com","password":"passw0rd","first_name":"nakai","last_name":"yu","username":"nakai-yu","language":"ja"}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  https://translango.y-nakai.com/user/profile

dashboard/top

$ curl \
  -H "Cookie: ${SESSION}" \
  https://translango.y-nakai.com/dashboard/top

edit dashboard/top

$ curl -XPOST \
  -d '{"languages": ["ja"]}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  https://translango.y-nakai.com/dashboard/top

get objects

$ curl \
  -H "Cookie: ${SESSION}" \
  https://translango.y-nakai.com/dashboard/histories

detect object

$ curl -XPOST \
  -d '{"id":"8d5eeaff-654d-4cde-a075-07d2a04c26be","image_url":"https://dime.jp/genre/files/2020/11/44817f7cc02f549d516a94cc2710c53f.png","original":{"id":"5c81a2c7-7075-4e61-9e22-897792d62510", "language":"en"}, "target": [{"id":"58cb8ec4-ab50-4cdc-a553-dcefb68aad2b", "language": "ja"}]}' \
  -H "content-type: application/json" \
  "https://translango.y-nakai.com/dashboard/histories?type=object"

$ curl -XPOST \
  -d '{"id":"8d5eeaff-654d-4cde-a075-07d2a04c26be","image_url":"https://car-mo.jp/mag/wp-content/uploads/2020/12/4201119-n-one_101H.jpg","original":{"id":"5c81a2c7-7075-4e61-9e22-897792d62510", "language":"en"}, "target": [{"id":"58cb8ec4-ab50-4cdc-a553-dcefb68aad2b", "language": "hi"}]}' \
  -H "content-type: application/json" \
  "https://translango.y-nakai.com/dashboard/histories?type=object"

$ curl -XPOST \
  -d '{"id":"8d5eeaff-654d-4cde-a075-07d2a04c26be","image_url":"https://dime.jp/genre/files/2020/11/44817f7cc02f549d516a94cc2710c53f.png","original":{"id":"5c81a2c7-7075-4e61-9e22-897792d62510", "language":"en"}, "target": [{"id":"58cb8ec4-ab50-4cdc-a553-dcefb68aad2b", "language": "ja"}]}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories?type=object"

translate

$ curl -XPOST \
  -d '{"id":"ee520bd2-ca07-48ff-9838-5549811cdb6d","original":{"id":"15fcfaa7-5fc1-4a3b-b86a-074f8dac6856","language":"en","text":"cat"}, "target": [{"id":"b2f65661-aabc-4b36-b3d4-d2823c6b9e1a", "language": "ja"}]}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories?type=text"

$ curl -XPOST \
  -d '{"id":"ee520bd2-ca07-48ff-9838-5549811cdb6d","original":{"id":"15fcfaa7-5fc1-4a3b-b86a-074f8dac6856","language":"en","text":"Tall Taller Tallest"}, "target": [{"id":"b2f65661-aabc-4b36-b3d4-d2823c6b9e1a", "language": "ja"}]}' \
  -H "content-type: application/json" \
  "https://translango.y-nakai.com/dashboard/histories?type=text"

get object

$ curl \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories/8d5eeaff-654d-4cde-a075-07d2a04c26be"

delete object

$ curl -XDELETE \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories/8d5eeaff-654d-4cde-a075-07d2a04c26be"

$ curl -XPOST \
  -d '{"liked":true}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories/8d5eeaff-654d-4cde-a075-07d2a04c26be/liked"

caption

$ curl -XPOST \
  -d '{"caption":"nice to meet you"}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories/8d5eeaff-654d-4cde-a075-07d2a04c26be/caption"

increment num failures

$ curl -XPOST \
  -d '{"num_failures": 1}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories/8d5eeaff-654d-4cde-a075-07d2a04c26be/num_failures"

update translation

$ curl -XPOST \
  -d '{"original":{"id":"5c81a2c7-7075-4e61-9e22-897792d62510","text":"猫","language":"zh"}}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/histories/8d5eeaff-654d-4cde-a075-07d2a04c26be/original"

list

$ curl \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/lists"

create custom list

$ curl -XPOST \
  -d '{"id":"8e23947f-5fc0-41a5-b24b-7adc1d4b8bf9","icon-name":"US","name":"english","objects":[{"id":"8d5eeaff-654d-4cde-a075-07d2a04c26be"}]}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/lists"

get custom list

$ curl \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/lists/8e23947f-5fc0-41a5-b24b-7adc1d4b8bf9?num_questions=2"

get all list

$ curl \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/lists/210e4cee-ba0b-4cb7-848c-2901e3310cee?num_questions=2"

update custom list

$ curl -XPUT \
  -d '{"id":"8e23947f-5fc0-41a5-b24b-7adc1d4b8bf9","icon-name":"US","name":"english","objects":[{"id":"8d5eeaff-654d-4cde-a075-07d2a04c26be"}]}' \
  -H "content-type: application/json" \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/lists/8e23947f-5fc0-41a5-b24b-7adc1d4b8bf9"

delete custom list

$ curl -XDELETE \
  -H "Cookie: ${SESSION}" \
  "https://translango.y-nakai.com/dashboard/lists/8e23947f-5fc0-41a5-b24b-7adc1d4b8bf9"

(back to top)

Roadmap

See the open issues for a full list of proposed features (and known issues).

(back to top)