An open Hawaiian indigenous plant identification and aggregation application. Developed for the 2018 Hawaii Annual Code Challenge. Learn more about the challenge and our app at devpost.

Table of Contents

• Overview
• Quick Start Application
• Complete Application Setup
• Data Collection
• Machine Learning
• Results
• Usage
• Sponsors

Overview

To get started, first read the introductory text under Quick Start Application. Or read the Data Collection, Machine Learning, and Results sections of this README.

When ready for more information about each step of the process, visit any of the following:

• Plant Identification Challenge Requirements
• API Server
• App Client
• Data Collection
• Machine Learning

Quick Start Application

Our app was built for the iPhone 6 running on iOS version 12. However, deploying the app to an iPhone device is a long process, especially if your Mac is not up-to-date with the latest OS or Xcode version. Therefore, we recommend these quick start steps for getting up-and-running quickly on the browser. Note, however, that the main feature of the app, plant identification, will not work because the native iOS camera cannot run on the browser.

If you would rather follow the complete instructions and deploy to an iOS device, follow the steps under Complete Application Setup.

Getting Started

1. Clone this repository by running the following command in a terminal: git clone git@github.com:HACC2018/ohia.ai.git.
2. Change into the ohia.ai project directory: cd ohia.ai.
3. When running anything related to the client or server, we require Python 2.7 to be installed, since that is the version that the tfjs-node package supports. Switch your terminal to use Python 2.7 (more information here).
4. Switch your terminal to use Node 10.12.0 (more information here).

Install server dependencies and run the server

1. From the project root directory, run npm install. Among other items, you should see "Downloading libtensorflow" and "Building TensorFlow Node.js bindings" run without error in your output.
2. Create a .env file in the project root by copying the template in .env.sample.
3. Source your environment by running: source .env.
4. You should now see these variables in your environment by running env.
5. Add the machine learning model weight files from our Google Drive by first downloading and unzipping weights.tar.gz using the command tar xzf weights.tar.gz, followed by replacing the /server/models/mobilenetv1-1.00 folder with the respective unzipped folder.

6. Run the server: npm run dev-server. You should see the following output:

   ▶ npm run dev-server
   
   > ohia.ai@0.0.1 dev-server /path/to/ohia.ai
   > nodemon server/index.js
   
   [nodemon] 1.18.4
   [nodemon] to restart at any time, enter `rs`
   [nodemon] watching: *.*
   [nodemon] starting `node server/index.js`
   2018-11-07 07:25:31.386112: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.2 AVX AVX2 FMA
   Server for ohia.ai listening on port 3000

Install Vue.js and Quasar and run the client in a browser

1. Open another terminal window and switch to Python 2.7 and Node 10.12.0.
2. Source the environment: source .env.
3. Install Vue CLI: npm install -g @vue/cli.
4. Install Vue CLI addon: npm install -g @vue/cli-init.
5. Install Quasar CLI: npm install -g quasar-cli.
6. From the project root directory, install client dependencies: npm run install:client.

7. Run npm run dev to view the app in a browser. You should see the following output:

   ▶ npm run install:client
   
   > ohia.ai@0.0.1 install:client ~/ohia.ai
   > cd client && npm install
   
   > fsevents@1.2.4 install ~/ohia.ai/client/node_modules/fsevents
   > node install
   
   [fsevents] Success: "~/ohia.ai/client/node_modules/fsevents/lib/binding/Release/node-v64-darwin-x64/fse.node" already installed
   Pass --update-binary to reinstall or --build-from-source to recompile
   added 1333 packages from 612 contributors and audited 16152 packages in 23.649s
   found 0 vulnerabilities
   
   (python2)

8. Visit http://localhost:8080 to view the app.
9. The app is best viewed by enabling the Chrome device toolbar and setting the device to iPhone 6/7/8.

Complete Application Setup

Getting Started

1. Clone this repository by running the following command in a terminal: git clone git@github.com:HACC2018/ohia.ai.git.
2. Change into the ohia.ai project directory: cd ohia.ai.
3. When running anything related to the client or server, we require Python 2.7 to be installed, since that is the version that the tfjs-node package supports. Switch your terminal to use Python 2.7 (more information here).
4. Switch your terminal to use Node 10.12.0 (more information here).

Installation and Execution

1. Follow the instructions on the API Server README.
2. Follow the instructions on the App Client README.

Data Collection

Machine learning involves learning from data, thus data collection is an integral part of our application. For more details see ohia.ai/data_collection. View the scraped data in our Google Drive, within the images folder, and our plant metadata within the plant_meta folder.

Machine Learning

Behind the scenes, ohia.ai is powered by machine learning and AI. Our team utilizes modern deep learning techniques and large open source datasets to achieve highly accurate classification on a wide range of flora found throughout the Hawaiian Islands. For more details see ohia.ai/machine_learning.

Results

Usage

Find info on the plants of Hawai'i.

Easy-to-follow steps.

AI produces top 3 most likely predictions of plant identity.

An example of most probable plant identity.

Detailed Hawai'i-related information on native plants, including endangerment status.

Historical uses of plants and gallery of uploaded images.

Confirm the plant's identity.

Sponsors