search

csueb-fall21-android / shopping-assist

0 stars 0 forks source link

readme

===

Shopping Assistant

Table of Contents

  1. Overview
  2. Product Spec
  3. Wireframes
  4. Schema
  5. Milestone Walkthroughs

Overview

Description

The aim is to build a shopping assistant app that will provide a seamless search capability for searching products on e-commerce websites using images captured by mobile cameras and it will show all related information of products with more descriptions, prices, and nearby store location options. Users can sort search results with cheaper prices, nearby locations, etc. Users can save search products that you've found but want to buy later to their account/profile and can see them.

App Evaluation

Product Spec

1. User Stories (Required and Optional)

Required Must-have Stories

Optional Nice-to-have Stories

2. Screen Archetypes

3. Navigation

Tab Navigation (Tab to Screen)

Flow Navigation (Screen to Screen)

Wireframes

[BONUS] Digital Wireframes & Mockups

Made with balsamiq.cloud.

Project link here.

Landing Page > Signup / Login

Signup / Login > Feed

Feed (Basic View)

Feed (with More Features)

Details Found (Screen after taking a picture)

No Details Found (Screen after taking a picture)

Add Details (After selecting to add more details)

Add Details (More features)

View Details (After successfully adding details)

Search (Clicking on search nearby or online buttons)

User Options

[BONUS] Interactive Prototype

Schema

Models

User

Property Type Description
userId String unique id for the user (default field)
email String email for the user
password String password for the user
defaultLocation Pointer to Location a location for the user for search criteria, if nothing else is provided
defaultLocationRadius Number a radius in miles for search criteria, if nothing else is provided
createdAt DateTime date when user is created (default field)
updatedAt DateTime date when user is last updated (default field)

Item

Property Type Description
itemId String unique id for the item (default field)
user Pointer to User user who created this item
location Pointer to Location location that this item was found
name String name of item
price Number price of the item
details String details about the item
brand String (optional) brand of the item
externalLink String external link associated with the item
isArchived Boolean whether the item is archived by the user
createdAt DateTime date when item is created (default field)
updatedAt DateTime date when item is last updated (default field)

[Relational] ItemRecommendedItem (One-to-many)

Property Type Description
itemRecommendedItemId String unique id for the relationship between an item and recommended item (default field)
item Pointer to Item an item
recommendedItem Pointer to RecommendedItem a recommended item that is related to the item
createdAt DateTime date when relationship is created (default field)
updatedAt DateTime date when relationship is last updated (default field)

RecommendedItem

Property Type Description
recommendedItemId String unique id for the recommended item (default field)
location Pointer to Location location that this item was found
name String name of item
price Number price of the item
details String details about the item
brand String (optional) brand of the item
externalLink String external link associated with the item
createdAt DateTime date when item is created (default field)
updatedAt DateTime date when item is last updated (default field)

Picture

Property Type Description
pictureId String unique id for the picture (default field)
item Pointer to Item item that the picture is related to
description String text description of the picture
pictureFile File the file of the picture
createdAt DateTime date when picture is created (default field)
updatedAt DateTime date when picture is last updated (default field)

Location

Property Type Description
locationId String unique id for the location (default field)
descriptor String an identifying description of the location
coordinates String the coordinates of the location
createdAt DateTime date when location is created (default field)
updatedAt DateTime date when location is last updated (default field)

Note: This table will integrate w/ Google Maps, so schema may change

Networking

List of network requests and Snippets by screen

Take a Picture Screen

(Create/POST) Post the picture to the backend for image processing, extract the name and price, search for the closest item online based on the name/details, and return name and price, and details (if found) (Cloud Vision API)[https://cloud.google.com/vision/docs/ocr#vision_text_detection_gcs-java]: using snippet from here for text detection in an image. Also look into integration w/ (Google's Firebase ML)[https://firebase.google.com/docs/ml]. Search API: ? Could use Google, but has limitations. Will have to search for an alternative.

Cloud Vision API Request:

    List<String> extractedText = new ArrayList<>();
    // Detects text in the specified local image.
    public static void detectText(String filePath) throws IOException {
        List<AnnotateImageRequest> requests = new ArrayList<>();

        ByteString imgBytes = ByteString.readFrom(new FileInputStream(filePath));

        Image img = Image.newBuilder().setContent(imgBytes).build();
        Feature feat = Feature.newBuilder().setType(Feature.Type.TEXT_DETECTION).build();
        AnnotateImageRequest request =
            AnnotateImageRequest.newBuilder().addFeatures(feat).setImage(img).build();
        requests.add(request);

        // Initialize client that will be used to send requests. This client only needs to be created
        // once, and can be reused for multiple requests. After completing all of your requests, call
        // the "close" method on the client to safely clean up any remaining background resources.
        try (ImageAnnotatorClient client = ImageAnnotatorClient.create()) {
            BatchAnnotateImagesResponse response = client.batchAnnotateImages(requests);
            List<AnnotateImageResponse> responses = response.getResponsesList();

            for (AnnotateImageResponse res : responses) {
                if (res.hasError()) {
                    Log.d("CloudVision", "Error: " + res.getError().getMessage());
                    return;
                }

                // For full list of available annotations, see http://g.co/cloud/vision/docs
                for (EntityAnnotation annotation : res.getTextAnnotationsList()) {
                    Log.d("CloudVision", "Text: " + annotation.getDescription());
                    Log.d("CloudVision", "Position: " + annotation.getBoundingPoly());
                    extractedText.add(annotation.getDescription());
                }
            }
        }
    }

    // do something with extractedText

Search API: [TODO]

(External API) post to Google Maps, to ask for the location that the user took this picture and get additional details for this picture Notes for integration: (Location Permissions)[https://developer.android.com/training/location/permissions], (Getting Current Location)[https://developer.android.com/training/location/retrieve-current]

Location:

fusedLocationClient.getCurrentLocation()
        .addOnSuccessListener(this, new OnSuccessListener<Location>() {
            @Override
            public void onSuccess(Location location) {
                // Got last known location. In some rare situations this can be null.
                if (location != null) {
                    // Logic to handle location object
                    latitude = location.getLatitude();
                    longitude = location.getLongitude();
                }
            }
        });

Google Maps API Request:

May specifically narrow down types, according to (this)[https://developers.google.com/maps/documentation/places/web-service/supported_types]. Ideally, we want only business establishments only.

OkHttpClient client = new OkHttpClient().newBuilder()
  .build();
Request request = new Request.Builder()
  .url("https://maps.googleapis.com/maps/api/place/nearbysearch/json?location={latitude}%{longitude}&radius=1500&key=YOUR_API_KEY")
  .method("GET", null)
  .build();
Response response = client.newCall(request).execute();

(Create/POST) When user confirms the details, picture and item details, and location data is saved to the database Note: https://github.com/templates-back4app/android-relations-java/blob/main/app/src/main/java/com/emre/one_to_many_java/book/AddBookActivity.java

ParseObject item, picture, location;

location = new ParseObject("Location");
location.put("descriptor", descriptor);
location.put("coordinates", coordinates);

// get any existing location, or save new location
location.saveInBackground(e -> {
    if (e == null) {
        // location saved
        // assume we can use location immediately, otherwise will need to fetch it again
    }
    else {
        Log.d("Location", "Error: " + e.getMessage());
    }
});

item = new ParseObject("Item");
item.put("name", name);
item.put("price", price);
item.put("details", details);
item.put("brand", brand);
item.put("externalLink", externalLink);
item.put("user", getCurrentUser());
item.put("location", location); // previous saved ParseObject

item.saveInBackground(e => {
    if (e == null) {
        // item saved
        // assume we can use item immediately, otherwise will need to fetch it again
    }
    else {
        Log.d("Location", "Error: " + e.getMessage());
    }
});

picture = new ParseObject("Picture");
picture.put("description", description);
picture.put("pictureFile", new ParseFile(pictureFile));
picture.put("item", item); // pointer to item

picture.saveInBackground(e => {
    if (e == null) {
        // picture saved
    }
    else {
        Log.d("Location", "Error: " + e.getMessage());
    }
});

User Settings

(Update/PUT) Update user email

ParseQuery<ParseObject> query = ParseQuery.getQuery("User");
ParseUser parseUser = ParseUser.getCurrentUser();
query.whereEqualTo("userID", parseUser);
query.findInBackground(new FindCallback<ParseObject>() {
    public void done(List<ParseObject> list, ParseException e) {
        if (e == null) {
            ParseObject person = list.get(0);
            person.put("email", emailAddress.getText().toString());
            person.saveInBackground();
        } else {
            Log.d("score", "Error: " + e.getMessage());
        }
    }
 });

(Update/PUT) Update user password

ParseQuery<ParseObject> query = ParseQuery.getQuery("User");
ParseUser parseUser = ParseUser.getCurrentUser();
query.whereEqualTo("userID", parseUser);
query.findInBackground(new FindCallback<ParseObject>() {
    public void done(List<ParseObject> list, ParseException e) {
        if (e == null) {
            ParseObject person = list.get(0);
            person.put("password", password.getText().toString());
            person.saveInBackground();
        } else {
            Log.d("score", "Error: " + e.getMessage());
        }
    }
 });

(Update/PUT) Update user default location

ParseQuery<ParseObject> query = ParseQuery.getQuery("User");
ParseUser parseUser = ParseUser.getCurrentUser();
query.whereEqualTo("userID", parseUser);
query.findInBackground(new FindCallback<ParseObject>() {
    public void done(List<ParseObject> list, ParseException e) {
        if (e == null) {
            ParseObject person = list.get(0);
            person.put("defaultLocation", defaultLocation.getText().toString());
            person.saveInBackground();
        } else {
            Log.d("score", "Error: " + e.getMessage());
        }
    }
 });

(Update/PUT) Update user location radius search

ParseQuery<ParseObject> query = ParseQuery.getQuery("User");
ParseUser parseUser = ParseUser.getCurrentUser();
query.whereEqualTo("userID", parseUser);
query.findInBackground(new FindCallback<ParseObject>() {
    public void done(List<ParseObject> list, ParseException e) {
        if (e == null) {
            ParseObject person = list.get(0);
            person.put("defaultLocationRadius", defaultLocationRadius.getText().toString());
            person.saveInBackground();
        } else {
            Log.d("score", "Error: " + e.getMessage());
        }
    }
 });

Search List

(Read/GET) Based on the criteria (item, location, radius, online (y/n), search for recommended items in a certain area and return a list of those items

ParseUser parseUser = ParseUser.getCurrentUser();
ParseQuery<ParseObject> query = new ParseQuery("items");
query.setLimit(25);
query.whereEqualTo("userID", ParseUser);
ParseQuery<ParseObject> parseInnerQuery = new ParseQuery<>("location");
ParseQuery<ParseObject> parseInnerQuery2 = new ParseQuery<>("User");
if(productName.getText().toString() != null){
    query.whereEqualTo("name", productName.getText().toString());
}
if(productPrice.getText().toString() != null){
query.whereEqualTo("price", productPrice.getText().toString());
}
query.whereMatchesQuery("userID", parseInnerQuery2);
query.whereMatchesQuery("defaultLocation", parseInnerQuery2);
query.whereMatchesQuery("defaultLocationRadius", parseInnerQuery2);
query.whereMatchesQuery("locationId", parseInnerQuery);
query.include("locationId");
query.findInBackground(new FindCallback<ParseObject>() {
    @Override
    public void done(List<ParseObject> list, ParseException e) {

        if (e == null && list != null && list.size() != 0) {
            List<Item> itemList = new ArrayList<Item>();
            for (ParseObject parseObject : list) {
                itemList.add(Item.getItemById(getValue(parseObject, "itemID")));
            }
            masterItemAdapter.setItems(itemList);
            } else {
            Log.e(TAG,"Something wrong!!",e);
            return;
        }
    }
});

(Create/POST) Save a recommended item to the database, in relation to the item

ParseQuery<ParseObject> recommendedItem, itemRecommendedItem, location;

location = new ParseQuery.getQuery("Location");
location.put("descriptor", descriptor);
location.put("coordinates", coordinates);

// get any existing location, or save new location
location.saveInBackground(e -> {
    if (e == null) {
        // location saved
        // assume we can use location immediately, otherwise will need to fetch it again
    }
    else {
        Log.d("Location", "Error: " + e.getMessage());
    }
});

recommendedItem = new ParseQuery.getQuery("RecommendedItem");
recommendedItem.put("name", name);
recommendedItem.put("price", price);
recommendedItem.put("details", details);
recommendedItem.put("externalLink", externalLink);
recommendedItem.put("location", location); // previous saved ParseObject

recommendedItem.saveInBackground(e => {
    if (e == null) {
        // recommendedItem saved
        // assume we can use recommendedItem immediately, otherwise will need to fetch it again
    }
    else {
        Log.d("Location", "Error: " + e.getMessage());
    }
});

Date currentTime = Calendar.getInstance().getTime();

itemRecommendedItem = new ParseQuery.getQuery("ItemRecommendedItem");
itemRecommendedItem.put("itemRecommendedItemId", itemRecommendedItemId);
itemRecommendedItem.put("itemId", item); // pointer to item
itemRecommendedItem.put("recommendedItemId", recommendedItem);
itemRecommendedItem.put("createdAt", currentTime);

itemRecommendedItem.saveInBackground(e => {
    if (e == null) {
        // itemRecommendedItem saved
    }
    else {
        Log.d("Location", "Error: " + e.getMessage());
    }
});

Detailed Item

(Read/GET) Get item, item details, item picture, item locations, and all recommended items that were previously saved

ParseUser parseUser = ParseUser.getCurrentUser();
ParseQuery<ParseObject> query = new ParseQuery("items");
query.setLimit(25);
query.whereEqualTo("userID", ParseUser);
ParseQuery<ParseObject> parseInnerQuery = new ParseQuery<>("location");
ParseQuery<ParseObject> parseInnerQuery2 = new ParseQuery<>("User");
ParseQuery<ParseObject> parseInnerQuery3 = new ParseQuery<>("Picture");
ParseQuery<ParseObject> parseInnerQuery4 = new ParseQuery<>("ItemRecommendedItem");
ParseQuery<ParseObject> parseInnerQuery5 = new ParseQuery<>("RecommendedItem");
query.whereMatchesQuery("userID", parseInnerQuery2);
query.whereMatchesQuery("defaultLocation", parseInnerQuery2);
query.whereMatchesQuery("defaultLocationRadius", parseInnerQuery2);
query.whereMatchesQuery("itemId", parseInnerQuery3);
query.whereMatchesQuery("itemId", parseInnerQuery4);
parseInnerQuery4.whereMatchesQuery("recommendedItemId", parseInnerQuery5);
query.whereMatchesQuery("locationId", parseInnerQuery);
query.selectKeys(Arrays.asList("itemId","name", "price","details","brand","externalLink","isArchived"));
query.include("locationId");
query.findInBackground(new FindCallback<ParseObject>() {
    @Override
    public void done(List<ParseObject> list, ParseException e) {

        if (e == null && list != null && list.size() != 0) {
            List<Item> itemList = new ArrayList<Item>();
            for (ParseObject parseObject : list) {
                itemList.add(Item.getItemById(getValue(parseObject, "itemID")));
            }
            masterItemAdapter.setItems(itemList);
            } else {
            Log.e(TAG,"Something wrong!!",e);
            return;
        }
    }
});

(Read/GET) Get a short string containing details of the item (that the user can then copy and message; or this could be created on the app side)

ParseUser parseUser = ParseUser.getCurrentUser();
ParseQuery<ParseObject> query = new ParseQuery("items");
query.whereEqualTo("userID", ParseUser);
query.whereEqualTo("itemId", itemID.getValue());
query.include("externalLink");
query.findInBackground(new FindCallback<ParseObject>() {
    @Override
    public void done(List<ParseObject> list, ParseException e) {

        if (e == null && list != null && list.size() == 1) {
            List<Item> itemList = new ArrayList<Item>();
            for (ParseObject parseObject : list) {
                itemList.add(Item.getItemById(getValue(parseObject, "externalLink")));
            }
            masterItemAdapter.setExternalLink(itemList);
            } else {
            Log.e(TAG,"Something wrong!!",e);
            return;
        }
    }
});

(Delete) Delete an item from database

ParseQuery<ParseObject> query = ParseQuery.getQuery("items");
query.whereEqualTo("itemId", itemID.getValue());
query.getInBackground(objectId, new GetCallback<ParseObject>() {
  public void done(ParseObject object, ParseException e) {
    if (e == null) {
       object.deleteInBackground();
    } else {
      Log.e(TAG,"Something wrong in delete Item",e);
      return;
    }
  }
});

[Optional] (Update/PUT) Archives an item, user doesn't get bothered by it anymore, but they can view it (somewhere)

ParseQuery<ParseObject> query = ParseQuery.getQuery("User");
ParseUser parseUser = ParseUser.getCurrentUser();
query.whereEqualTo("userID", parseUser);
query.whereEqualTo("itemId", itemId.getValue());
query.findInBackground(new FindCallback<ParseObject>() {
    public void done(List<ParseObject> list, ParseException e) {
        if (e == null) {
            ParseObject person = list.get(0);
            person.put("isArchived", (isArchived.getValue()==0 ? 1: 0));
            person.saveInBackground();
        } else {
            Log.d("score", "Error: " + e.getMessage());
        }
    }
 });

Milestone Walkthroughs

Milestone 1: Setup Projects and Designs

Milestone 2: Essential Activity Functionality

Signup New User

Feed Page

Search Page

Take a Picture Screen

Milestone 3: Additional Functionality

Details Found Screen with Location

Retake Picture Functionality

Save & Display Recommended Items on Home Page

Created Google Maps and Default/Current Location Functionality

User Default Location Settings Page

Milestone 4: Refine Details

Search for Similar Items from Item Detail Screen

Suggest Item on the Take Picture By Using Search API

Delete Item