search

IBM / augmented-reality-powered-interior-decorator

Use IBM Cloud services to create an AR powered interior decorator
Apache License 2.0
36 stars 26 forks source link

readme

Build a 'try and buy' application with AR capabilities for a furniture store

Augmented Reality suits the furniture/interior business perfectly. It is also a fine tool for personalization of customer needs, especially when it comes to furniture/interior. Shoppers really want the ability to see how the items will look at their homes or offices. People want to see virtual interior design ideas in real time, and AR provides them with such ability.

This code pattern will enable developers to build powerful Augmented Reality applications for Android devices, with the help of Google ARCore SDK and IBM Mobile Foundation. To make the interior decorator application more immersive to the user, we will make use of open source 3D poly objects. In order to store all the product information, we will use IBM Cloudant database, which is a fully managed JSON document database that offers independent serverless scaling of provisioned throughput capacity and storage. And to retrieve the product information securely and display it dynamically in the mobile application, we will make use of IBM Mobile Foundation, which provides a rich set of back end capabilities for building, managing and updating your next gen cognitive, engaging, secure and personalized mobile and web applications.

In this code pattern, we will develop an android mobile application on IBM Mobile Foundation with AR capabilities that gives users the ability to select and place the furniture in their desired places, in real time.

Flow

  1. User launches the mobile application.
  2. Mobile application requests IBM Mobile Foundation for product details.
  3. IBM Mobile Foundation fetches the product details from Cloudant.
  4. Product details are displayed on the mobile application.
  5. Users can now view how the furniture would look in their space.

Demo

Pre-requisites

Steps

Please follow the below to setup and run this code pattern.

  1. Clone the repo
  2. Create Cloudant database and populate it with sample data
  3. Setup IBM Mobile Foundation Server & CLI, Build & Deploy Adapter
  4. Setup Poly Objects in Android Studio
  5. Run the Android App

1. Clone the repo

Clone this git repo. Else, in a terminal, run:

$ git clone https://github.com/IBM/augmented-reality-powered-interior-decorator.git

2. Create Cloudant database and populate it with sample data

2.1 Create Cloudant database

Create Database in Cloudant NoSQL DB

Once the database is created, the dashboard will update to show the documents inside mydatabase database (which, as expected, will be empty to begin with).


{
  "_id": "1c8cc4ad5ce9a1823434ebf9004e258e",
  "id": 1,
  "title": "Couch",
  "shortdesc": "Almost Black",
  "rating": 3.2,
  "price": 40000,
  "img": "black_sofa.png"
}

Click Create Document to create/save the document.

The mydatabase database should now list the five documents as shown below under Table view.

Cloudant database populated with sample data

2.2 Generate Cloudant API Key

Make a note of the following things as it will be used in step x.x:

Sl no. Note The Following Example
1. Cloudant URL https://xxxx-xxx-xxxx-xxxx-xxxxx-bluemix.cloudant.com
2. Cloudant Key abcdefxyz1234
3. Cloudant Password 12345678abcd
4. DB Name mydatabase

3. Setup IBM Mobile Foundation Server & CLI, Build & Deploy Adapter

3.1 Setup Ionic and MFP CLI

Note: If you are on Windows, instead of using sudo, run the above command without sudo in a command prompt opened in administrative mode.

Note: While installing MFP CLI, if you hit an error saying npm ERR! package.json npm can't find a package.json file in your current directory., then it is most likely due to MFP CLI not being supported in your npm version. In such a case, downgrade your npm as below, and then install MFP CLI. $ sudo npm install -g npm@3.10.10

3.2 Create Mobile Foundation service and configure MFP CLI

NOTE: The user, password and url is Important as it will be used in subsequent steps.

NOTE: Make Sure the Cloud Foundry App for Mobile Foundation-Server gets at least 768MB of Memory.(Recommended is 1GB) You can verify it by going to IBM Cloud Dashboard > Resources > Cloud Foundry Apps > MobileFoundation-Server as shown below.

Create IBM Mobile Foundation memory

Note: If Mobile Foundation service is not available with your current account type, then you can:

  • Upgrade your account, and avail the Mobile Foundation service's free Developer plan which allows the use of the service free for up to ten daily client devices for development and testing activities.

Note: For Enter the fully qualified URL of this server:, enter the url mentioned in credentials followed by :443 (the default HTTPS port).

$ mfpdev server add

3.3 Deploy the MFP Adapter and Test it

3.3.1 Build and Deploy the MFP adapters
$ cd /CloudantJava

...
<mfpfUrl>https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net:443/mfpadmin </mfpfUrl>
<mfpfUser>admin </mfpfUser>
<mfpfPassword>******** </mfpfPassword>
<mfpfRuntime>mfp </mfpfRuntime>
...

Note: In [Step 3.2], if you specified No to Make this server the default?, then you need to specify the name of your server profile (MyServer in our case) at the end of mfpdev adapter deploy command as shown below.

$ mfpdev adapter deploy MyServer
3.3.2 Launch MFP dashboard and update adapter configurations

Launch MFP Dashboard as below:

  • In the IBM Cloud dashboard, under Cloud Foundry Services, click on the Mobile Foundation service you created in [Step 3.2]. The service overview page that gets shown, will have the MFP dashboard embedded within it. You can also open the MFP dashboard in a separate browser tab by appending /mfpconsole to the url https://mobilefoundation-xxxx-xxxxx.xx-xx.mybluemix.net. Example: https://mobilefoundation-xxxx-xxxxx.xx-xx.mybluemix.net/mfpconsole

NOTE: username & password can be found in Service credentials in step 3.2.

  • Inside the MFP dashboard, in the list on the left, you will see the CloudantJava adapter listed.

Update MFP Adapter configuration as below:

3.3.3 Test the CloudantJava adapter

To Test the adapter use any REST Clients like Postman. After Installing postman type the url created in [step 3.2] and append it with /mfp/api/adapters/CloudantJava/.

Example: https://mobilefoundation-xxxx-xxxxxx.xx-xx.mybluemix.net/mfp/api/adapters/CloudantJava/.

4. Setup Poly Objects in Android Studio

4.1 Google Sceneform Tools Setup

sceneform

NOTE: The Initial Gradle Build may take much longer time please be patient.

4.2 Importing Google Poly Objects into Android Studio

  1. Extract the .zip files.

  2. In Android Studio, in the left panel right click and select New > Sample Data Directory as shown.

sceneform
  1. Right click on the newly created sampledata directory and select Reveal in Finder for Mac.

    On Windows Right click on the newly created sampledata directory and select Show in Explorer.

sceneform

  1. Rename the directories and .obj files which were extracted from the .zip as shown.

    NOTE: Rename as exact same names as shown else the app wont work

sceneform

  1. Now copy paste it in the sampledata directory.

  2. Back to Android Studio, go to sampledata > black_couch and right click on black_couch.obj file and select Import Sceneform Asset as shown.

sceneform
  1. You will get an Import Wizard, make sure the .sfa output path name and .sfb output path name is same as the file name as shown.
sceneform
  1. Wait for the Gradle Build to finish. Upon Successful build you will see the poly image on your screen as shown.
sceneform sceneform

4.3 Resizing the Poly Objects

Once the Poly Objects are loaded into android studio, the size of each poly object is not uniform. Follow the below steps to calibrate the size of each poly object.

model: { attributes: [ 'Position', 'Orientation', ], collision: {}, file: 'sampledata/black_couch/black_couch.obj', name: 'black_couch', recenter: 'root', scale: 1.75, }, version: '0.54:2', }

5. Run the Android App

At this point you will have setup Cloudant DB, Mobile Foundation Server for the Application and setup the Poly Objects in Android Studio. One last thing before you run the android app is to register the application to Mobile Foundation Server so that it can make necessary API calls to Cloudant.

5.1 Register the Android App to IBM Mobile Foundation

Note: In [Step 3.2], if you specified No to Make this server the default?, then you need to specify the name of your server profile (MyServer in our case) at the end of mfpdev app deploy command as shown below. $ mfpdev app register MyServer

5.2 Build/Run the Android App

Note: If you have android adb tools you can check whether your device is connected or not by entering adb devices.

androidstudio

Note: You will get a prompt displaying adb access required, allow the access only then the device will be shown in the above window.

androidstudio androidstudio androidstudio

For a similar retail pattern, check out Integrate a virtual mirror for e-commerce products.

To explore more on Argument Reality, refer to our argument reality related patterns at https://developer.ibm.com/patterns/category/virtual-reality/.

Troubleshooting

Please see troubleshooting guide for solutions to some commonly occuring problems.

androidstudio

License

This code pattern is licensed under the Apache License, Version 2. Separate third-party code objects invoked within this code pattern are licensed by their respective providers pursuant to their own separate licenses. Contributions are subject to the Developer Certificate of Origin, Version 1.1 and the Apache License, Version 2.

Apache License FAQ