The Social Marketplace Automotive Risk Engine (SMARE) Project. This project aims to detect supicious listings and potential instances of insurance fraud posted on the most popular social marketplace sites, such as Craigslist and Facebook Marketplace. In partnership between Statefarm and UTA CSE Senior Design. View the live deployment at smare.lryanle.com.

🔍 Table of Contents

• 💻 Stack

• 📝 Project Summary

• ⚙️ Setting Up

• 🚀 Run Locally

• 🙌 Contributors

• 📄 License

💻 Stack

• frontend
  • Next: A React framework for building web applications with server-side rendering.
  • Typescript: Typed superset of JavaScript that compiles to plain JavaScript.
  • shadcn/ui: A UI library for React, built using Tailwind CSS.
  • Tailwind: Utility-first CSS framework for rapidly building custom designs.
  • Prisma: Next-generation ORM for Node.js and TypeScript.
  • Nextauth: Authentication for Next.js.
  • Framer Motion: A React library to power animations.
  • Lucide: Open-source icon library.
  • Rechart: A composable charting library built on React components.
  • Remark: A Markdown processor powered by plugins part of the unified collective.
  • SWR: React Hooks library for data fetching.
  • zod: TypeScript-first schema validation with static type inference.
• backend
  • Selenium: A suite of tools for automating web browsers.
  • BS4 (Beautiful Soup): A Python library for pulling data out of HTML and XML files.
  • Pymongo: Python driver for MongoDB.
  • Pandas: A fast, powerful, flexible, and easy-to-use open-source data analysis and manipulation tool.
  • Imblearn: Python library to tackle the problem of imbalanced datasets.
  • Loguru: A Python logging library that aims to make life easier for developers.
  • OpenAI API: A generative AI API.

📝 Project Summary

• frontend: Contains the frontend application with various components and settings.
  • frontend/app: The main page for the client-/customer-facing portion of our SaaS. Main feature is a dashboard to present information retreived from our pipeline.
  • frontend/app/api: Our middleware to retrieve, process, and present information from our DB to the web application.
• backend: Houses backend logic including cleaners, models, scrapers, and utilities.
  • backend/src/cleaners: Contains all functionality related to cleaning data in the backend pipeline.
  • backend/src/models: Contains our 6 models to score and help flag social marketplace listings that are suspicious.
  • backend/src/cleaners: Retrieves data from various social media marketplaces and places them in our data pipeline flow.
  • backend/src/utilities: Utilities for all backend-related processes such as logging and our custom DB adapter.
• documentation: Contains project documentation including sprint reports and charters.

⚙️ Setting Up

Database Access

Make a copy of the .env.example file and make the following changes.

1. remove .example from the extension

2. Paste the username and password provided in MongoDB Atlas (if you should have access but do not, please contact @waseem-polus)

3. Paste the connection URL provided provided in MongoDB Atlas. Include the password and username fields using ${VARIABLE} syntax to embed the value of the variable

Run Scrapers locally

Prerequisites

• python3
• pipenv

Installing dependencies
Navigate to scrapers/ and open the virtual environment using

pipenv shell

Then install dependencies using

pipenv install

Scraper Usage
To create build a production-ready Docker Image use

pipenv run build

To create build a development Docker Image use

pipenv run dev

If there is an existing smarecontainer, run the following:

pipenv run stop

To run a docker container "smarecontainer" use (Note: delete any containers with the same name before running)

pipenv run cont

then

# Scrape Craigsist homepage
pipenv run craigslist

# Scrape Facebook Marketplace homepage
pipenv run facebook

🙌 Contributors

Waseem Polus

Ryan Lahlou

Temitayo Aderounmu

Athiya Manoj

Yeabsra Gebremeskel

📊 Statistics

📄 License

This project is licensed under the MIT License - see the MIT License file for details.