search

SaddlebackCollegeRobotics / .github

0 stars 0 forks source link

Web-scraper for Unifi web-interface #15

Closed Supernova1114 closed 8 months ago

Supernova1114 commented 9 months ago

Datasheet for rover AP: https://dl.ubnt.com/datasheets/bullet_ac/Bullet_AC_DS.pdf

devloos commented 9 months ago

Project Plan

  1. Determine Ubiquti Web GUI API Availability:

    • Investigate whether the Ubiquti Web GUI provides API endpoints for data retrieval.

  2. Fetch Data through API (if available):

    • If API endpoints are found, retrieve the necessary data using the API and plan how to present it effectively.

  3. Explore Web Scraping as an Alternative (if no API):

    • If API endpoints are not available, explore the option of web scraping to collect the required data.
    • Utilize the BeautifulSoup library in Python, designed to facilitate web scraping.

  4. Address Web Scraping Challenges:

    • Recognize potential challenges, such as the need for continuous data scraping due to frequent updates.
    • Evaluate the suitability of BeautifulSoup for handling data from locally hosted websites.

If everything magically works here is my plan on how I will web scrap

import requests
from bs4 import BeautifulSoup

URL = "https://192.168.172.1/#dashboard"
page = requests.get(URL, verify=False)

soup = BeautifulSoup(page.content, "html.parser")
results = soup.find_all(class_="freq")
print(results.prettify())
from bs4 import BeautifulSoup
from selenium import webdriver

url = "http://legendas.tv/busca/walking%20dead%20s03e02"
browser = webdriver.PhantomJS()
browser.get(url)
html = browser.page_source
soup = BeautifulSoup(html, 'lxml')
a = soup.find('section', 'wrapper')

Resources