COS301-SE-2024 / Web-Exploration-Engine

Tech Odyssey - The Web Exploration Engine (WEE) - WEE automates the extraction of critical website information. Employing cutting-edge scraping technologies and natural language processing, WEE delivers user-friendly insights and reports through an intuitive and responsive website
5 stars 3 forks source link

Web Exploration Engine

A Capstone Project for Domain Name Services (Pty) Ltd.

![GitHub issues](https://img.shields.io/github/issues/COS301-SE-2024/Web-Exploration-Engine) ![GitHub pull requests](https://img.shields.io/github/issues-pr/COS301-SE-2024/Web-Exploration-Engine) ![GitHub pull requests](https://img.shields.io/github/languages/count/COS301-SE-2024/Web-Exploration-Engine) ![GitHub pull requests](https://img.shields.io/github/last-commit/COS301-SE-2024/Web-Exploration-Engine) ![GitHub pull requests](https://img.shields.io/github/repo-size/COS301-SE-2024/Web-Exploration-Engine) [![lint-build-test-deploy](https://github.com/COS301-SE-2024/Web-Exploration-Engine/actions/workflows/main.yaml/badge.svg?branch=master)](https://github.com/COS301-SE-2024/Web-Exploration-Engine/actions/workflows/main.yaml) [![codecov](https://codecov.io/gh/COS301-SE-2024/Web-Exploration-Engine/graph/badge.svg?token=YpKw4Ek8cq)](https://codecov.io/gh/COS301-SE-2024/Web-Exploration-Engine)

The Web Exploration Engine (WEE) automates the extraction of critical website information. Employing cutting-edge scraping technologies and natural language processing, WEE delivers user-friendly insights and reports through an intuitive Progressive Web Application (PWA). With customised scraping capabilities, users can benefit from the power of web scraping for diverse needs, from market research and competitor analysis to lead generation.

Documentation and Resources

| Resource | Description | |--------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------| | [Software Requirements Specification](https://drive.google.com/file/d/1wwzlQgMsZSCOejlKFO5LxF2ENHWYg7bx/view?usp=sharing) | Details on project requirements | | [Architectural Specification](https://drive.google.com/file/d/1ZD3nqFtdKrYqurdR8HHw5Yvjg8JevZcI/view?usp=sharing) | System design overview | | [Architecture Diagram](https://drive.google.com/file/d/19bszabKay4TnxgkcHCeUG_R5ovr6d5wl/view?usp=drive_link) | Diagram of system overview | | [Class Diagram](https://drive.google.com/file/d/16yyAfoXCBQyYV_wkpCMy29WbaI8auSaB/view?usp=drive_link) | Visual representation of WEE structure | | [Deployment Diagram](https://drive.google.com/file/d/113kGo7tnekoP_bN0uDI7ym5CjJaA8XVo/view?usp=drive_link) | Plan for project deployment | | [Project Board](https://github.com/orgs/COS301-SE-2024/projects/71) | Team organisation and progress tracking | | [User Manual](https://drive.google.com/file/d/13PiIBegn0emiqJLaUfwy20Ir0Iw4LkrR/view?usp=sharing) | Guidelines to using WEE | | [Contributing](CONTRIBUTING.md) | Guidelines for contributing to WEE | | [Coding Standards](documentation/coding-standards.md) | Guidelines for writing code | | [Previous Documentation Iterations](https://drive.google.com/drive/folders/1H4omJ2xESlP8_STEu7QdVYtMf1rgsZ-S?usp=sharing) | See how our documentation has progressed |

Demo Resources

| Demo | Video | Slides | |---------------------------------|---------------------------------|---------------------------------| | Demo 1 | [Demo 1 Video](https://drive.google.com/file/d/1k34Z3DkhtiaEYbC_AdqoMCpW2_Fb44wW/view?usp=drive_link) | [Demo 1 Slides](https://www.canva.com/design/DAGGIC5EA4E/Sgas9HDvyEG1E8QXFao5xw/edit?utm_content=DAGGIC5EA4E&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton) | | Demo 2 | [Demo 2 Video](https://drive.google.com/file/d/1fmjxiQVnpMjOAuttFO75xRElCkPtDDWh/view?usp=sharing) | [Demo 2 Slides](https://www.canva.com/design/DAGHjFa1XEo/w8trIf8dZMyJ7xtJLpWiag/edit?utm_content=DAGHjFa1XEo&utm_campaign=designshare&utm_medium=link2&utm_source=sharebutton) |

Technologies

Frontend

Backend

Project Management & Deployment

Testing


Meet Tech Odyssey

Johané Breytenbach

Data Engineer, Services Engineer, Testing Engineer
I am a third-year BSc Information and Knowledge System student with a strong focus on Data Science modules and I am well-prepared to contribute significantly to the Web exploration project. My coursework in modules has provided me with a solid foundation in machine learning algorithms and data analysis techniques.

Mignon Erasmus

UI Engineer, Integration Engineer
I am a third year BSc Information and Knowledge Systems student. My degree gives me a strong foundation in Computer Science and Software Development. I am an ambitious, high-achieving and curious individual who always strives for excellence. I find problem solving as an exhilarating experience where one can design and implement unique software solutions that can solve real world problems. I believe that in the ever evolving world of software engineering I am always open to feedback from fellow colleagues and experts to further improve my skills and knowledge. My tenacity ensures that I persist with a problem until I have unravelled it.

Siyamthanda Ndlovu

DevOps, UI Engineer, Testing Engineer
As a final year computer science student, I am deeply engaged in honing my skills and evolving as a software developer and engineer. Currently, I am enrolled in the Artificial Intelligence (COS 314) module at my university, and I will be taking on the Computer Security and Ethics module (COS 330) in the upcoming semester this year. Over the past 14 months, I have actively participated as a member of various teams, from group projects in my degree modules to my work as a Computer Lab Technician. This experience has given me valuable insights into what it takes to deliver high-quality outcomes and has allowed me to leverage my skills to contribute as much as possible to the teams I am a part of.

Caitlin Simon

Project Manager, Services Engineer, Integration Engineer
As a computer science student with a passion for data science, I bring valuable skills and a collaborative approach to this project. My experience in data science includes modules, such as Applications in data science (WST 212) and Artificial Intelligence (COS 314) through languages such as C++, Java and R. I gained practical experience through a quantitative science internship, Quantify Your Future where I was able to apply my theoretical knowledge in machine learning and data analytics to real-world problems in the financial sector.

Katlego Zondo

Services Engineer, Integration Engineer
I am a computer science student with a keen interest in software development. My foundation in computer science principles makes me a valuable asset. I have gained practical experience and honed my skills through hands-on projects during my academic journey. I take every situation as an opportunity to learn and grow, leveraging practical experiences in school to enhance my knowledge and expertise. I am driven by the idea of broadening my experiences and developing my abilities further. I view criticism as a constructive tool for improvement and am dedicated to continually challenging myself to reach new standards of performance.