winiar93 / FastEstate

scraper of otodom.pl
MIT License
3 stars 1 forks source link

FastEstate

This project is a estate offers scraper written in Python. It allows you to scrape real estate offers from a specific source and store them in a PostgreSQL database. It also provides an API built with FastAPI, which enables users to fetch the offers and rank them based on certain criteria.

Installation

  1. Clone the repository:

    
    https://github.com/winiar93/real-estate-price-scraper.git
  2. Navigate to the project directory

  3. Build and run the Docker containers using Docker Compose:

    
    docker-compose up --build

This command will build the necessary Docker images and start the containers for the scraper, PostgreSQL database, and FastAPI server.

Usage

Fetching Offers using FastAPI

To access the API for fetching and ranking offers, make requests to the FastAPI server running in the Docker container. The server is accessible at http://localhost:8000.

Additionaly You can use pgadmin to view data or make some analysis, it is accessible at http://localhost:8080/ .

Endpoints

scrape_data

To trigger scraper, make a GET request to http://localhost:8000/scrape_data.

get_data

To fetch all offers, make a GET request to http://localhost:8000/get_data.

get_file

To get csv file with offers, make a GET request to http://localhost:8000/get_file.