This project provides a set of Python scripts to scrape data from Google Maps using the Playwright library for automated web browsing. The data scraped includes names, addresses, phone numbers, websites, pluscode, longitude / latitude, ratings and other relevant details of listings found on Google Maps.
scraper.py
: Contains functions to interact with Google Maps through a browser session, scroll through listings, and extract listing details.utils.py
: Includes utility functions for reading search terms, saving data to Excel, merging Excel files, adjusting column widths in Excel, and parsing coordinates from URLs.run.py
: Orchestrates the scraping process using functions from other modules, handles user inputs, and manages the overall flow of the application.Query.txt
file.Clone this repository:
git clone https://github.com/r7avi/Google-Maps-Scrapper.git
Navigate to the project directory:
cd google-maps-scraping
Update PIP:
python -m pip install --upgrade pip
Install the required Python packages:
pip install -r requirements.txt
or
pip install playwright pandas openpyxl tqdm playwright install
To use this script, follow these steps:
Start the script run.py
.
python run.py
Choose to input a search term manually or use Query.txt
.
doctors in new york
dentist in new yersy
Enter the total number of results you want to scrape in run.py Line : 16
Min: 20 (Default) & Max: 150 ## Recommended 120
output
directory.Line 45 in scraper.py & Line 15 in run.py #Change the path of chrome.exe as Required
executable_path='C:\Program Files\Google\Chrome\Application\chrome.exe' : Make sure you have chrome installed in your system and the path is correct
Headless Browser Mode
headless=False : Set to True if you don't want to see the browser window
If you want to run in Docker ,In Docker/app : update Query.txt and Modify Server details in Scrapper.py and add your Linux SSH Details so Scrapped data can be stored in your Linux server else It will be stored Locally in app/output.
Execute below command in Docker folder . [Note You Must Have Docker Installed in your Win/Linux]
docker-compose up --build