search

gosom / google-maps-scraper

scrape data data from Google Maps. Extracts data such as the name, address, phone number, website URL, rating, reviews number, latitude and longitude, reviews,email and more for each place
MIT License
851 stars 115 forks source link

Choosing fields #57

Open paquino11 opened 3 months ago

paquino11 commented 3 months ago

Hi!

First of all, thank you very much for creating this amazing repo.

I was wondering if it is possible to choose just the fields we want to scrape instead of scraping the entire list available.

Thanks!

gosom commented 3 months ago

@paquino11 choosing the fields to scrape will require significant modifications that do not worth the effort.

Do you want to improve perfomance or you just want the output to have only the fields you want?

paquino11 commented 3 months ago

Hi,

Mainly performance and resources since I just need 4-ish fields.

I was trying to modify the entry.go file by commenting fields I don't need on EntryFromJSON and CsvRow functions and Entry type, but didn't work.

Are the changes done inside the Docker file? I can support you on that if you don't mind.

gosom commented 3 months ago

To do it properly you need to pass as parameters the fields you need and then based on that dynamically execute only the relevant code. Probably also the Job's properties need to be mdified to be pointers and if a value is nil needs to be excluded for the CSV.

Regarding your questions I don't fully understand this. If you change something locally then you need to build another docker image .

a Pull request is more than welcome @paquino11 .

In any case I will try to add this feature since it was requested also by others.