meibenjin / GoogleSearchCrawler

a tool for crawl Google search results
MIT License
390 stars 257 forks source link

Suggesting TODO list #15

Open ahangarha opened 5 years ago

ahangarha commented 5 years ago

I have this todo list for the beginning. if there is no objection, I work on it.

meibenjin commented 5 years ago

that's great, I have not much time to maintain this project, I'm agree with your suggestions

ahangarha commented 5 years ago

Regarding 3rd todo, is there any consideration in the format of the out put? is there any program build on top of this script which is dependent on the output format of the this script?

I have made some changes on my system to make it like the following. tell me if it is fine to push it to the script.

✗ python gsearch.py "hello world"
[INFO] 5 results found for the search term
[INFO] 5 results found for the search term
  1 TITLE     : "Hello, World!" program - Wikipedia
  1 URL       : https://en.wikipedia.org/wiki/%2522Hello,_World!%2522_program
  1 CONTENT   : A "Hello, World!" program is a computer program that outputs or displays the 
message "Hello, World!". Because it is very simple in most programming 
languages, ...

  2 TITLE     : Hello, World! - Learn Python - Free Interactive Python Tutorial
  2 URL       : https://www.learnpython.org/en/Hello,_World!
  2 CONTENT   : Hello, World! Python is a very simple language, and has a very straightforward 
syntax. It encourages programmers to program without boilerplate (prepared) ...

  3 TITLE     : Hello World - Rust By Example - Rust Documentation
  3 URL       : https://doc.rust-lang.org/rust-by-example/hello.html
  3 CONTENT   : This is the source code of the traditional Hello World program. // This is a 
comment, ... rustc will produce a hello binary that can be executed. $ ./hello Hello 
World ...
meibenjin commented 5 years ago

Regarding 3rd todo, is there any consideration in the format of the out put? is there any program build on top of this script which is dependent on the output format of the this script?

I have made some changes on my system to make it like the following. tell me if it is fine to push it to the script.

✗ python gsearch.py "hello world"
[INFO] 5 results found for the search term
[INFO] 5 results found for the search term
  1 TITLE     : "Hello, World!" program - Wikipedia
  1 URL       : https://en.wikipedia.org/wiki/%2522Hello,_World!%2522_program
  1 CONTENT   : A "Hello, World!" program is a computer program that outputs or displays the 
message "Hello, World!". Because it is very simple in most programming 
languages, ...

  2 TITLE     : Hello, World! - Learn Python - Free Interactive Python Tutorial
  2 URL       : https://www.learnpython.org/en/Hello,_World!
  2 CONTENT   : Hello, World! Python is a very simple language, and has a very straightforward 
syntax. It encourages programmers to program without boilerplate (prepared) ...

  3 TITLE     : Hello World - Rust By Example - Rust Documentation
  3 URL       : https://doc.rust-lang.org/rust-by-example/hello.html
  3 CONTENT   : This is the source code of the traditional Hello World program. // This is a 
comment, ... rustc will produce a hello binary that can be executed. $ ./hello Hello 
World ...

I suggest making format style configurable, you can use env.

ahangarha commented 5 years ago

I agree with you. Some output such as those starting with [INFO] can be used if the app is started in verbose mode. Counting might be useful for me, but not for all for sure.

Yingjie4Science commented 5 years ago

will you update the code for py3.7?

ahangarha commented 5 years ago

The code needs refactoring. If you think we should migrate totally into Python3, then it should be done ASAP before making any farther change