search

starnightcyber / ZoomEye

A script used to get data from ZoomEye
32 stars 23 forks source link

Query #3

Closed SinThroughCode closed 4 years ago

SinThroughCode commented 4 years ago

So, I've been messing around with this a bit. I notice the query isn't quite working right or maybe I'm not using it properly but I have tried numerous ways. I looked through the .py file and found this line: query = 'port:80 weblogic country:China' I tried this exact line with and without the ' part added and each ip is not from china I was trying to do netwave country:US and tried multiple variations of it with no results. It finds the netwave but not the country and I'm not sure on why. I even tried dates and that didn't seem to work much either. I've been looking for a script that searches by exact pages like this one but the query just won't work for me. Am I doing something wrong? I'd love to use this to it's full effect.

starnightcyber commented 4 years ago

helo, can you read chinese? you can refer the doc : search-filters More than one year passed, maybe some small changes being to api.

SinThroughCode commented 4 years ago

I cannot read Chinese but I can probably use translater. I'll check out the search filter

SinThroughCode commented 4 years ago

helo, can you read chinese? you can refer the doc : search-filters More than one year passed, maybe some small changes being to api.

It doesn't seem to be changed. I'm using those exact search filters. Would you mind trying out yourself to search for app:netwave service:http country:China

When I run it gets netwave but ignores all other filters. Not sure why. This is a good tool and it would be cool if you can check this out

starnightcyber commented 4 years ago

There is something wrong with my account I cannot test the script, and i find something might be useful for you:

https://github.com/Xyntax/POC-T/tree/2.0/lib/api/zoomeye

I see the only difference is facets, and i'm not sure whether the api returns the right query result exactly.

SinThroughCode commented 4 years ago

I have a script that works but it grabs from random pages. That's what makes your script good, You can specify what pages to start from and end from. I tried doing the search term you gave me and it doesn't seem to work still. I could link you to the script and maybe you can check out what the difference between yours and that script?

starnightcyber commented 4 years ago

I have a script that works but it grabs from random pages. That's what makes your script good, You can specify what pages to start from and end from. I tried doing the search term you gave me and it doesn't seem to work still. I could link you to the script and maybe you can check out what the difference between yours and that script?

try again... please use the exact query condition by copy from the website that you want to search.

SinThroughCode commented 4 years ago

That's the thing.. I've tried copying the search by clicking filters and everything and I even tried manipulating the filters in every way I could think of and no matter what way or what search I do it doesn't grab the country code filter. It only grabs netwave for some reason. Even putting " around it and ' around the filter does not work either. https://github.com/adcar/zoomeye-search/blob/master/zoomeye.py this one works with the filters that I tried perfectly fine but it's kinda trash script for grabbing because it parses random pages

starnightcyber commented 4 years ago

The script is updated, and tests ok. if you find anything wrong, please open a new issue.

SinThroughCode commented 4 years ago

Okay awesome. I'll run some tests right now to see how things go

SinThroughCode commented 4 years ago

okay, So I ran a test. What I found out is, you did indeed fix the filters which is awesome. I ran a test using string: netwave +after:"2020-04-13" +before:"2021-01-01" +country:"US" +service:"http" and I geo located the ips and all of them but 1 was US. For some reason it grabbed a Brazilian IP which is odd. But then I put that IP in zoom eye search and it came up as US so idk what that is about. But I also monitored the pages that I was grabbing. I started at page 1 and let it grab 2 pages as a short test. None of the IPs that it grabs are from page 1 or 2 on zoomeye for some reason.

So, I ran another test. I check the amount of pages there were (40) and wanted to see if it would go through all the pages. It went until page 10 then I got this: [-] info : 'matches' [11/40] fetch page: 11 query==> netwave +after:"2020-04-13" +before:"2021-01-01" +country:"US" +service:"http"

which is odd because there are 40 pages of IPs to search.

I also noticed that after getting this, all the IPs that were grabbed and found were not put into the .txt output for some reason. Almost as if it wasn't even collecting them. Those are the only issues I've found so far.

starnightcyber commented 4 years ago

you cannot fetch all the result 40 pages, the api has the limitation. in my memory,that you can only get first 20%~40% of the results that you query 。 please check service :https://www.zoomeye.org/business

SinThroughCode commented 4 years ago

Oh, well that explains how I can get 2k results with different search queries. Well, with that being said.. then the script works fine now then. Just have to do less accurate searches to get more IPs. Thanks for giving the time to fix the script. It will work wonders.