Cgboal / SonarSearch

A rapid API for the Project Sonar dataset
MIT License
641 stars 96 forks source link

When searching for subdomains, I'd like the input to be used as is #19

Closed dee-see closed 3 years ago

dee-see commented 3 years ago

Hello!

When hitting the /subdomains/{domain} endpoint, if domain is abc.domain.com, the subdomain search will run for domain.com because of https://github.com/Cgboal/SonarSearch/blob/3135da3283dcb9c463aca01a0e3924be9c1f499f/crobat-server/server.go#L99

I wish it ran on abc.domain.com as provided. For example if I want sports.yahoo.com subdomains, searching for yahoo.com instead provides many uninteresting results and makes the execution much longer.

I'd be happy to contribute this feature but I'm wondering how you'd prefer to see this implemented. A new route? Or a query parameter on the existing route?

Thanks!

Cgboal commented 3 years ago

Basically this is due to the way the data is indexed in the database. I dont think we'd manage to index it properly with all the various levels of sub-sub domains, however, should be simple enough to filter the results once they've been extracted from the database. Will implement this sometime this week.

On Sat, Dec 5, 2020, 2:48 AM Dominic notifications@github.com wrote:

Hello!

When hitting the /subdomains/{domain} endpoint, if domain is abc.domain.com, the subdomain search will run for domain.com because of https://github.com/Cgboal/SonarSearch/blob/3135da3283dcb9c463aca01a0e3924be9c1f499f/crobat-server/server.go#L99

I wish it ran on abc.domain.com as provided. For example if I want sports.yahoo.com subdomains, searching for yahoo.com instead provides many uninteresting results and makes the execution much longer.

I'd be happy to contribute this feature but I'm wondering how you'd prefer to see this implemented. A new route? Or a query parameter on the existing route?

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Cgboal/SonarSearch/issues/19, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4Q7H77LYF3QKDWJFAEHLSTGNPZANCNFSM4UOG3IDQ .

Cgboal commented 3 years ago

This is done now, give it a shot.

Cgboal commented 3 years ago

Ah actually it's not perfect, 1 sec

Cgboal commented 3 years ago

Okay now it should work great

dee-see commented 3 years ago

Awesome, thanks!