search

gwen001 / github-subdomains

Find subdomains on GitHub.
MIT License
653 stars 97 forks source link

Token limit reached #1

Closed giany closed 4 years ago

giany commented 4 years ago

Hi @gwen001,

I'm running this tool for a domain called: braze.com and after sometime seems that the token limit is reached

github-subdomains me$ github-subdomains -d braze.com -t <token>

        █▀▀ ▀█▀ ▀█▀ █ █ █ █ █▀▄   █▀▀ █ █ █▀▄ █▀▄ █▀█ █▄█ █▀█ ▀█▀ █▀█ █▀▀
        █ █  █   █  █▀█ █ █ █▀▄   ▀▀█ █ █ █▀▄ █ █ █ █ █ █ █▀█  █  █ █ ▀▀█
        ▀▀▀ ▀▀▀  ▀  ▀ ▀ ▀▀▀ ▀▀    ▀▀▀ ▀▀▀ ▀▀  ▀▀  ▀▀▀ ▀ ▀ ▀ ▀ ▀▀▀ ▀ ▀ ▀▀▀
                        by @gwendallecoguic

[17:05:07] Domain:braze.com, Output:braze.com.txt, Tokens:1, Delay:2100ms
[17:05:07] Languages:19, Noise:6
[17:05:07] sort:indexed, order:desc, language:, noise:[]
[17:05:09] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=1
1
[17:05:12] current search returned too much results, language filter added for later search
20
[17:05:12] https://github.com/LagSeeN/my-nextdns-setting/blob/0db2e095698336baccf99e6117b9b5db7204d2b5/setting.txt
[17:05:12] .sdk.iad-03.braze.com
[17:05:12] https://github.com/Ultimate-Hosts-Blacklist/WaLLy3K_Blacklist/blob/35f9bcf034860d26ce6b4c028abae2c8798310fe/volatile.list
[17:05:12] sdk.iad-01.braze.com
[17:05:12] https://github.com/Ultimate-Hosts-Blacklist/WaLLy3K_Blacklist/blob/35f9bcf034860d26ce6b4c028abae2c8798310fe/whitelisted.list
[17:05:12] sdk.iad-02.braze.com
[17:05:12] sdk.iad-03.braze.com
[17:05:12] https://github.com/Ultimate-Hosts-Blacklist/WaLLy3K_Blacklist/blob/35f9bcf034860d26ce6b4c028abae2c8798310fe/clean.list
[17:05:12] sdk.iad-04.braze.com
[17:05:12] sdk.iad-06.braze.com
[17:05:12] https://github.com/MISP/misp-warninglists/blob/23418c54831c6f082f76e89af62e4a5c32ccfa53/lists/cisco_top5k/list.json
[17:05:12] iad-01.braze.com
[17:05:12] iad-03.braze.com
[17:05:13] https://github.com/dead-hosts/iOSAdblockList-hosts_git_BlackJack8/blob/4d361b83d35de12bcb7622c1b12158858911550e/output/domains/ACTIVE/list
[17:05:13] alya.iad-03.braze.com
[17:05:13] andromeda.iad-01.braze.com
[17:05:13] atlas.iad-03.braze.com
[17:05:13] belgikanst.iad-03.braze.com
[17:05:13] birigum.iad-03.braze.com
[17:05:13] blucher.iad-03.braze.com
[17:05:13] bond.iad-03.braze.com
[17:05:13] caesar.iad-03.braze.com
[17:05:13] carinus.iad-03.braze.com
[17:05:13] ceres.iad-03.braze.com
[17:05:13] cetus.iad-03.braze.com
[17:05:13] clair.iad-03.braze.com
[17:05:13] cursa.iad-03.braze.com
[17:05:13] https://github.com/dead-hosts/adaway.github.io_git_AdAway/blob/14effaae69cffa3929ddc03ccc032facc0de5eb3/output/domains/ACTIVE/list
[17:05:13] customer.iad-03.braze.com
[17:05:13] daviik.iad-03.braze.com
[17:05:13] epsilon.iad-03.braze.com
[17:05:13] erulus.iad-03.braze.com
[17:05:13] https://github.com/Ultimate-Hosts-Blacklist/quidsup_notrack_trackers/blob/9e2aefb3d671bf195df71b6a745a626df6893709/volatile.list
[17:05:13] www.braze.com
[17:05:13] gaspra.iad-03.braze.com
[17:05:13] giausar.iad-03.braze.com
[17:05:13] gudja.iad-02.braze.com
[17:05:13] hassaleh.iad-03.braze.com
[17:05:13] himbrecha.iad-02.braze.com
[17:05:13] hiram.iad-03.braze.com
[17:05:13] hubert.iad-03.braze.com
[17:05:13] https://github.com/hectorm/hmirror/blob/8450b83804a0cbc99a4385f7cc3ae5be4c87f3b1/data/adaway.org/list.txt
[17:05:13] ibilisk.iad-01.braze.com
[17:05:13] ibis.iad-03.braze.com
[17:05:13] inari.iad-03.braze.com
[17:05:13] https://github.com/Appboy/braze-docs/blob/986f124f4ce48ea37736084bb02cb09da47ad312/_docs/_user_guide/onboarding_with_braze.md
[17:05:13] lab.braze.com
[17:05:13] indus.iad-01.braze.com
[17:05:13] inferna.iad-03.braze.com
[17:05:13] izar.iad-03.braze.com
[17:05:13] lalande.iad-03.braze.com
[17:05:13] lignon.iad-03.braze.com
[17:05:13] mag.iad-03.braze.com
[17:05:13] maia.iad-03.braze.com
[17:05:13] markab.iad-03.braze.com
[17:05:13] mondarigloo.iad-03.braze.com
[17:05:13] moyo.iad-03.braze.com
[17:05:13] nabina.iad-03.braze.com
[17:05:13] muliphein.iad-01.braze.com
[17:05:13] nashira.iad-06.braze.com
[17:05:13] pali.iad-03.braze.com
[17:05:13] phillateemis.iad-03.braze.com
[17:05:13] https://github.com/jmhenrique/adblock/blob/26f7103e8c9d6345f17b2c074d369a83db1cf3b9/etc/adblock_hosts
[17:05:13] rastaban.iad-03.braze.com
[17:05:13] regor.iad-03.braze.com
[17:05:13] rest.iad-01.braze.com
[17:05:13] rest.iad-03.braze.com
[17:05:13] rhea.iad-03.braze.com
[17:05:13] https://github.com/10ZiN/downloads/blob/99fa4cc92213a4e3f64a28202b3e1a8a74939797/iOSAdsLite.txt
[17:05:13] dashboard-03.braze.com
[17:05:13] rotanev.iad-03.braze.com
[17:05:13] ruba.iad-03.braze.com
[17:05:13] https://github.com/austinheap/sophos-xg-block-lists/blob/b5abe83dd44ab517725a4206d95e17fbd81bce60/adaway.txt
[17:05:13] sagan.iad-03.braze.com
[17:05:13] sagittarius.iad-01.braze.com
[17:05:13] sargas.iad-03.braze.com
[17:05:13] sdk.iad-08.braze.com
[17:05:13] segin.iad-03.braze.com
[17:05:13] seginus.iad-03.braze.com
[17:05:13] serpens.iad-01.braze.com
[17:05:13] sirius.iad-03.braze.com
[17:05:13] skai.iad-03.braze.com
[17:05:13] spica.iad-03.braze.com
[17:05:13] talar.iad-03.braze.com
[17:05:13] tali.iad-03.braze.com
[17:05:13] taukibreez.iad-03.braze.com
[17:05:13] timbol.iad-03.braze.com
[17:05:13] vela.iad-01.braze.com
[17:05:13] wahacondrax.iad-03.braze.com
[17:05:13] wurren.iad-03.braze.com
[17:05:13] https://github.com/dead-hosts/adaway.github.io_git_AdAway/blob/14effaae69cffa3929ddc03ccc032facc0de5eb3/clean.list
[17:05:13] xenon.iad-03.braze.com
[17:05:13] yildun.iad-03.braze.com
[17:05:13] https://github.com/Ultimate-Hosts-Blacklist/WaLLy3K_Blacklist/blob/e2977b5957de638fb008c7b9e32d2522cdb595e0/domains.list
[17:05:13] sdk.iad-05.braze.com
[17:05:16] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=2
20
20
[17:05:19] https://github.com/Appboy/appboy-android-sdk/blob/93ebda87f3e33826463970879103166546ce9df0/android-sdk-ui/src/main/java/com/appboy/AppboyAdmReceiver.java
[17:05:19] jira.braze.com
[17:05:19] https://github.com/Appboy/braze-docs/blob/7c232885758e3fd163589df8a1c0ff4182527131/_docs/_developer_guide/platform_integration_guides/ios/content_cards/data_model.md
[17:05:19] blog.braze.com
[17:05:19] https://github.com/Appboy/appboy-android-sdk/blob/93ebda87f3e33826463970879103166546ce9df0/droidboy/src/main/java/com/appboy/sample/activity/SettingsPreferencesActivity.java
[17:05:19] elsa.braze.com
[17:05:19] https://github.com/nextdns/metadata/blob/48fd4dec4f0d28fcf06b0cf1bd0973cade09f14a/privacy/affiliate-tracking-domains
[17:05:19] dashboard-01.braze.com
[17:05:19] https://github.com/Appboy/braze-docs/blob/973b2b9644f7d6d87cb7454dde17770f6dd38f0b/_docs/_user_guide/message_building_by_channel/in-app_messages/beta-preview.md
[17:05:19] cdn.braze.com
[17:05:19] https://github.com/ScentreGroup/braze-mock-server/blob/60028d89323457b58ea04234ec685c20193314cf/braze_swagger.json
[17:05:19] rest.iad-02.braze.com
[17:05:19] rest.iad-04.braze.com
[17:05:19] rest.iad-06.braze.com
[17:05:20] https://github.com/Appboy/appboy-react-sdk/blob/1d10e3aa13b33897b5fcf18aaaa30b14eaae203d/__tests__/index.test.js
[17:05:20] cdn-staging.braze.com
[17:05:22] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=3
20
20
[17:05:24] https://github.com/Appboy/braze-docs/blob/42f795005188ad2d125cb42751b2a1a5e1a92b19/_docs/_user_guide/administrative/access_braze/braze_instances.md
[17:05:24] dashboard.braze.com
[17:05:24] dashboard-02.braze.com
[17:05:24] https://github.com/Appboy/braze-docs/blob/42f795005188ad2d125cb42751b2a1a5e1a92b19/_docs/_user_guide/administrative/access_braze/getting_your_account.md
[17:05:24] alerts.braze.com
[17:05:24] dashboard-04.braze.com
[17:05:24] dashboard-06.braze.com
[17:05:24] dashboard-08.braze.com
[17:05:24] rest.iad-08.braze.com
[17:05:25] https://github.com/Appboy/braze-docs/blob/3e8654891c048bcc28703d0d2a600a1a9f1274b7/_docs/_partners/data_and_infrastructure_agility/customer_data_platform/mparticle.md
[17:05:25] sdk.api.braze.com
[17:05:25] rest.iad.braze.com
[17:05:25] https://github.com/AdroitAdorKhan/EnergizedProtection/blob/2ee328e1c8ae1c7a8ae9362884f5e9d7abdc9a99/core/hosts-xtreme
[17:05:25] learning.braze.com
[17:05:28] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=4
20
20
[17:05:30] https://github.com/Appboy/braze-docs/blob/8bade1b6c41eeaaa5de748b0764f51d62f1f46cf/_docs/_api/endpoints/user_data/post_user_track.md
[17:05:30] status.braze.com
[17:05:30] https://github.com/And96/AdBlock-Additional-Filters/blob/296f2cbd0a07af2b765191c17f786ee51a1923c9/filters/whitelist.txt
[17:05:30] www.dashboard-03.braze.com
[17:05:31] https://github.com/Appboy/braze-docs/blob/739898b85d7f587fb1ece4c811fccd81a3f249fd/_docs/_user_guide/message_building_by_channel/email/email_private_relay_apple_SSO.md
[17:05:31] abmail.docs.braze.com
[17:05:31] docs.braze.com
[17:05:33] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=5
20
20
[17:05:35] https://github.com/GrantFox22/CCPods/blob/2573fd7dc64bfb391145e8558ba9b3345b1f7ff3/6/8/4/Appboy-iOS-SDK/3.16.0/Appboy-iOS-SDK.podspec.json
[17:05:35] documentation.braze.com
[17:05:36] https://github.com/SLED-5/QuantumultX-Resource/blob/009b94363ef7ee50290a55d5060d5d4567110235/fliter/Global.list
[17:05:36] dialacrum.iad-03.braze.com
[17:05:39] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=6
20
20
[17:05:42] https://github.com/dead-hosts/hosts_git_lightswitch05/blob/d5a7c1762b91b017c502ea2f3b1e68c1c1fe1680/output/hosts/INACTIVE/hosts
[17:05:42] alertmanager.k8s.stc-002.p-use-1.braze.com
[17:05:42] android.k8s.test-001.p-aze-us.braze.com
[17:05:42] apns.k8.region-001.p-aze-us.braze.com
[17:05:42] apns.k8s.region-001.s-euc-1.braze.com
[17:05:42] apple.k8s.test-001.d-usw-2.braze.com
[17:05:42] blue.k8s.test-001.d-usw-2.braze.com
[17:05:42] churn-pangolin.k8s.region-001.p-euc-1.braze.com
[17:05:42] churn-pangolin.k8s.region-001.s-euc-1.braze.com
[17:05:42] churn-pangolin.p-euc-1.braze.com
[17:05:42] churn-pangolin.s-euc-1.braze.com
[17:05:42] data-lake-connect.k8s.region-001.p-aze-us.braze.com
[17:05:42] example-ruby-service.k8s.test-001.d-usw-2.braze.com
[17:05:42] grafana.k8s.stc-002.p-use-1.braze.com
[17:05:42] grape.k8s.test-001.d-usw-2.braze.com
[17:05:42] iad-05.braze.com
[17:05:42] k8s.cluster-001.p-use-1.braze.com
[17:05:42] k8s.cluster-001.s-use-1.braze.com
[17:05:42] k8s.infra-001.i-use-1.braze.com
[17:05:42] k8s.region-001.p-use-1.braze.com
[17:05:42] k8s.region-001.s-use-1.braze.com
[17:05:42] k8s-sondheim.braze.com
[17:05:42] k8s.stc-001.p-use-1.braze.com
[17:05:42] k8s.stc-001.s-use-1.braze.com
[17:05:42] k8s.stc-002.p-use-1.braze.com
[17:05:42] k8s-sweeney.braze.com
[17:05:42] k8s.test-001.s-usw-2.braze.com
[17:05:42] k8s.test-002.d-usw-2.braze.com
[17:05:42] k8s.test-002.s-usw-2.braze.com
[17:05:42] k8s-todd.braze.com
[17:05:42] kubecost.k8s.stc-002.p-use-1.braze.com
[17:05:42] labpages.braze.com
[17:05:42] logstash-dashboard.k8s.test-001.d-usw-2.braze.com
[17:05:42] love-is-blind-service.k8s.test-001.d-usw-2.braze.com
[17:05:42] mx.braze.com
[17:05:42] observer.k8s.test-001.d-usw-2.braze.com
[17:05:42] orange.k8s.test-001.d-usw-2.braze.com
[17:05:42] pink.k8s.test-001.d-usw-2.braze.com
[17:05:42] platform-api.k8s.test-001.d-usw-2.braze.com
[17:05:42] platform-dashboard.k8s.test-001.d-usw-2.braze.com
[17:05:42] prometheus.k8s.stc-002.p-use-1.braze.com
[17:05:42] services.iad.braze.com
[17:05:42] test.k8s.test-002.d-usw-2.braze.com
[17:05:42] tiger-king-service.k8s.test-001.d-usw-2.braze.com
[17:05:42] vm.k8s.test-001.d-aze-us.braze.com
[17:05:42] webhook.k8s.region-001.d-usw-2.braze.com
[17:05:42] webpush.k8s.region-001.p-euc-1.braze.com
[17:05:42] white.k8s.test-001.d-usw-2.braze.com
[17:05:42] www.lab.braze.com
[17:05:42] www.partnerlab.braze.com
[17:05:43] https://github.com/Appboy/braze-growth-shares-braze-to-bigquery/blob/8c9d0ed2daba4908c3e5ebc8245482db9898094f/README.md
[17:05:43] rest.braze.com
[17:05:46] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=7
20
20
[17:05:51] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=8
20
20
[17:05:57] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=9
20
20
[17:05:59] https://github.com/Appboy/braze-docs/blob/e7e9eddc6dec28f4509fd030647169488d96e79e/_docs/_user_guide/engagement_tools/campaigns/scheduling_and_organizing/campaign_alerts.md
[17:05:59] sweeney.braze.com
[17:06:00] https://github.com/krbi/pihole/blob/1f08bb56bd01bc1f35fc89bc876960043ba20d67/data/ads-and-tracking.txt
[17:06:00] horus.iad-03.braze.com
[17:06:00] iad-04.braze.com
[17:06:00] iad-08.braze.com
[17:06:03] https://api.github.com/search/code?per_page=100&sort=indexed&order=desc&q=%22braze.com%22&page=10
[17:06:03] token limit reached, token removed from the list
[17:06:03] tokens limit reached, no more token available, exiting...
gwen001 commented 4 years ago

Hi,

Sorry for being late, holidays applied...

I was able to reproduce the problem. The following error is triggered: "Abuse rate limits" https://developer.github.com/v3/#abuse-rate-limits rl

It seems to be a "variation" of the token limit (30 requests/mn). It doesn't occur when you make a "normal" use of the script but when you relaunch it several times in a row.

I increased the fixed delay between requests a little bit (100ms -> 200ms). Pull the script and try again. For me, the best option is still to use several tokens but I don't know if GitHub also check your IP for abuse.

giany commented 4 years ago

Thanks, will test it out.

gwen001 commented 4 years ago

I added an option to make the program looping when all tokens are rate limited. Disable this feature is possible by using the option -k