search

kevinelliott / agent_orange

Parse and process User Agents like a secret one
126 stars 36 forks source link

Undetected User Agents #3

Closed davidjrice closed 11 years ago

davidjrice commented 12 years ago

Hey Kevin,

Have run agent_orange over a sample of user_agents. Failed on only 46 of around 12,000. Not bad!

Here's a list of the failed ones.

1PasswordThumbs/1 CFNetwork/454.11.12 Darwin/10.7.0 (i386) (MacBookPro7%2C1)    
Aboundex/0.2 (http://www.aboundex.com/crawler/) 
Anemone/0.6.0   
Baiduspider+(+http://www.baidu.com/search/spider.htm)   
cmsworldmap.com 
Covario-IDS/1.0 (Covario; http://www.covario.com/ids; support at covario dot com)   
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)   
Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 1 subscribers; feed-id=7611448681530808886)   
Feedfetcher-Google; (+http://www.google.com/feedfetcher.html; 3 subscribers; feed-id=8646581579435208956)   
Googlebot-Image/1.0 
Java/1.4.1_04   
Java/1.6.0_04   
Java/1.6.0_16   
Java/1.6.0_20   
Java/1.6.0_27   
libwww-perl/6.02    
linkdex.com/v2.0    
MetaURI API/2.0 +metauri.com    
Microsoft URL Control - 6.00.8169   
Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html) 
Mozilla/5.0 (compatible; Embedly/0.2; +http://support.embed.ly/)    
Mozilla/5.0 (compatible; ScoutJet; +http://www.scoutjet.com/)   
Mozilla/5.0 (compatible; Yahoo! Slurp; http://help.yahoo.com/help/us/ysearch/slurp) 
percbotspider   
Plesk   
PostRank/2.0 (postrank.com; 1 subscribers)  
R6_CommentReader(www.radian6.com/crawler)   
radian6_default_(www.radian6.com/crawler)   
Reeder/1.5.1 CFNetwork/485.12.7 Darwin/10.4.0   
Reeder/1.5.1 CFNetwork/485.13.9 Darwin/11.0.0   
Reeder/1.5.1 CFNetwork/544 Darwin/11.0.0    
Reeder/1.5.1 CFNetwork/547 Darwin/11.0.0    
Reeder/1010.19.00 CFNetwork/520.0.13 Darwin/11.1.0 (i386) (MacBook4%2C1)    
Reeder/1010.29.00 CFNetwork/520.0.13 Darwin/11.0.0 (x86_64) (MacBookPro5%2C3)   
Reeder/1010.29.00 CFNetwork/520.0.13 Darwin/11.0.0 (x86_64) (MacBookPro6%2C2)   
Reeder/1010.29.00 CFNetwork/520.0.13 Darwin/11.1.0 (i386) (MacBookPro6%2C2) 
Reeder/1010.29.00 CFNetwork/520.0.13 Darwin/11.1.0 (x86_64) (iMac11%2C1)    
Reeder/1010.29.00 CFNetwork/520.0.13 Darwin/11.1.0 (x86_64) (MacBook5%2C1)  
Reeder/1010.29.00 CFNetwork/520.0.13 Darwin/11.1.0 (x86_64) (MacBookAir3%2C1)   
Reeder/1010.29.00 CFNetwork/520.0.13 Darwin/11.1.0 (x86_64) (MacBookPro5%2C3)   
Reeder/2.5.1 CFNetwork/485.13.9 Darwin/11.0.0   
Reeder/2.5.1 CFNetwork/547 Darwin/11.0.0    
Ruby    
SBIder/Nutch-1.0-dev (http://www.sitesell.com/sbider.html)  
ThumbnailService/39001 CFNetwork/520.0.13 Darwin/11.1.0 (x86_64) (MacBookPro5%2C2)  
xpymep.exe
kevinelliott commented 12 years ago

Thanks David! I've merged in your changes. I will close this issue once I merge in another request that contains fixes to detection.