google / recaptcha

PHP client library for reCAPTCHA, a free service to protect your website from spam and abuse.
http://www.google.com/recaptcha/
BSD 3-Clause "New" or "Revised" License
3.47k stars 767 forks source link

Recaptcha v3 always returns a 0.1 score #248

Closed SeriaWei closed 1 year ago

SeriaWei commented 5 years ago

When i use recaptcha v3 with host "www.recaptcha.net" the score is always 0.1, but when i change to use host "www.google.com", the score is 0.9. So is it the v3 can not use with www.recaptcha.net? Scripts

https://www.recaptcha.net/recaptcha/api.js?render={key}

Siteverify

https://www.recaptcha.net/recaptcha/api/siteverify
jasonfedor commented 5 years ago

Thanks for the report - not able to reproduce this, do you have a sample page you ran this? What happened when you mixed and matched siteverify and api.js locations?

SeriaWei commented 5 years ago

No, it was just localhost. I will push it to live to test later. Thanks

SeriaWei commented 5 years ago

Push to live is the same, the score is always 0.1. This is my test keys:

Site key

6Lc[redacted]Kdz

Secret key

[redacted]

With "www.recaptcha.net"

image

Change to "www.google.com"

image

But when i change to use another key, it works perfect with both "www.recaptcha.net" and "www.google.com"

rowan-m commented 5 years ago

Please do not post your secret key in bug reports. I've removed it from your comment, but please also go to https://www.google.com/recaptcha/admin#list to create a new key.

SeriaWei commented 5 years ago

@rowan-m Thanks! But it was just a testing key. :)

SeriaWei commented 5 years ago

After change the key, it works at beginning. But i found that the score would continue to drop from 0.9 until 0.1, and then it would have been 0.1.

amarnathm commented 5 years ago

I'm also getting 0.1. But it's on localhost.

bankeris commented 5 years ago

Hello, I'm getting always score 0.1 on developer PC only. No matter it's localhost or normal address, it's always 0.1. If i try from other pc/mobile/tablet works like a charm It's a bit pain to test then dev pc is always "bot" 🤕

rowan-m commented 5 years ago

If you are running automated tests in a staging/development environment, I would suggest creating a mock service for reCAPTCHA. I updated the various RequestMethod classes to accept a $siteVerifyUrl parameter meaning you can override the default location. Alternatively you could also create a RequestMethod class that just returns a hardcoded response.

marcotibben commented 5 years ago

Sort of same problem here. Programmed ReCaptcha v3 into my own build multitenant framework and started implementing ReCaptcha for websites.. yesterday it all worked fine; 0.9 for me typing and filling contact form, etc.. But today I suddenly, and always, get a 0.1 score.. even when using my iPhone instead of my laptop (same wifi IP), or disabling the wifi.. (mobile provider IP) and this is when clearly showing i'm human by extensively using the mouse and navigating the website.. typing in the forms instead of using the browsers autofill, but nothing helps getting the score up..

No, I didn't change anything in the code between yesterday and today.. And it worked perfectly yesterday.. so something happened overnight at google? (additional scoring or tests or something?)

One other thing I noticed yesterday; when there's a login form at the startpage, and one hasn't got enough navigating on the website (because the only thing to do is fill in the login form), and the browser uses autofill, the score also always returns 0.1.. so really not workable I would say..

I'm clearly human and filling in forms by hand, moving mouse, etc, but still I get a 0.1 ?!?

update; I normally use Edge, but just tried with Chrome to no avail.. still the same result.. 0.1 score..

update; just used a phone from another person; works fine.. so I suggest putting some kind of feature in the admin screen to 'unblock/reset' a certain visiting ip address score if needed.. so when a costumer comes to me with the message 'I have someone who cant use my contact form on the website', I have the possibility to 'reset' the build up score of a website visitor.. because really.. if this happens to legit customers of my costumers than I have a big problem.. and meanwhile I cant do anything now! i'm implementing ReCaptcha in all my costumers websites, and cant do anything at the moment really being a bot who isn't a bot.. :( So I do not feel confident implementing this ReCaptcha v3 solution is the right thing to do at the moment..

ykcai commented 5 years ago

I am seeing this when I try on Safari, but Chrome returns 0.9. Also testing on localhost. Could localhost be the problem here?

marcotibben commented 5 years ago

I've never used localhost; so for me that's not the problem.. it feels more like google is putting some data together overnight and does some calculations, and after those decides that i'm an abuser filling in those forms with a same kind of message (for my costumers, saying ReCaptcha is activated on their website)..

I've deleted the key and created a new key, and now i'm a 0.9 again..

rowan-m commented 5 years ago

Scores you see returned on development machines are not reflective of production. If you need a predictable score returned in a given environment, then you should absolutely mock the service to return a configurable score. To determine where to set appropriate thresholds to act on the score, you need to look at your dashboard in the admin interface for your site and see the spread of scores being returned.

The score returned is also only an indicator of confidence in that request. If it's below your threshold then you need to determine the appropriate response. For example, if it's just viewing a product page maybe you want to rate limit the user but if it's something like completing a purchase maybe you want to add an SMS verification before letting the user continue.

marcotibben commented 5 years ago

Well its happening again. 2 weeks in production and suddenly ReCaptcha keeps returning 0.1 for most of the requests even genuine ones. Customers are complaining. Apparently the machine learning is not good enough yet. I cannot keep creating new keys, just to keep the ReCaptcha performance up to scratch. V3 doesn't seem ready yet. Also I use actions, but most of the sites do not pick the actions up in the admin panel. Some do, but then just one action of the 4 I use.. This is not usable sadly.. It looked very promising, but its not delivering at the moment.

jasonfedor commented 5 years ago

Marco, please send your public site key and a url on which you're experiencing trouble to support@recaptcha.net so we can take a look.

crmpicco commented 5 years ago

I have implemented reCAPTCHA v3 in our application and we are looking to move this into production soon. I am, however, concerned about some of the scores it returns and a handful of the comments I have read online about it. The comment above from @marcotibben in particular:

One of the points I am concerned with is the ability to test failures. There doesn’t appear to be a lot of advice in the official docs on how to test this accurately, so what I done was change my User Agent to “Googlebot” and then call the reCAPTCHA API. Initially this returned a failure with a score of 0.1, which is below our pre-defined threshold of 0.5 and would denote bot behaviour. However, now when I do this it returns a score of 0.9 and sometimes 0.999999999. This leads me to believe that Google is in some way whitelisting our staging server IP and returning an acceptable score regardless of what we do.

It would obviously be bad for bots to get through, but I’m more concerned about valid users having a score of 0.1 which was reported on this Github issue a day ago.

Can you advise on recommended testing procedures and how we can go about getting some confidence in this before we think about deploying this to production?

marcotibben commented 5 years ago

it was bad for a couple of days, but it seems to have restored now and results are good again. Maybe it's part of the machine learning process?!?

I've made changes to my application in that I don't solely rely on recaptcha's opinion anymore.. bad score resulted in not sending e-mail from contact forms. Now I take the opinion of ReCaptcha in advice and thén decide on some other factors in the message if it's spam or not.. and I log all contact, replies, and failures from recaptcha in my database, so learn where it works and doesn't to improve my application.. I was hoping google would do all the work ;)

I sure hope ReCaptcha gets more reliable, so I can fully rely on it again, because the idea is great..

mostafaznv commented 5 years ago

I have this problem too. recaptcha returns always 0.1 on chrome. but everything is ok in firefox. I tested it a lot in chrome and I think it returns 0.1 for this reason.

But the main question is this: if recaptcha analysis my actions and blah blah, why he thinks I'm a robot?

doekia commented 5 years ago

Seems the same problem still occur. How can you could deliver to public release with such major issue ?

justid commented 5 years ago

I can confirm this issue which the score keeps on 0.1 even user didn't visit our page before. After regen the sitekey it return score 0.9, that's ok,but i don't think keep changing the sitekey is a good idea. Any advice will be appreciated

crmpicco commented 5 years ago

I'd be quite keen to read some reviews or blog posts on someone who has successfully integrated reCAPTCHA v3 and have released it to production with a positive outcome. It does appear that people are reluctant to deploy this into a production environment at the moment, certainly, I am one of those people.

justid commented 5 years ago

I will using reCAPTCHA v3 into our production environment, and after reading the doc repeatedly,I found these text maybe important:

reCAPTCHA works best when it has the most context about interactions with your site, which comes from seeing both legitimate and abusive behavior

And I will follow this lead in code, I will update the progress as soon as possible, our production has very large network flow and i think it's good for recaptcha's analytic model

neha425 commented 5 years ago

can anyone help me for adding recpatcha v3

andrebruton commented 5 years ago

I feel like going back to V2 if the V3 issue is not resolved for localhost. It's frustrating to test and fix if there is suddenly a low score. At least with V2 you can make sure you have a high score.

maxex0000 commented 5 years ago

I have the same issue in my site. After a dramatic climbing of suspicious rate someday, most results become 0.1.

Is there a mechanism to train the model by site or by key? If yes, maybe the unsupervised learning mechanism needs to be improved.

chxj1992 commented 5 years ago

+1 , I load recaptcha v3 from a hidden plain web page in my iOS App, and the score drop from 0.9 until 0.1. Any idea to solve my problem ? 🤔

gingerlime commented 5 years ago

bumped into this today as well. We switched to www.recaptcha.net recently. One of our employees was getting 0.1 scores, even though she was logged-in on Chrome, no incognito or any crazy browser extensions etc... Weird.

CodemasterJamal commented 5 years ago

I'm having the same issue. I'm trying to test a code on my server and google keeps returning a low score, saying that I'm a bot. I really like v3 but, if I have to, I'll have to use v2.

mennokreukniet commented 5 years ago

I'm having this issue as well. May be a little different I'm not sure.

When I log in using autofill on my form, google always gives me a 0.1 score. If I wait about 10 seconds after autofill I always get a 0.7 score. If I don't wait, but change something in the autofilled field, I get a 0.7 score as well.

I assume this isn't supposed to happen though? Many people would use autofill to login..

ltctech commented 5 years ago

One page on my site nearly always returns 0.1 for me. Another page returns 0.7 or 0.9. I am at a loss.

FriendOfTheNight commented 5 years ago

Same issue here. A few months into production and my client's customers are complaining about the failed captcha check. Legitimate human actions are getting a score of 0.1 and blocking users from doing anything (I was blocking anything under 0.3). I had to reduce it to only block a score of 0 for now which I fear may let some bots through. It seems this should automatically give the user a second chance with an "I'm not a Robot" checkbox if there are going to be so many false positives...

chenxinlong commented 5 years ago

Similar problem here. When I specify the action parameter as homepage, I always got a 0.9 score with chrome, but under safari, it's a 0.1.

CodemasterJamal commented 5 years ago

Hey guys, I kinda solved this issue on stackoverflow (a question in which I asked my self and then answered it on my own). It took a while but, depending on what type of method you are using the recaptcha works well and is relatively easy to use. If you need any assistance feel free to either message me here or contact me on Stackoverflow or Instagram. I'm known as Codemaster Jamal. You can honestly just google that to find me.

My problem was that I had two tags with the same id. Once, I changed the id's and made sure to rewrite the javascript to post to the second-tag, I was fine.

marcotibben commented 5 years ago

you didnt kinda solve it.. not using any same id tags here...

CodemasterJamal commented 5 years ago

you didnt kinda solve it.. not using any same id tags here...

You will need to supply some coding examples if you need help. Also, I'll overlook this entire topic and then respond to them the best I can. You also need to specific what type of languages you are using and how you are implementing this method. I use a PHP and Javascript method for my website. What languages do you use?

marcotibben commented 5 years ago

the recaptcha code the end user uses in his browser is always javascript.. so that doesn't really matter now does it.. and if you had read entire topic you would also have read it works for two weeks, and then suddenly it doesn't, and then it does again.. and then doesn't.. without changing any code.. so the problem more seems to be an unreliable recaptcha solution, than any form of misuse of the code.. if it was as easy as using double id's there wouldn't be a problem..

CodemasterJamal commented 5 years ago

the recaptcha code the end user uses in his browser is always javascript.. so that doesn't really matter now does it.. and if you had read entire topic you would also have read it works for two weeks, and then suddenly it doesn't, and then it does again.. and then doesn't.. without changing any code.. so the problem more seems to be an unreliable recaptcha solution, than any form of misuse of the code.. if it was as easy as using double id's there wouldn't be a problem..

https://stackoverflow.com/questions/55521795/google-recaptcha-v3-returns-success-as-false

I read this topic 2 months ago when I really needed help with my recaptcha. There's an easier way to apply recaptcha with php and javascript. Did you ever think of just rewriting the code? Or changing the format of the code? Perhaps think of a different method to get recaptcha to work because obviously, your first method isn't. I mean, you're a programmer. Problem solve. You're not going to be able to google everything, a lot of this stuff is testing things and figuring it out on your own. Just have to figure out what works best.

With that aside, there has to be a reasonable solution as to why the code is doing that. Been using recatpcha for months and it works fine. When was the last time you tested your code?

jjxtra commented 5 years ago

Seeing this as of today. Code has not changed in months, but suddently 0.1 score always. What is going on?

quicirman commented 5 years ago

A couple of days ago, I implanted reCAPTCHA V3 to my website and I am in tests. All the tests I made from production, I do not use localhost. The key validations are correct, the JSON response is correct (success = true), but I am having problems with the score. The tests are through a contact form, so I put action = contact. I fill in all the fields of the form by hand, check the box of Privacy Policy and submit. It does not matter if I use Chrome, Firefox or Opera, it always gives me score = 0.1, so I can not validate those cases where score <0.5 is a bot, because nobody would pass the filter. I'm afraid I'm going to have to use V2, since I do not know how to solve this problem :(

doekia commented 5 years ago

All the tests I have done proves V3 is totally unreliable, not to mention it requires a complete revision of your business logic in many cases. You receive score varying from .9 to .1 over the same connection, same user agent, ... When you want to protect a form that requires end-user input it is even worst. You receive timeout - sorry end-user may takes more than 1mn to fill-up their name, address, phone, ... Implementing some cleaver logic to address such makes it spammer prone. Some browser are definitly unsupported, the code is negatively impacting performances (loading various fonts, and other third party resources. This code is useless, support non-existent, documentation fuzzy when not misleading...

davit555 commented 4 years ago

Anyu updates here ?

splpa commented 4 years ago

I agree with @doekia what a mess. We like the idea of not needing user input, but consistency/reliability is a must obviously. And implementing this has been confusing for sure, receiving errors with practically no guidance on how to resolve, now scores are unreliable, feeling very frustrated.

crmpicco commented 4 years ago

The comments on here seem like it's RIP to reCAPTCHA v3. This issue was opened a year ago and it hasn't been resolved yet. đź’€

kosmeln commented 4 years ago

We are experiencing the same issue in production. The automated traffic from the malicious sources (doing constant unlimited request from one IP) got 0.9 scores, meaning, reCaptcha v3 is USELESS. There is no way to at least lower the score if the bot fraud traffic gets 0.9 all the time. What a mess Google!

andrebruton commented 4 years ago

It looks like the solution is to either only use Ver 2, or use Ver 3 on initial page and then use Ver 2 on any other pages where Ver 3 does not work. I thought it would do this automatically, but clearly not.

mostafaznv commented 4 years ago

many people are talking about this problem. but there is no reaction from recaptcha team. what a support! thanks

jjxtra commented 4 years ago

Would be nice to have an official Google response...

crmpicco commented 4 years ago

im out

nuenuewei commented 4 years ago

Why would Google respond? You're not the customer, you are an advertising/data product they sell to their customers.

superern commented 4 years ago

since this issue is still open. is there any other alternatives we can use for the meantime?