Validation prior to signup to reduce spammers

[btbonval]

For email validation we'd want a list (call it a table) that tracks the email addresses and sign up keys for each person who tried to sign up. we want to ensure anyone trying to sign up isn't already signed up (email address found in rusers). If the email address already exists in this list, we'll simply resend the old sign up key.

Once a user clicks over to signup using the key, the email address for the user and the signup key (via URL) must match what's in the signup list. If so, the new user is created, the signup key and email are removed from the list, and we have a new winner!

So it'd be a two step process for the user:

1. type in email address, wait for email with URL
2. click URL, sign up as normal.

[btbonval]

reset key shows how to generate a key and validate against it: https://github.com/jywarren/plots2/blob/5c36a9887f91d098f82afef81f44b8d2b565fbdf/app/controllers/users_controller.rb#L127

user creation needs to be modified to use a similar randomly generated key, indexed by email address: https://github.com/jywarren/plots2/blob/5c36a9887f91d098f82afef81f44b8d2b565fbdf/app/controllers/users_controller.rb#L9

the table will need a migration to add it into the database: https://github.com/jywarren/plots2/blob/7931f6c369a47b69df55eb945ab5ea27df897132/db/migrate/20130402232059_create_like_and_follow.rb

The table schema will be something like:

    create_table :newuserkeys, :id => false do |t|
      t.string :key
      t.string :email
    end
    add_index :newuserkeys, :key, :unique => true
    add_index :newuserkeys, :email, :unique => true

By making key and email independently unique, we guarantee each email address may only have one key (as each email address may appear exactly one time) and we guarantee that each key has one email address (as each key may only appear once).

The code to generate keys will probably want to check the database to see if the email address and/or random key are already in there. In SQL this would be like "SELECT * FROM newuserkeys WHERE key = 'thekey' OR email = 'emailaddress';". Two rows mean neither is unique, one row would need to check which was not unique and determine how to proceed (if email: bail, if key: regen key and try again)

[btbonval]

When checking the email address against the table above, the email address should also be checked against users / rusers (still not sure which one is considered authoritative) to see if the user is already registered.

[btbonval]

Working on a human test using 4 questions with 2 choices. All 4 must be chosen correctly, which carries a 1/16 chance of being randomly guessed. In theory, this will reduce the automated spam users by an order of magnitude. http://pad.publiclab.org/p/WQXq8OQ030

Additionally, the Ruby on Rails security guide had a neat idea for honey pots, although these do not appear to be pre-implemented. http://guides.rubyonrails.org/security.html

Most bots are really dumb, they crawl the web and put their spam into every form's field they can find. Negative CAPTCHAs take advantage of that and include a "honeypot" field in the form which will be hidden from the human user by CSS or JavaScript.

The most simple negative CAPTCHA is one hidden honeypot field. On the server side, you will check the value of the field: If it contains any text, it must be a bot. 

[btbonval]

So far there are 6 statements for humans and 6 statements for robots. Each set will be randomly permuted using the Array.permutation() feature picking only 4 from the 6. The submitted value for each should be some kind of CSRF-like craziness, but the display value should simply be the statement.

Regarding the negative CAPTCHA, there is no reason to hide it. The label for the text field should say "Prove you can follow instructions by leaving this text field blank." If anything appears in the text field it's either a spambot or someone who cannot read/follow directions.

[btbonval]

In the @user form, need to send some things over to the @spamaway model. http://stackoverflow.com/questions/892624/how-to-handle-multiple-models-in-a-rails-form ^ Super handy!

Also need to create a Spam validator that ensures the honeypot is blank and that the four questions are contained properly. This requires custom validation. Quick examples of how to do that here: http://jonathanhui.com/ruby-rails-3-model-data-validation

[btbonval]

4 questions and honeypot seem to work, but when the spam validation model fails, nothing is highlighted, not even proper user form errors.

The formatting of the /register page messes with radio buttons while the /signup page renders the same thing fine. There was an additional problem written into #85 , but it was fixed by adding @action = create. The alternate rendering is still a problem.

The user profile/bio is not drawn in on creation. It may be added from editing the profile after creation, but otherwise that blurb is completely lost in this branch.

Besides the highlighting (meh) and missing profile, it appears to work!!

[btbonval]

Check to see if the aforementioned two problems exist in the master branch. If so, anti-spam will be merged into master. If not, find out why these changes broked things.

[btbonval]

On master branch, the /signup page will forward to /register with error highlighting done correctly. The /register page will forward to /register with error highlighting done correctly as well. So it looks like the spamaway branch has added some problem here.

bio profile, however, fails to work. That problem is not caused by the spamaway branch: https://github.com/publiclab/plots2/issues/14

[btbonval]

weird. so I'm testing problems and things look okay now.

On /signup and /register, putting in an invalid email address forwarded to /register with a proper error about email address. So that's good!

Password mismatch errors properly with highlights.

Both password and invalid email highlight together just fine.

No error highlighting or comments if you fail the Turing test. None at all, in fact. That is the crux of the problem. Fail any of the Turing test and the other errors don't appear.

[btbonval]

ah. @spamaway.valid? checked and if it fails, @user.valid is not checked at all. So maybe toss in a custom error message about "No robots allowed" and also try to run @user.valid? to see if that will add some errors for output. It looks like errors are generated by a failed @user.save right now, but I think save calls valid.

[btbonval]

sweet. adding @user.valid? added those errors. The form errors display only errors from @user. Need to add @spamaway errors to the @user errors. This page seems relevant: http://guides.rubyonrails.org/v2.3.11/activerecord_validations_callbacks.html#working-with-validation-errors

[btbonval]

seems like errors changed from 2.3 to 3.1, so ... not sure where to look. I guess rails console since the docs aren't helping.

[btbonval]

no idea where this errors stuff is coming from. it was all outmoded after 2.3.8. there was a note about nifty-generators adding some methods which were deprecated, but nothin doin for docs on that regarding errors.

accessing the errors has been resolved. @spamaway.errors.full_messages.each.

If this had been the 2.3 documented version, @user.errors.add_to_base, but that isn't a thing here. Errors has a bunch of add methods, but without docs, I have no idea how to use them or why they're different. :add, :add_on_empty, :add_on_blank, :added?. Need to figure out how to add a verbatim message.

[btbonval]

This is taking too long and I don't care anymore. Just going to fake an attribute and hope it doesn't bite me later.

[jywarren]

Sorry I missed these, Bryan, i was away for the weekend. Error handling is done in the models, using the before_save, after_create, etc. filter functions, and a bunch of validates_foo_of statements, like here:

https://github.com/jywarren/plots2/blob/master/app/models/drupal_node.rb#L45 (although this uses a custom validator which is a little atypical, but do show how to add messages)

here's more examples:

https://github.com/jywarren/plots2/blob/master/app/models/user.rb#L22

https://github.com/jywarren/plots2/blob/master/app/models/image.rb#L16

[btbonval]

The problem is not validation itself. The problem is how to access/modify the errors returned by validations.

We have two models being validated: user and spamaway. However, the automagic Rails error rendering only shows the user validation errors. I need to shoehorn the spamaway errors into the user errors so that they display.

The biggest problem is that we are running Rails 3, but we are not making use of Rails 3 error handling. For example, our models come back with an errors attribute, which was not a thing since Rails 2.x. In fact, in Rails 3, the erroring fields are supposed to be highlighted and have messages right on them. Some gem is adding our current form validation error handling. I think I figured out which, but the gem had zero docs about the error attribute and how to muck with it.

Here's what I'm working with: https://github.com/jywarren/plots2/blob/51c28f2424535bcaeb0d7a9aa9a6187ab1f8d4cb/app/controllers/users_controller.rb#L38-L41

[btbonval]

What I'd really love to use is errors.add_to_base documented for Rails 2.x: http://guides.rubyonrails.org/v2.3.11/activerecord_validations_callbacks.html#working-with-validation-errors

Whatever gem gives us the errors attribute on a model resulting from form validation failure does not make use of add_to_base.

[btbonval]

Oh the closest I could come up with is the nifty generators gem. Someone had mentioned it as a way to do Rails 2.x style error management in Rails 3. Couldn't find any error handling docs for nifty generators, besides some templating stuff.

[jywarren]

Well, the first example i linked to shows a custom validator with custom messages: https://github.com/jywarren/plots2/blob/master/app/models/drupal_node.rb#L3

Is that helpful?

On Mon, Dec 16, 2013 at 3:31 PM, Bryan Bonvallet notifications@github.comwrote:

Oh the closest I could come up with is the nifty generators gem. Someone had mentioned it as a way to do Rails 2.x style error management in Rails

3. Couldn't find any error handling docs for nifty generators, besides some templating stuff.

— Reply to this email directly or view it on GitHubhttps://github.com/jywarren/plots2/issues/195#issuecomment-30697434 .

[btbonval]

Oooo. I think I need to use the base symbol. That should do the trick. On Dec 18, 2013 2:14 PM, "Jeffrey Warren" notifications@github.com wrote:

Well, the first example i linked to shows a custom validator with custom messages: https://github.com/jywarren/plots2/blob/master/app/models/drupal_node.rb#L3

Is that helpful?

On Mon, Dec 16, 2013 at 3:31 PM, Bryan Bonvallet notifications@github.comwrote:

Oh the closest I could come up with is the nifty generators gem. Someone had mentioned it as a way to do Rails 2.x style error management in Rails

3. Couldn't find any error handling docs for nifty generators, besides some templating stuff.

— Reply to this email directly or view it on GitHub< https://github.com/jywarren/plots2/issues/195#issuecomment-30697434> .

— Reply to this email directly or view it on GitHubhttps://github.com/jywarren/plots2/issues/195#issuecomment-30871114 .