Not receiving recommendations

[bhardin]

Whenever I call, current_user.recommendations, I always receive a blank list.

[davidcelis]

Can you verify that your current_user has liked or disliked at least one thing? If current_user hasn't rated anything, they'll get no recommendations. Likewise, if there are no other users in the system (or nobody has rated anything yet), nobody will have recommendations.

[bhardin]

Yes, there are two users on the system. Both of which have likes (current_user.liked) and dislikes (current_user.dislikes).

I am running on heroku, and I am curious about the command

QUEUE=recommendable rake environment resque:work

How can I verify that heroku is spinning up recommendable?

[davidcelis]

Heroku uses Redis To Go. Unless you set up your Redis instance as per their instructions, there's essentially no Redis connection: http://blog.redistogo.com/2010/07/26/resque-with-redis-to-go/

[bhardin]

I've setup RedisToGo and have it configured. I didn't know how to check the status of recommendable to see if there is anything in the queue it is trying to consume.

[davidcelis]

Could you gist your recommendable initializer?

[bhardin]

https://gist.github.com/1821282

The REDIS_TOGO Env variable is setup on heroku using:

heroku config:add REDISTOGO_URL='URL'

with the URL being what I get from the redistogo details.

[davidcelis]

What is the result of

$ heroku console Recommendable.redis.inspect
$ heroku console Resque.inspect

[bhardin]

$ heroku console Recommendable.redis.inspect
"#<Redis client v2.2.2 connected to redis://chubb.redistogo.com:9156/0 (Redis v2.4.6)>"

$ heroku console Resque.inspect
"Resque Client connected to redis://chubb.redistogo.com:9156/0"

[davidcelis]

Check the queue size:

$ heroku console
>> Resque.size(:recommendable)

[bhardin]

$ heroku console
Ruby console for myapp.heroku.com
>> Resque.size(:recommendable)
=> 7

Looks like they are queued, but not being consumed.

[davidcelis]

Hot. My guess is it may have something to do with Heroku's naming of their rake tasks. The link I pasted above had this code block:

require 'resque/tasks'

task "resque:setup" => :environment do
  ENV['QUEUE'] = '*'
end

desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "resque:work"

It looks like you have to do that alias. This block should go in lib/tasks/resque.rake

[bhardin]

Edited lib/tasks/resque.rake with that code block. But doesn't looks like it is working.

$ heroku console Resque.size :recommendable
7

[bhardin]

So out of curiousity I attempted to run

$ heroku rake resque:work QUEUE=*

which is in the same tutorial, but higher up. When executing to check on my queue, I am now down to 5. However, that command is obviously not returning and isn't a long term solution. Let me see if it completes and returns.

[davidcelis]

Is your Procfile configured correctly to automatically call this command upon boot? The following line should be in there:

worker: QUEUE=recommendable [bundle exec] rake resque:work

(bundle exec may or may not be needed depending on your configuration)

[bhardin]

I'm on the bamboo stack, not the cedar stack. Looks like I probably need to roll over. So I will go ahead and update. Let you know what happens after.

[bhardin]

I have rolled everything over to the Cedar stack. Still having the same issue. It looks like the queue is getting filled, but nothing is grabbing them.

I have attempted to modify my procfile from both:

worker: QUEUE=recommendable bundle exec rake resque:work

and

worker: QUEUE=recommendable rake resque:work

EDIT: Heroku definitely sees the procfile and worker... Console says the following:

-----> Discovering process types
       Procfile declares types      -> worker
       Default types for Ruby/Rails -> console, rake, web
-----> Compiled slug size is 26.9MB
-----> Launching... done, v18

[davidcelis]

what about rake environment resque:work?

[bhardin]

Upon running heroku run rake environment resque:work it emptied the queue. Can I put that command into my Procfile?

[davidcelis]

Emptied the queue? Like, your queue is just fuggin' gone (I'm so sorry), or the jobs got processed?

[bhardin]

Before executing rake enviornement reqsque:work, The Resque.size :recommendable returned 2. After issuing heroku run rake environment reque:work, the same command returned 0.

> Resque.size :recommendable
=> 0

Jobs, look processed... But it still doesn't look like I have recommendations.

[davidcelis]

Assuming your model is User, and the current_user has an ID of one, can you tell me what the following returns:

$ heroku console
>> Recommendable.redis.zcard "User:1:similarities"

[bhardin]

irb(main):015:0> Recommendable.redis.zcard "User:1:similarities"
=> 2

[davidcelis]

Recommendable.redis.zrangebyscore("User:1:similarities", -1, 1).map do |user|
  Recommendable.redis.zscore "User:1:similarities", user
end

Also, what version of Recommendable are you using?

[bhardin]

Recommendable.redis.zrangebyscore("User:1:similarities", -1, 1).map do |user|
irb(main):038:1*   Recommendable.redis.zscore "User:1:similarities", user
irb(main):039:1> end
=> ["0", "0"]

version 0.1.4

[davidcelis]

Looks like the similarity values are, indeed, not being calculated correctly. I'll look into this.

[bhardin]

Word. Thanks.

[davidcelis]

However, the recommendations themselves are still generating locally for me, so it's strange that you're getting an empty array.

[davidcelis]

Wow. Well, the similarity bug is extremely obvious now that I see it. Still unsure of your empty recommendations array, though. Let me patch this up and we'll see if that, for some reason, fixes that for you.

[davidcelis]

Update to 0.1.5 and let me know if your problem persists.

Please, please, please read the CHANGELOG, though. There's a migration you'll need to run.

Also, keep in mind that Recommendable isn't going to work well with the Nano (smallest) Redis To Go instance. You'll need, at the least, the 20 MB Mini instance. Nano is not persistent.

[bhardin]

Did Recommendable work with Nano before version 0.1.5?

[davidcelis]

Recommendable has always required a persistent Redis DB; I made sure to state this in the README at the beginning

[davidcelis]

Any word on this? Ready to close it, as this seems to be a Heroku configuration problem rather than a bug in Recommendable

[bhardin]

Go ahead and close it. I am going to take a look at it this weekend. I am pretty sure it is an issue on my end, from not using a persistent redis instance. I will update this bug so if anyone in the future comes by, they will know that's what it was. Thanks David.

On Fri, Feb 17, 2012 at 8:43 AM, David Celis reply@reply.github.com wrote:

Any word on this? Ready to close it, as this seems to be a Heroku configuration problem rather than a bug in Recommendable

---

Reply to this email directly or view it on GitHub: https://github.com/davidcelis/recommendable/issues/7#issuecomment-4023234

Brett Hardin www.sourceninja.com http://angel.co/sourceninja

P: 408.685.1569

[bhardin]

Still not getting recommendations after updating to 0.1.5. The user similarities are working, but no recommendations. If you would like to take a look at the code, check out here: https://github.com/bhardin/lunchSpot

[davidcelis]

Try replacing ENV["QUEUE"] = "*" with ActiveRecord::Base.descendants.each { |klass| klass.columns } in your lib/tasks/resque.rake task. I was having problems locally with your app but, after changing that line, the worker gets through its tasks without a hitch.

Also note (and I think this was in the changelog), when a user likes a restaurant, they'll only update their similarity values with others that liked that restaurant. My thought behind this was that their similarity values with other users wouldn't be affected. However, now that I think about it, this is sounding like a bad assumption. I think I'm going to revert that change. Would you mind pointing your app at the git repo instead of a specific version and see if that helps your problem?

gem "recommendable", :git => "git://github.com/davidcelis/recommendable.git"

[davidcelis]

Insight from a friend has shown me that Resque workers on heroku require a worker dyno to function correctly. I believe this may be the source of your issues. Unfortunately, a worker dyno on Heroku is $30/mo. Heroku, while okay for small apps that don't need anything special, kind of sucks in that it gets expensive extremely fast. That being said, Heroku does not appear to be a very viable solution for apps using redis/resque. Sorry! If you choose to stay with heroku, add a worker dyno, and still have issues, I'll be happy to keep helping.

[bhardin]

Understood regarding heroku, I was spinning down the web app, and spinning up worker apps in order to determine whether it was working. Most recently however, I was running locally and it wasn't working. That was my most recent comment was running local. You said that you got it working in my app? If that is the case, can you walk me through the process. Maybe I am missing something small.

On Mon, Feb 20, 2012 at 4:48 PM, David Celis reply@reply.github.com wrote:

Insight from a friend has shown me that Resque workers on heroku require a worker dyno to function correctly. I believe this  may be the source of your issues. Unfortunately, a worker dyno on Heroku is $30/mo. Heroku, while okay for small apps that don't need anything special, kind of sucks in that it gets expensive extremely fast. That being said, Heroku does not appear to be a very viable solution for apps using redis/resque. Sorry!

---

Reply to this email directly or view it on GitHub: https://github.com/davidcelis/recommendable/issues/7#issuecomment-4064959

Brett Hardin www.sourceninja.com http://angel.co/sourceninja

P: 408.685.1569

[davidcelis]

If you take the changes in my previous comment, you should be good to go (pointing to the git repo and that replacement in your resque.rake file). Let me know.

[paniwani]

Has anyone tried using a service like hirefire to automatically scale worker dynos? This way, you'd only pay when the worker is being used, which should dramatically reduce the cost.

[davidcelis]

Hirefire looks like a good solution to this, though I'm still going to avoid recommending Heroku because of the cost. To be honest, I don't think I'd use heroku for anything other than a small blog application. The ease of deployment ends up coming at too great of a cost, which is reduced freedom in how you run your app as well as huge premiums for bare services. I'll end up preferring a VPS like Linode 99% of the time with Capistrano set up for deployment.

[bhardin]

@davidcelis Still no dice. For some reason my app does not want to create recommendations, even locally.

[Kelend]

I'm also having an issue on Heroku, curious as to if it may be the same one. Whats your result of trying to manually generate the recommendations in the console ie:

user.send :update_recommendations

[davidcelis]

@bhardin or @Kelend, are either of your applications available as a public repo that I can clone and attempt to run locally? I can't replicate any of these problems assuming I have my redis-server and resque worker running.

@Kelend, your issue is of particular interest to me, as the Beer model I have in http://goodbre.ws/ is pretty much just as bare as yours. goodbre.ws, however, is seeing no such issue.

[Kelend]

@davidcelis to clarify, I'm having no troubles running locally. Under more testing everything seems to work perfectly for me under local conditions, but in my production environment (Heroku) the method liked_by fails.

The problem is 100% Heroku I believe at this point, it must be injecting some code somewhere that is causing collection_proxy to misbehave.

NameError: uninitialized constant Beer::

Is such a weird error, it looks like its thinking my model name is a namespace.

Unfortunately the repo client work and I can't make it public.

[davidcelis]

Ah, I thought this was a local issue, not a Heroku issue. Unfortunately, as stated before, I have not tested on nor have I endorsed Heroku because of the high costs associated with needing a worker dyno for Resque. However, if either of you would like to open a support ticket on Heroku, I would be more than happy to participate in it and offer help alongside them.

[bhardin]

@davidcelis Yes, my public repo is here: https://github.com/bhardin/lunchSpot

@Kelend I have temporarily given up on heroku until I can get recommendations working locally. Once I do, I will then put it on Heroku to see what's up.

[blackgold9]

I'm also getting no recommendations I have device 1 and device 2 (device is my user object) Device1.like Feed.first Device2.like Feed.first Device2.like Feed.last Device1.send :update_similarities Device1.send :update_recommendations Device2.send :update_similarities Device2.send :update_recommendations

Device1.recommendations still returns []

Some context i saw from above: Recommendable.redis.zrangebyscore("Device:3:similarities", -1, 1).map do |user| Recommendable.redis.zscore "Device:3:similarities", user end => ["0.20000000000000001", "0.20000000000000001", "0.40000000000000002"]

Looks like there are similarities. I'm not relying on resque to process updates, so resque isnt it. Somehow im still just not getting recs.

[blackgold9]

In trying to debug, i inserted puts all over the recommendations method. I'm not sure if the output will help, but here goes.

def recommendations(options = {})
        defaults = { :count => 10 }
        options = defaults.merge options
        return [] if likes.count + dislikes.count == 0

        puts 'not aborting'
        unioned_predictions = "#{self.class}:#{id}:predictions"
        puts "prediction string #{unioned_predictions}"
        Recommendable.redis.zunionstore unioned_predictions, Recommendable.recommendable_classes.map {|klass|
          puts "making reccomendations for #{klass}"
          preds = predictions_set_for(klass)
          puts "preds: #{preds}"
          preds
        }

        recommendations = Recommendable.redis.zrevrange(unioned_predictions, 0, options[:count]).map do |object|
          klass, id = object.split(":")
          puts "rec id: #{id}"
          klass.constantize.find(id)
        end

        puts "recs: #{recommendations}"

        Recommendable.redis.del(unioned_predictions) and return recommendations
      end

Output:

Device.first.recommendations Device Load (0.1ms) SELECT "devices".* FROM "devices" LIMIT 1 (0.2ms) SELECT COUNT(_) FROM "recommendable_likes" WHERE "recommendable_likes"."userid" = 3 (0.2ms) SELECT COUNT() FROM "recommendable_dislikes" WHERE "recommendable_dislikes"."user_id" = 3 not aborting prediction string Device:3:predictions making reccomendations for Feed preds: Device:3:predictions:Feed recs: [] => []

[blackgold9]

Fixed it! The problem with calculation was the foreign keys on acts_As_reccomendable.rb. I had them wrong before. The relationships should read

 has_many :likes, :as => :likeable, :dependent => :destroy, :class_name => "Recommendable::Like", :foreign_key => :likeable_id
          has_many :dislikes, :as => :dislikeable, :dependent => :destroy, :class_name => "Recommendable::Dislike", :foreign_key => :dislikeable_id
          has_many :liked_by, :through => :likes, :source => :user
          has_many :disliked_by, :through => :dislikes, :source => :user
          has_many :ignores, :as => :ignoreable, :dependent => :destroy, :class_name => "Recommendable::Ignore", :foreign_key => :likeable_id
          has_many :stashes, :as => :stashable, :dependent => :destroy, :class_name => "Recommendable::StashedItem", :foreign_key => :dislikeable_id

[davidcelis]

Wait, so specifying the :foreign_key on each of those relationships and then re-running method to grab recommendations gives you recommendations instead of an empty array?