guymorita / recommendationRaccoon

A collaborative filtering based recommendation engine and NPM module built on top of Node.js and Redis. The engine uses the Jaccard coefficient to determine the similarity between users and k-nearest-neighbors to create recommendations. This module is useful for anyone with a database of users, a database of products/movies/items and the desire to give their users the ability to like/dislike and receive recommendations.
MIT License
813 stars 131 forks source link

recommendedFor empty array with remote connection #37

Open joshbedo opened 8 years ago

joshbedo commented 8 years ago

Same thing happened to me, I simply put code in my test that was displayed in the example. I logged into my redis server and made sure that the keys do exist. Here is the code i have.

NOTE: I'm connecting to a docker container by the docker machine ip address so its a remote connection

var expect = require('chai').expect;
var raccoon = require('raccoon');
raccoon.connect(6379, '192.168.99.100');

describe('this doesnt work', function() {

  before(function(done) {
    raccoon.flush();
    done();
  });

  before(function(done) {
    raccoon.liked('garyId', 'movieId');
    raccoon.liked('garyId', 'movie2Id');
    raccoon.liked('chrisId', 'movieId');
    done();
  });

  it('should show movie2', function(done) {
    raccoon.recommendFor('chrisId', 10, function(results) {
      console.log(results, 'results'); // empty array returned
      expect(true).to.eq(false);
      done();
    })
  });
})
joshbedo commented 8 years ago

I just tried this with my local version of redis and the same thing happened. I see the keys but no results are returned.

guymorita commented 8 years ago

Did you add auth?

guymorita commented 8 years ago

Sorry haven't had lots of time to maintain this.

guymorita commented 8 years ago

Feel free to help debug ;)

joshbedo commented 8 years ago

I didn't add an auth, what would that entail? I've never done that before.

Also where is setClient being called? - https://github.com/guymorita/recommendationRaccoon/blob/master/lib/raccoon.js#L11

guymorita commented 8 years ago

Connect raccoon to your redis instance:

raccoon.connect(port, url, auth); // example of localhost: // raccoon.connect(6379, '127.0.0.1'); // auth is optional, but required for remote redis instances

guymorita commented 8 years ago

It's part of the quickstart

guymorita commented 8 years ago

Raccoon.prototype.connect = function(port, url, auth){ port = port || 6379; url = url || '127.0.0.1'; auth = auth || '';

client = redis.createClient(port, url); if (auth){ client.auth(auth, function (err) { if (err) { throw err; } }); } };

joshbedo commented 8 years ago

What would be an example of a valid auth input? I'm pretty sure my docker container of redis is wide open. I can connect to it with redis-cli -h <ip-address>.

guymorita commented 8 years ago

http://redis.io/commands/AUTH

joshbedo commented 8 years ago

Ah i get this when i add auth - 'Warning: Redis server does not require a password, but a password was supplied.'

joshbedo commented 8 years ago

when i run monitor on my redis server all i see is this, does that look correct?

➜  recommendationRaccoon git:(master) redis-cli -h 192.168.99.100
192.168.99.100:6379> keys *
(empty list or set)
192.168.99.100:6379> MONITOR
OK
1450819157.119884 [0 192.168.99.1:49878] "info"
1450819157.145418 [0 192.168.99.1:49878] "sismember" "movie:movieId:liked" "garyId"
1450819157.145440 [0 192.168.99.1:49878] "sismember" "movie:movie2Id:liked" "garyId"
1450819157.145445 [0 192.168.99.1:49878] "sismember" "movie:movieId:liked" "chrisId"
1450819157.146175 [0 192.168.99.1:49878] "zincrby" "movie:mostLiked" "1" "movieId"
1450819157.146283 [0 192.168.99.1:49878] "sadd" "movie:garyId:liked" "movieId"
1450819157.146353 [0 192.168.99.1:49878] "zincrby" "movie:mostLiked" "1" "movie2Id"
1450819157.146364 [0 192.168.99.1:49878] "sadd" "movie:garyId:liked" "movie2Id"
1450819157.146369 [0 192.168.99.1:49878] "zincrby" "movie:mostLiked" "1" "movieId"
1450819157.146375 [0 192.168.99.1:49878] "sadd" "movie:chrisId:liked" "movieId"
1450819157.146591 [0 192.168.99.1:49878] "zrevrange" "movie:chrisId:recommendedSet" "0" "10"
1450819157.146966 [0 192.168.99.1:49878] "sadd" "movie:movieId:liked" "garyId"
1450819157.146975 [0 192.168.99.1:49878] "sadd" "movie:movie2Id:liked" "garyId"
1450819157.146980 [0 192.168.99.1:49878] "sadd" "movie:movieId:liked" "chrisId"
1450819157.160715 [0 192.168.99.1:49878] "sunion" "movie:garyId:liked" "movie:garyId:disliked"
1450819157.160754 [0 192.168.99.1:49878] "sunion" "movie:garyId:liked" "movie:garyId:disliked"
1450819157.160761 [0 192.168.99.1:49878] "sunion" "movie:chrisId:liked" "movie:chrisId:disliked"
joshbedo commented 8 years ago

@guymorita Also worth noting, when i switch the function over to mostSimilarUsers i get this error callback is not defined. Does this work for you locally?

joshbedo commented 8 years ago

@guymorita would i be able to pick your brain for 10 minutes on gitter :)

joshbedo commented 8 years ago

So it looks like other methods are working, however mostSimilarUsers and a few others aren't.

guymorita commented 8 years ago

Sure, want to have a call? I'd be happy to chat.

On Tue, Dec 22, 2015 at 7:20 PM, Josh Bedo notifications@github.com wrote:

So it looks like other methods are working, however mostSimilarUsers and a few others aren't.

— Reply to this email directly or view it on GitHub https://github.com/guymorita/recommendationRaccoon/issues/37#issuecomment-166795140 .

Guy Morita

T 206.240.5846 l guymorita@gmail.com guymorita@gmail.com

LinkedIn http://www.linkedin.com/in/guymorita l Github https://github.com/guymorita l Developer Blog http://guymorita.tumblr.com l Svbtle http://alchemist.svbtle.com

gsmeros commented 8 years ago

Do you have a solution for this problem ? I am using auth for my redis server but recommendFor and similarUsers return a nil array. If however I put the recommend in the callback of a .like or .dislike call then it returns the array.

gsmeros commented 8 years ago

I used the .liked and .disliked with a callback. then recommendFor did not work. Without a callback it seems they work. But only after the user rates again something.

guymorita commented 8 years ago

It's because the first two aren't finished yet. Hence the callbacks which force the next function to wait until the first is finished. All the methods are async. On Sun, Feb 7, 2016 at 9:48 PM George Smeros notifications@github.com wrote:

I used the .liked and .disliked with a callback. then recommendFor did not work. Without a callback it seems they work. But only after the user rates again something.

— Reply to this email directly or view it on GitHub https://github.com/guymorita/recommendationRaccoon/issues/37#issuecomment-181019777 .

gsmeros commented 8 years ago

well it worked for one new user but then it stopped. I use the callback function after the .liked to ask for the recommend but still it does not work. I have many data into redis server imported. for example user with id = 1 has rated 300. if i use .recommendFor(1,10,... i get an empty array. Same for a new user with 2-3 ratings. and also the allLikedFor and allDislikedFor , return values but wrong values.

app.get('/like/:userid/:movieid', function(request,response) {
    var userId = request.params.userid;
    var movieId = request.params.movieid;
    recommender.liked(userId,movieId,function(){
         recommender.recommendFor(userId,10,function(results){
         response.send(results);
         response.end();
  });
});
});
cpatti97100 commented 7 years ago

same here

guymorita commented 7 years ago

@gsmeros @clivend The recommendations only run for that user when you call .liked. This means that if you add a bunch of ratings for user 1, then user 2, then user 3. Then ask for recommendations from user 1, the array will be empty.

Two ways to fix this:

  1. Add another review of user 1
  2. I added an additional api for testing that I haven't added to the readme yet. The itemId doesn't matter but it needs to be valid.:

    raccoon.updateSequence(userId, itemId, {updateWilson: false}).then(() => {
    
    });
cpatti97100 commented 7 years ago

thanks @guymorita for your time. right now, I am using version 0.2.3. my workflow is something like this:

1) I load a list of receipt for a customer, and for every item in the receipt i do

raccoon.liked('customer' + receipt.customerId.toString() + 'Id', product.productId.toString(), function() { resolve2('Customer ' + receipt.customerId + ' liked ' + product.productId); });

2) after some time, I call raccoon.recommendFor('customer' + request.params.customerId.toString() + 'Id', 10, function(recommendations) { return reply(recommendations); }); and this is empty.

if I look at redis I find 1) "product:2:liked" 2) "product:customer7Id:liked" 3) "product:customer8Id:similaritySet" 4) "product:customer8Id:liked" 5) "product:1:liked" 6) "product:customer7Id:similaritySet" 7) "product:scoreBoard" 8) "product:mostLiked"

but not something like "product:2:recommendedSet

what am I doing wrong?

guymorita commented 7 years ago

@clivend Was customer 2 one of the first entries? If so there will be no recs until you insert another .liked for that user. I'd recommend upgrading to the latest version if possible.

cpatti97100 commented 7 years ago

actually there's no customer 2. the liked calls are these

raccoon.liked('customer7Id', '1') raccoon.liked('customer7Id', '2') raccoon.liked('customer8Id', '1') raccoon.liked('customer8Id', '2') raccoon.liked('customer7Id', '1') raccoon.liked('customer7Id', '2')

then redis get like this 1) "product:2:liked" 2) "product:customer7Id:liked" 3) "product:customer8Id:similaritySet" 4) "product:customer8Id:liked" 5) "product:1:liked" 6) "product:customer7Id:similaritySet" 7) "product:scoreBoard" 8) "product:mostLiked"

and I call

raccoon.recommendFor('customer7Id', ...

unfortunately upgrading is not easy right now

cpatti97100 commented 7 years ago

PS can I still use callbacks with 0.2.8 or I need to refactor to promise style?

EDIT I upgraded to latest version, but still got the same issue :(

cpatti97100 commented 7 years ago

hi @guymorita , here is my last implementation based on 0.2.8

var productsPromises = receipt.ReceiptProducts.map(function(product) { return raccoon.liked('customer' + receipt.customerId.toString() + 'Id', product.productId.toString(), { updateRecs: false }); }); var updatePromises = receipt.ReceiptProducts.map(function(product) { return raccoon.updateSequence('customer' + receipt.customerId.toString() + 'Id', product.productId.toString(), { updateWilson: false }); });

after this, redis is like

1) "product:item:1:liked" 2) "product:scoreboard" 3) "product:user:customer7Id:liked" 4) "product:item:2:liked" 5) "product:user:customer7Id:similarityZSet" 6) "product:user:customer8Id:similarityZSet" 7) "product:user:customer8Id:liked" 8) "product:mostLiked"

and the recommendations for customer8Id and customer7Id are still empty arrays :(