actionml / harness

Harness is a Machine Learning/AI Server with plugins for many algorithms including the Universal Recommender
Apache License 2.0
283 stars 49 forks source link

User recs stop changing after certain activity #283

Closed AgusPietra closed 3 years ago

AgusPietra commented 3 years ago

Description

For a given user, using an engine with only one indicator (views), after a number of visualizations, the recommendations that it receives from UR stop changing.

Details:

I'm using default IndicatorParams config for the views indicator. So the number of events get restricted to 100. The function that has the problem is getUserHistMatcher

This is the specific part of the code that I think has the problem:

.groupBy(_.event)
          .flatMap { case (name, events) =>
            events.sortBy(_.eventTime) // implicit ordering
              .take(indicatorsMap(name)
                .maxIndicatorsPerQuery.getOrElse(DefaultURAlgoParams.MaxQueryEvents))
          }.toSeq

I propose to change it to:

.groupBy(_.event)
          .flatMap { case (name, events) =>
            events.sortBy(_.eventTime).reverse // implicit ordering
              .take(indicatorsMap(name)
                .maxIndicatorsPerQuery.getOrElse(DefaultURAlgoParams.MaxQueryEvents))
          }.toSeq

Of course, I could change the default value of 100 to any number I want, but I think it'd loose sense, because the recommendations should be sensible to the last N indicator events, and not to the first N events.

Thanks for reading!

pferrel commented 3 years ago

There are many reasons for recs not changing but they all point to a lack of discovery in your app or nt collecting all user behavior. Remember that the 100 user-history indicators used are the most recent. So to have no changes in recs means the user's most recent history has not changed. This is one big reason we allow more than just the conversion indicator. If a user searches or browses new things they will get different recs, if you do not collect pageviews or search terms you man not get new recs

I can tell you how to change the number of user history events but be aware that this will not AUTOMATICALLY cause different recs to be made, in fact allowing more will tend to make them change less. I strongly recommend that you try to include more indicator types if you want recs to follow user behavior and change with it -- add indicator types. Also it may be your app does not have sufficient ways for a user to find new things. Amazon has search and category browsing as well as recs. If you only have recs then the recommender with 'over-fit" and you will have recs that change very seldom.

All this said, increase the 100 by using maxIndicatorsPerQuery, set it to >100 https://actionml.com/docs/h_ur_config#complete-ur-engine-configuration-specification

AgusPietra commented 3 years ago

Hi Pat. Thanks for your answer. I think you are wrong. It's not using the maxIndicatorsPerQuery recent events, it's using the oldest. I'm debugging the code and I'm seeing it. Could you check that part of the code?

.groupBy(_.event)
          .flatMap { case (name, events) =>
            events.sortBy(_.eventTime) // implicit ordering
              .take(indicatorsMap(name)
                .maxIndicatorsPerQuery.getOrElse(DefaultURAlgoParams.MaxQueryEvents))
          }.toSeq

events are increasing sorted