search

NVIDIA-Merlin / Merlin

NVIDIA Merlin is an open source library providing end-to-end GPU-accelerated recommender systems, from feature engineering and preprocessing to training deep learning models and running inference in production.
Apache License 2.0
748 stars 113 forks source link

[RMP] Tensorflow support for session based recommendations integration in Merlin #433

Closed viswa-nvidia closed 1 year ago

viswa-nvidia commented 2 years ago

Problem:

Session-based and sequential-based models are an active research area for providing personalized recommendations. Transformers4Rec library was built to support the definition of such architectures and the results of our experiments conducted in the T4Rec paper showed the effectiveness of Transformers in modeling short sequences observed in session-based tasks. The T4Rec library was also used to win various RecSys challenges. We also observe a growing interest and active engagement from customers in using Transformers4Rec.

T4Rec was not actively updated for several months as the team shifted its focus to developing the Merlin Model library. MM does not currently support sequential and session-based recsys architectures. MM should support these sequential architectures and provide all necessary support to our users so that they can build such effective models.

Goal:

Definition of Done

Have an example that serves TF session based model in conjunction with a NVT workflow where the session based models scores the whole catalog

Constraints:

Starting Point:

Training

Proposed API:

inputs = InputBlock(
  schema=con_schema + seq_schema,
  post=BroadcastToSequence(con_schema, seq_schema)
)
model = RetrievalModel(
  XLNetEncoder(inputs, n_head=4, n_layer=2),
  CategoricalOutput(inputs.select_by_tag(Tags.ITEM_ID))
)

topk = TopKEncoder(model)
topk.evaluate(...)

loader = mm.Loader(
  dataset, 
  batch_size=1000,
  transforms=PredictMasked(seq_schema, target=Tags.ITEM_ID)
)

model.fit(loader)

Inputs

Add support for sequential-inputs & shared embeddings @edknv, @oliverholworthy & @marcromeyn

  • [x] NVIDIA-Merlin/models#674
  • [x] NVIDIA-Merlin/models#694
  • [x] NVIDIA-Merlin/models#697
  • [x] NVIDIA-Merlin/models#696

Masking

Add training strategies for sequence models @gabrielspmoreira

  • [x] NVIDIA-Merlin/models#713
  • [x] NVIDIA-Merlin/models#691
  • [x] NVIDIA-Merlin/models#692

RetrievalModel

Make RetrievalModel more generic to session-based use-cases + allow encoders being served @marcromeyn

  • [x] NVIDIA-Merlin/models#698
  • [x] NVIDIA-Merlin/models#729
  • [x] NVIDIA-Merlin/models#730

Outputs

Improve model-outputs to handle session-based recsys @marcromeyn & @sararb

  • [x] NVIDIA-Merlin/models#715
  • [x] NVIDIA-Merlin/models#718
  • [x] NVIDIA-Merlin/models#731

Port Sequence Architectures

Add sequence-encoding blocks like transformers @sararb

Fixes for the GTC tutorial on session-based recommendation

Inference support

Save schema on model save

Systems

Documentation

Examples

Small scale (see #352)

EvenOldridge commented 2 years ago

@sararb @gabrielspmoreira can you flesh this out as best as possible in @marcromeyn's absence.

EvenOldridge commented 2 years ago

@gabrielspmoreira what does the architecture look like for the system for session based? Are we planning to use session generation to feed into a candidate generation stage?

gabrielspmoreira commented 2 years ago

@

@gabrielspmoreira what does the architecture look like for the system for session based? Are we planning to use session generation to feed into a candidate generation stage?

@EvenOldridge the session-based recommendation works as a next-item prediction task. It can be seen as a retrieval model, where the query tower users a sequential model (e.g. RNN, Transformer) and outputs a query representation/vector. During inference, such vectors can be used to retrieve the similar items from ANN the same way a retrieval model does. So we believe we wouldn't need anything special on Merlin Systems related to the output of a session-based recommendation model. The main different is in the input, as such sequential models expect list features as input. NVTabular already supports processing and storing such list features as you know, but there might be some challenges on Systems building the Triton ensemble with list features support.

viswa-nvidia commented 2 years ago

@karlhigley , is the systems section here updated. Pleaes review

karlhigley commented 2 years ago

It is up to date with the current state of our knowledge of the work

viswa-nvidia commented 2 years ago

@marcromeyn , in one of the meetings, I made a note that this task is dependent on some tasks covered in RMP479-EMBEDDINGS initiative [RMP] Enable users to pass embedding tables directly into the input block in order to more easily support new functionality (non-trainable embeddings, different dimensions, model parallel, etc) . Is this correct ? which are these tasks. ? @EvenOldridge for vis.

viswa-nvidia commented 1 year ago

@rnyak , please link the systems - multi hot related development

viswa-nvidia commented 1 year ago

@oliverholworthy , please add the input output schema related tickets to this ticket

oliverholworthy commented 1 year ago

@viswa-nvidia For the saving method:

This is the parent issue for that:

We have implemented the save method to save input schema, but currently missing output schema.

oliverholworthy commented 1 year ago

Also identified an error in Merlin Models that may impact ability to serve Transformer-based models that affects issues with saving a model after loading.

https://github.com/NVIDIA-Merlin/models/issues/878

viswa-nvidia commented 1 year ago

@karlhigley , please add the Triton related PR ( serving signatures haven't matched up with what model expects ) in the ticket.

viswa-nvidia commented 1 year ago

@rnyak to follow up with @radekosmulski for blocker ( 23.04 )

EvenOldridge commented 1 year ago

It was pointed out to me that we should be consistent about when we consider something done or not so I'm going to reopen this and move it to 22.05. @bbozkaya You've had the only remaining ticket (review the API) on your todo for the past two weeks with no progress. Is this something you're able to take on so that we can close the ticket. If not let us know and we can reassign.