AmandaRP
commented
1 month ago Workaround:
iterator_test = MINDIterator(hparams)
iterator_test.init_news(test_news_file)
iterator_test.init_behaviors(test_behaviors_file)
iterator_test.load_data_from_file(test_news_file, test_behaviors_file)
model.test_iterator = iterator_test
model.run_eval(test_news_file, test_behaviors_file)Or to run on validation split:
iterator_valid = MINDIterator(hparams)
iterator_valid.init_news(valid_news_file)
iterator_valid.init_behaviors(valid_behaviors_file)
iterator_valid.load_data_from_file(valid_news_file, valid_behaviors_file)
model.test_iterator = iterator_valid
model.run_eval(val_news_file, val_behaviors_file)
Description
When a recommender.recommenders.models.newsrec.models.nrms model is created, the API fails to load a new dataset for inference by the model (instead, inferencing on the same dataset for the first load). The code is written to suggest that new files can be used for inference. Shown in recommenders 1.1.0 (also 1.1.1), Python 3.8.10 with tensorflow version 2.8.0. The two lower level init_news and init_behaviors in the mind_iterator.MINDIterator also don't seem to reset the iteration to the new files.
In which platform does it happen?
Jupyter Lab running in Ubuntu 20.04.4 LTS (Focal Fossa). Using Python version 3.8.10 with tensorflow version 2.8.0.
How do we replicate the issue?
How to recreate: run a "run_eval" on two files (see here). Then, try to use different input files in the method (such as the validation and test sets). Example:
The model inference results will be the same for both sets of files, which is unexpected behavior (the second input files should have different outputs).
Expected behavior (i.e. solution)
The expected behavior is for run_eval to produce different output for two different inputs.
Proposed way to fix: exposing a new iterator setup method for the base_model or NRMS_model classes should fix the issue (ala something based off the iterator_creator call in the init, see here, should probably fix the problem).
Other Comments
Thanks for maintaining this project!