search

recommenders-team / recommenders

Best Practices on Recommendation Systems
https://recommenders-team.github.io/recommenders/intro.html
MIT License
19.12k stars 3.09k forks source link

[BUG] Upgrade to Tensorflow>2.10.1 breaks several of the DeepRec algos #2018

Closed miguelgfierro closed 6 months ago

miguelgfierro commented 1 year ago

Description

We had a security alert https://github.com/recommenders-team/recommenders/pull/2017 and we had to update the TF version.

When we moved to >2.10.1, several algos stopped working: xDeepFM, SUM, LightGCN and SliRec

In which platform does it happen?

=========================== short test summary info ============================
FAILED tests/unit/recommenders/models/test_deeprec_model.py::test_xdeepfm_component_definition
FAILED tests/unit/recommenders/models/test_deeprec_model.py::test_slirec_component_definition
FAILED tests/unit/recommenders/models/test_deeprec_model.py::test_sum_component_definition
============= 3 failed, 12 passed, 7 warnings in 153.99s (0:02:33) =============
=================================== FAILURES ===================================
______________________ test_xdeepfm_component_definition _______________________

deeprec_resource_path = PosixPath('/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/tests/resources/deeprec')

    @pytest.mark.gpu
    def test_xdeepfm_component_definition(deeprec_resource_path):
        data_path = os.path.join(deeprec_resource_path, "xdeepfm")
        yaml_file = os.path.join(data_path, "xDeepFM.yaml")

        if not os.path.exists(yaml_file):
            download_deeprec_resources(
                "https://recodatasets.z20.web.core.windows.net/deeprec/",
                data_path,
                "xdeepfmresources.zip",
            )

        hparams = prepare_hparams(yaml_file)
>       model = XDeepFMModel(hparams, FFMTextIterator)
E       NameError: name 'XDeepFMModel' is not defined

tests/unit/recommenders/models/test_deeprec_model.py:131: NameError
----------------------------- Captured stdout call -----------------------------
INFO:recommenders.datasets.download_utils:Downloading https://recodatasets.z20.web.core.windows.net/deeprec/xdeepfmresources.zip
----------------------------- Captured stderr call -----------------------------

  0%|          | 0.00/10.3k [00:00<?, ?KB/s]
 68%|██████▊   | 6.97k/10.3k [00:00<00:00, 69.7kKB/s]
100%|██████████| 10.3k/10.3k [00:00<00:00, 81.3kKB/s]
------------------------------ Captured log call -------------------------------
INFO     recommenders.datasets.download_utils:download_utils.py:38 Downloading https://recodatasets.z20.web.core.windows.net/deeprec/xdeepfmresources.zip
_______________________ test_slirec_component_definition _______________________

sequential_files = ('/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/tests/resources/deeprec/slirec', '/mnt/azureml/cr/j/e49a1d...ab.pkl', '/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/tests/resources/deeprec/slirec/category_vocab.pkl')
deeprec_config_path = PosixPath('/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/recommenders/models/deeprec/config')

    @pytest.mark.gpu
    def test_slirec_component_definition(sequential_files, deeprec_config_path):
        yaml_file = os.path.join(deeprec_config_path, "sli_rec.yaml")
        data_path, user_vocab, item_vocab, cate_vocab = sequential_files

        hparams = prepare_hparams(
            yaml_file,
            train_num_ngs=4,
            embed_l2=0.0,
            layer_l2=0.0,
            learning_rate=0.001,
            epochs=1,
            MODEL_DIR=os.path.join(data_path, "model"),
            SUMMARIES_DIR=os.path.join(data_path, "summary"),
            user_vocab=user_vocab,
            item_vocab=item_vocab,
            cate_vocab=cate_vocab,
            need_sample=True,
        )

>       model = SLI_RECModel(hparams, SequentialIterator)

tests/unit/recommenders/models/test_deeprec_model.py:248: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
recommenders/models/deeprec/models/sequential/sequential_base_model.py:51: in __init__
    super().__init__(hparams, iterator_creator, graph=self.graph, seed=seed)
recommenders/models/deeprec/models/base_model.py:56: in __init__
    self.logit = self._build_graph()
recommenders/models/deeprec/models/sequential/sequential_base_model.py:71: in _build_graph
    model_output = self._build_seq_graph()
recommenders/models/deeprec/models/sequential/sli_rec.py:61: in _build_seq_graph
    rnn_outputs, _ = dynamic_rnn(
/azureml-envs/azureml_c4eff4a5e459820972fa4e3ca572161c/lib/python3.9/site-packages/tensorflow/python/util/deprecation.py:383: in new_func
    return func(*args, **kwargs)
/azureml-envs/azureml_c4eff4a5e459820972fa4e3ca572161c/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py:153: in error_handler
    raise e.with_traceback(filtered_tb) from None
/tmp/__autograph_generated_filekos1if_z.py:110: in tf__call
    ag__.if_stmt(ag__.ld(self)._linear1 is None, if_body_4, else_body_4, get_state_4, set_state_4, ('self._linear1',), 1)
/tmp/__autograph_generated_filekos1if_z.py:106: in if_body_4
    ag__.ld(self)._linear1 = ag__.converted_call(ag__.ld(_Linear), ([ag__.ld(inputs), ag__.ld(m_prev)], 4 * ag__.ld(self)._num_units, True), None, fscope)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <recommenders.models.deeprec.models.sequential.rnn_cell_implement._Linear object at 0x146050352070>
args = [<tf.Tensor 'time4lstm/time4lstm_cell/strided_slice_2:0' shape=(None, 32) dtype=float32>, <tf.Tensor 'Placeholder_3:0' shape=(None, 40) dtype=float32>]
output_size = 160, build_bias = True, bias_initializer = None
kernel_initializer = None

    def __init__(
        self,
        args,
        output_size,
        build_bias,
        bias_initializer=None,
        kernel_initializer=None,
    ):
        self._build_bias = build_bias

>       if args is None or (nest.is_sequence(args) and not args):
E       AttributeError: in user code:
E       
E           File "/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/recommenders/models/deeprec/models/sequential/rnn_cell_implement.py", line 227, in call  *
E               self._linear1 = _Linear([inputs, m_prev], 4 * self._num_units, True)
E           File "/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/recommenders/models/deeprec/models/sequential/rnn_cell_implement.py", line 584, in __init__  **
E               if args is None or (nest.is_sequence(args) and not args):
E       
E           AttributeError: module 'tensorflow.python.util.nest' has no attribute 'is_sequence'

recommenders/models/deeprec/models/sequential/rnn_cell_implement.py:584: AttributeError
---------------------------- Captured stdout setup -----------------------------
INFO:recommenders.datasets.download_utils:Downloading http://snap.stanford.edu/data/amazon/productGraph/categoryFiles/reviews_Movies_and_TV_5.json.gz
INFO:recommenders.datasets.download_utils:Downloading http://snap.stanford.edu/data/amazon/productGraph/categoryFiles/meta_Movies_and_TV.json.gz
INFO:root:start reviews preprocessing...
INFO:root:start meta preprocessing...
INFO:root:start create instances...
INFO:root:creating item2cate dict
INFO:root:getting sampled data...
INFO:root:start data processing...
INFO:root:data generating...
INFO:root:vocab generating...
INFO:root:start valid negative sampling
INFO:root:start test negative sampling
---------------------------- Captured stderr setup -----------------------------

  0%|          | 0.00/692k [00:00<?, ?KB/s]
  0%|          | 30.0/692k [00:00<55:28, 208KB/s]
  0%|          | 69.0/692k [00:00<48:14, 239KB/s]
  0%|          | 140/692k [00:00<32:43, 353KB/s] 
  0%|          | 251/692k [00:00<22:24, 515KB/s]
  0%|          | 421/692k [00:00<14:09, 815KB/s]
  0%|          | 533/692k [00:00<12:57, 889KB/s]
  0%|          | 792/692k [00:00<08:49, 1.31kKB/s]
  0%|          | 968/692k [00:01<08:03, 1.43kKB/s]
  0%|          | 1.44k/692k [00:01<05:05, 2.26kKB/s]
  0%|          | 1.80k/692k [00:01<04:22, 2.63kKB/s]
  0%|          | 2.72k/692k [00:01<02:42, 4.26kKB/s]
  1%|          | 3.53k/692k [00:01<02:10, 5.27kKB/s]
  1%|          | 5.25k/692k [00:01<01:22, 8.30kKB/s]
  1%|          | 6.68k/692k [00:01<01:08, 9.95kKB/s]
  1%|▏         | 10.4k/692k [00:01<00:40, 16.8kKB/s]
  2%|▏         | 12.9k/692k [00:01<00:35, 19.2kKB/s]
  3%|▎         | 17.4k/692k [00:01<00:25, 26.5kKB/s]
  3%|▎         | 21.6k/692k [00:02<00:23, 28.5kKB/s]
  4%|▍         | 26.1k/692k [00:02<00:20, 32.9kKB/s]
  4%|▍         | 30.1k/692k [00:02<00:20, 33.0kKB/s]
  5%|▌         | 34.8k/692k [00:02<00:18, 36.4kKB/s]
  6%|▌         | 38.4k/692k [00:02<00:20, 31.6kKB/s]
  6%|▌         | 41.7k/692k [00:02<00:22, 28.9kKB/s]
  7%|▋         | 45.2k/692k [00:02<00:23, 27.5kKB/s]
  7%|▋         | 48.6k/692k [00:02<00:22, 29.2kKB/s]
  7%|▋         | 51.7k/692k [00:03<00:23, 27.4kKB/s]
  8%|▊         | 55.0k/692k [00:03<00:23, 26.7kKB/s]
  8%|▊         | 57.9k/692k [00:03<00:23, 27.3kKB/s]
  9%|▉         | 61.0k/692k [00:03<00:23, 27.1kKB/s]
  9%|▉         | 64.0k/692k [00:03<00:22, 27.8kKB/s]
 10%|▉         | 67.1k/692k [00:03<00:22, 27.5kKB/s]
 10%|█         | 70.2k/692k [00:03<00:21, 28.3kKB/s]
 11%|█         | 73.4k/692k [00:03<00:21, 28.2kKB/s]
 11%|█         | 76.5k/692k [00:03<00:21, 28.9kKB/s]
 12%|█▏        | 79.8k/692k [00:04<00:20, 30.1kKB/s]
 12%|█▏        | 82.8k/692k [00:04<00:21, 28.8kKB/s]
 12%|█▏        | 86.1k/692k [00:04<00:20, 30.0kKB/s]
 13%|█▎        | 89.1k/692k [00:04<00:20, 28.7kKB/s]
 13%|█▎        | 92.6k/692k [00:04<00:19, 30.3kKB/s]
 14%|█▍        | 95.6k/692k [00:04<00:19, 29.9kKB/s]
 14%|█▍        | 98.6k/692k [00:04<00:20, 29.4kKB/s]
 15%|█▍        | 102k/692k [00:04<00:19, 30.5kKB/s] 
 15%|█▌        | 105k/692k [00:04<00:19, 30.4kKB/s]
 16%|█▌        | 108k/692k [00:05<00:18, 31.3kKB/s]
 16%|█▌        | 112k/692k [00:05<00:18, 31.2kKB/s]
 17%|█▋        | 115k/692k [00:05<00:19, 30.3kKB/s]
 17%|█▋        | 118k/692k [00:05<00:18, 31.6kKB/s]
 18%|█▊        | 121k/692k [00:05<00:18, 31.4kKB/s]
 18%|█▊        | 125k/692k [00:05<00:17, 32.3kKB/s]
 18%|█▊        | 128k/692k [00:05<00:17, 31.5kKB/s]
 19%|█▉        | 131k/692k [00:05<00:18, 31.1kKB/s]
 19%|█▉        | 134k/692k [00:05<00:20, 27.5kKB/s]
 20%|█▉        | 137k/692k [00:06<00:21, 25.6kKB/s]
 20%|██        | 140k/692k [00:06<00:23, 23.6kKB/s]
 21%|██        | 143k/692k [00:06<00:21, 25.5kKB/s]
 21%|██        | 145k/692k [00:06<00:22, 23.9kKB/s]
 21%|██▏       | 148k/692k [00:06<00:23, 23.0kKB/s]
 22%|██▏       | 151k/692k [00:06<00:22, 24.2kKB/s]
 22%|██▏       | 153k/692k [00:06<00:23, 23.0kKB/s]
 23%|██▎       | 156k/692k [00:06<00:22, 24.4kKB/s]
 23%|██▎       | 159k/692k [00:06<00:22, 23.4kKB/s]
 23%|██▎       | 162k/692k [00:07<00:21, 24.7kKB/s]
 24%|██▎       | 164k/692k [00:07<00:22, 23.7kKB/s]
 24%|██▍       | 167k/692k [00:07<00:21, 24.4kKB/s]
 24%|██▍       | 169k/692k [00:07<00:21, 24.3kKB/s]
 25%|██▍       | 172k/692k [00:07<00:21, 24.7kKB/s]
 25%|██▌       | 175k/692k [00:07<00:20, 25.4kKB/s]
 26%|██▌       | 177k/692k [00:07<00:20, 25.0kKB/s]
 26%|██▌       | 180k/692k [00:07<00:20, 25.2kKB/s]
 26%|██▋       | 182k/692k [00:07<00:20, 24.9kKB/s]
 27%|██▋       | 185k/692k [00:08<00:20, 24.7kKB/s]
 27%|██▋       | 188k/692k [00:08<00:20, 25.1kKB/s]
 27%|██▋       | 190k/692k [00:08<00:19, 25.4kKB/s]
 28%|██▊       | 193k/692k [00:08<00:19, 25.5kKB/s]
 28%|██▊       | 196k/692k [00:08<00:19, 25.7kKB/s]
 29%|██▊       | 198k/692k [00:08<00:19, 25.4kKB/s]
 29%|██▉       | 201k/692k [00:08<00:19, 25.7kKB/s]
 29%|██▉       | 203k/692k [00:08<00:19, 25.5kKB/s]
 30%|██▉       | 206k/692k [00:08<00:18, 26.1kKB/s]
 30%|███       | 209k/692k [00:08<00:18, 26.0kKB/s]
 31%|███       | 211k/692k [00:09<00:18, 25.4kKB/s]
 31%|███       | 214k/692k [00:09<00:18, 25.6kKB/s]
 31%|███▏      | 217k/692k [00:09<00:18, 25.9kKB/s]
 32%|███▏      | 220k/692k [00:09<00:17, 26.3kKB/s]
 32%|███▏      | 222k/692k [00:09<00:18, 25.8kKB/s]
 33%|███▎      | 225k/692k [00:09<00:17, 26.3kKB/s]
 33%|███▎      | 228k/692k [00:09<00:17, 26.1kKB/s]
 33%|███▎      | 231k/692k [00:09<00:17, 26.5kKB/s]
 34%|███▎      | 233k/692k [00:09<00:17, 26.5kKB/s]
 34%|███▍      | 236k/692k [00:09<00:17, 25.7kKB/s]
 35%|███▍      | 239k/692k [00:10<00:16, 26.8kKB/s]
 35%|███▍      | 242k/692k [00:10<00:16, 26.8kKB/s]
 35%|███▌      | 244k/692k [00:10<00:17, 26.2kKB/s]
 36%|███▌      | 247k/692k [00:10<00:16, 26.7kKB/s]
 36%|███▌      | 250k/692k [00:10<00:16, 26.6kKB/s]
 36%|███▋      | 253k/692k [00:10<00:19, 22.1kKB/s]
 37%|███▋      | 256k/692k [00:10<00:16, 26.0kKB/s]
 37%|███▋      | 259k/692k [00:10<00:18, 23.6kKB/s]
 38%|███▊      | 262k/692k [00:11<00:19, 21.9kKB/s]
 38%|███▊      | 264k/692k [00:11<00:19, 22.1kKB/s]
 38%|███▊      | 266k/692k [00:11<00:20, 20.8kKB/s]
 39%|███▉      | 268k/692k [00:11<00:21, 20.2kKB/s]
 39%|███▉      | 270k/692k [00:11<00:20, 20.2kKB/s]
 39%|███▉      | 273k/692k [00:11<00:20, 20.8kKB/s]
 40%|███▉      | 275k/692k [00:11<00:21, 19.8kKB/s]
 40%|███▉      | 277k/692k [00:11<00:20, 20.4kKB/s]
 40%|████      | 279k/692k [00:11<00:20, 19.7kKB/s]
 41%|████      | 281k/692k [00:12<00:20, 20.5kKB/s]
 41%|████      | 283k/692k [00:12<00:20, 20.0kKB/s]
 41%|████▏     | 286k/692k [00:12<00:19, 20.5kKB/s]
 42%|████▏     | 288k/692k [00:12<00:20, 20.1kKB/s]
 42%|████▏     | 290k/692k [00:12<00:19, 20.9kKB/s]
 42%|████▏     | 292k/692k [00:12<00:19, 20.8kKB/s]
 43%|████▎     | 294k/692k [00:12<00:19, 20.6kKB/s]
 43%|████▎     | 297k/692k [00:12<00:19, 20.3kKB/s]
 43%|████▎     | 299k/692k [00:12<00:18, 20.8kKB/s]
 43%|████▎     | 301k/692k [00:13<00:18, 20.6kKB/s]
 44%|████▍     | 303k/692k [00:13<00:18, 21.0kKB/s]
 44%|████▍     | 306k/692k [00:13<00:18, 20.7kKB/s]
 45%|████▍     | 308k/692k [00:13<00:17, 21.7kKB/s]
 45%|████▍     | 311k/692k [00:13<00:18, 20.7kKB/s]
 45%|████▌     | 313k/692k [00:13<00:17, 22.3kKB/s]
 46%|████▌     | 315k/692k [00:13<00:18, 20.6kKB/s]
 46%|████▌     | 318k/692k [00:13<00:16, 22.2kKB/s]
 46%|████▋     | 320k/692k [00:13<00:17, 20.9kKB/s]
 47%|████▋     | 323k/692k [00:14<00:16, 22.6kKB/s]
 47%|████▋     | 325k/692k [00:14<00:17, 21.4kKB/s]
 47%|████▋     | 328k/692k [00:14<00:17, 21.1kKB/s]
 48%|████▊     | 331k/692k [00:14<00:16, 22.4kKB/s]
 48%|████▊     | 333k/692k [00:14<00:17, 20.9kKB/s]
 48%|████▊     | 336k/692k [00:14<00:15, 22.5kKB/s]
 49%|████▉     | 338k/692k [00:14<00:16, 21.3kKB/s]
 49%|████▉     | 341k/692k [00:14<00:14, 23.5kKB/s]
 50%|████▉     | 343k/692k [00:14<00:15, 22.1kKB/s]
 50%|████▉     | 346k/692k [00:15<00:16, 21.4kKB/s]
 50%|█████     | 348k/692k [00:15<00:15, 22.3kKB/s]
 51%|█████     | 351k/692k [00:15<00:15, 21.7kKB/s]
 51%|█████     | 353k/692k [00:15<00:15, 22.6kKB/s]
 51%|█████▏    | 356k/692k [00:15<00:15, 21.8kKB/s]
 52%|█████▏    | 358k/692k [00:15<00:14, 22.7kKB/s]
 52%|█████▏    | 361k/692k [00:15<00:15, 21.8kKB/s]
 52%|█████▏    | 363k/692k [00:15<00:14, 22.7kKB/s]
 53%|█████▎    | 366k/692k [00:15<00:14, 21.9kKB/s]
 53%|█████▎    | 368k/692k [00:16<00:14, 22.7kKB/s]
 54%|█████▎    | 370k/692k [00:16<00:14, 22.0kKB/s]
 54%|█████▍    | 373k/692k [00:16<00:13, 22.9kKB/s]
 54%|█████▍    | 375k/692k [00:16<00:14, 22.0kKB/s]
 55%|█████▍    | 378k/692k [00:16<00:13, 22.8kKB/s]
 55%|█████▍    | 380k/692k [00:16<00:14, 21.9kKB/s]
 55%|█████▌    | 383k/692k [00:16<00:13, 23.0kKB/s]
 56%|█████▌    | 385k/692k [00:16<00:13, 22.0kKB/s]
 56%|█████▌    | 388k/692k [00:16<00:13, 22.8kKB/s]
 56%|█████▋    | 390k/692k [00:17<00:13, 22.1kKB/s]
 57%|█████▋    | 393k/692k [00:17<00:13, 23.0kKB/s]
 57%|█████▋    | 395k/692k [00:17<00:13, 22.1kKB/s]
 57%|█████▋    | 398k/692k [00:17<00:12, 22.9kKB/s]
 58%|█████▊    | 400k/692k [00:17<00:13, 22.1kKB/s]
 58%|█████▊    | 403k/692k [00:17<00:12, 23.0kKB/s]
 59%|█████▊    | 405k/692k [00:17<00:13, 22.1kKB/s]
 59%|█████▉    | 408k/692k [00:17<00:12, 22.6kKB/s]
 59%|█████▉    | 410k/692k [00:17<00:12, 22.3kKB/s]
 60%|█████▉    | 412k/692k [00:18<00:12, 22.5kKB/s]
 60%|█████▉    | 415k/692k [00:18<00:12, 22.4kKB/s]
 60%|██████    | 417k/692k [00:18<00:12, 22.4kKB/s]
 61%|██████    | 420k/692k [00:18<00:12, 22.5kKB/s]
 61%|██████    | 422k/692k [00:18<00:12, 22.4kKB/s]
 61%|██████▏   | 425k/692k [00:18<00:11, 22.4kKB/s]
 62%|██████▏   | 427k/692k [00:18<00:11, 22.5kKB/s]
 62%|██████▏   | 430k/692k [00:18<00:11, 22.3kKB/s]
 62%|██████▏   | 432k/692k [00:18<00:12, 21.3kKB/s]
 63%|██████▎   | 434k/692k [00:19<00:13, 19.3kKB/s]
 63%|██████▎   | 436k/692k [00:19<00:14, 17.8kKB/s]
 63%|██████▎   | 438k/692k [00:19<00:14, 18.0kKB/s]
 64%|██████▎   | 440k/692k [00:19<00:14, 17.2kKB/s]
 64%|██████▍   | 442k/692k [00:19<00:14, 17.6kKB/s]
 64%|██████▍   | 444k/692k [00:19<00:14, 16.6kKB/s]
 64%|██████▍   | 445k/692k [00:19<00:14, 16.5kKB/s]
 65%|██████▍   | 447k/692k [00:19<00:14, 17.0kKB/s]
 65%|██████▍   | 449k/692k [00:19<00:14, 16.7kKB/s]
 65%|██████▌   | 451k/692k [00:20<00:13, 17.3kKB/s]
 65%|██████▌   | 453k/692k [00:20<00:14, 16.7kKB/s]
 66%|██████▌   | 455k/692k [00:20<00:13, 17.4kKB/s]
 66%|██████▌   | 457k/692k [00:20<00:13, 17.0kKB/s]
 66%|██████▋   | 459k/692k [00:20<00:13, 17.5kKB/s]
 67%|██████▋   | 461k/692k [00:20<00:13, 17.2kKB/s]
 67%|██████▋   | 463k/692k [00:20<00:12, 17.9kKB/s]
 67%|██████▋   | 465k/692k [00:20<00:13, 17.4kKB/s]
 67%|██████▋   | 467k/692k [00:20<00:12, 18.2kKB/s]
 68%|██████▊   | 468k/692k [00:21<00:12, 18.2kKB/s]
 68%|██████▊   | 470k/692k [00:21<00:12, 17.6kKB/s]
 68%|██████▊   | 472k/692k [00:21<00:11, 18.4kKB/s]
 68%|██████▊   | 474k/692k [00:21<00:12, 17.2kKB/s]
 69%|██████▉   | 476k/692k [00:21<00:11, 18.6kKB/s]
 69%|██████▉   | 478k/692k [00:21<00:12, 17.5kKB/s]
 69%|██████▉   | 481k/692k [00:21<00:11, 18.4kKB/s]
 70%|██████▉   | 482k/692k [00:21<00:11, 17.7kKB/s]
 70%|██████▉   | 485k/692k [00:21<00:11, 18.8kKB/s]
 70%|███████   | 486k/692k [00:22<00:11, 17.8kKB/s]
 71%|███████   | 489k/692k [00:22<00:11, 17.6kKB/s]
 71%|███████   | 491k/692k [00:22<00:10, 19.8kKB/s]
 71%|███████▏  | 493k/692k [00:22<00:10, 18.2kKB/s]
 72%|███████▏  | 496k/692k [00:22<00:11, 17.8kKB/s]
 72%|███████▏  | 498k/692k [00:22<00:09, 20.0kKB/s]
 72%|███████▏  | 500k/692k [00:22<00:10, 18.4kKB/s]
 73%|███████▎  | 502k/692k [00:22<00:10, 18.0kKB/s]
 73%|███████▎  | 505k/692k [00:23<00:09, 19.7kKB/s]
 73%|███████▎  | 507k/692k [00:23<00:10, 18.5kKB/s]
 74%|███████▎  | 509k/692k [00:23<00:10, 18.2kKB/s]
 74%|███████▍  | 512k/692k [00:23<00:09, 19.7kKB/s]
 74%|███████▍  | 514k/692k [00:23<00:09, 18.8kKB/s]
 75%|███████▍  | 516k/692k [00:23<00:09, 19.5kKB/s]
 75%|███████▍  | 518k/692k [00:23<00:09, 18.7kKB/s]
 75%|███████▌  | 521k/692k [00:23<00:08, 19.6kKB/s]
 75%|███████▌  | 522k/692k [00:23<00:09, 18.6kKB/s]
 76%|███████▌  | 525k/692k [00:24<00:08, 19.6kKB/s]
 76%|███████▌  | 527k/692k [00:24<00:08, 18.8kKB/s]
 76%|███████▋  | 529k/692k [00:24<00:08, 18.3kKB/s]
 77%|███████▋  | 532k/692k [00:24<00:07, 20.6kKB/s]
 77%|███████▋  | 534k/692k [00:24<00:08, 18.7kKB/s]
 77%|███████▋  | 536k/692k [00:24<00:08, 18.3kKB/s]
 78%|███████▊  | 539k/692k [00:24<00:07, 19.8kKB/s]
 78%|███████▊  | 541k/692k [00:24<00:08, 18.9kKB/s]
 78%|███████▊  | 543k/692k [00:25<00:08, 18.6kKB/s]
 79%|███████▉  | 546k/692k [00:25<00:07, 20.1kKB/s]
 79%|███████▉  | 548k/692k [00:25<00:07, 19.1kKB/s]
 79%|███████▉  | 550k/692k [00:25<00:07, 20.1kKB/s]
 80%|███████▉  | 552k/692k [00:25<00:07, 19.1kKB/s]
 80%|████████  | 554k/692k [00:25<00:07, 18.5kKB/s]
 80%|████████  | 557k/692k [00:25<00:06, 20.5kKB/s]
 81%|████████  | 559k/692k [00:25<00:07, 18.6kKB/s]
 81%|████████  | 561k/692k [00:25<00:06, 19.6kKB/s]
 81%|████████▏ | 563k/692k [00:26<00:06, 18.8kKB/s]
 82%|████████▏ | 566k/692k [00:26<00:06, 18.5kKB/s]
 82%|████████▏ | 568k/692k [00:26<00:05, 20.8kKB/s]
 82%|████████▏ | 570k/692k [00:26<00:06, 19.0kKB/s]
 83%|████████▎ | 573k/692k [00:26<00:06, 19.7kKB/s]
 83%|████████▎ | 575k/692k [00:26<00:06, 18.9kKB/s]
 83%|████████▎ | 577k/692k [00:26<00:06, 18.5kKB/s]
 84%|████████▎ | 580k/692k [00:26<00:05, 20.0kKB/s]
 84%|████████▍ | 582k/692k [00:27<00:05, 18.9kKB/s]
 84%|████████▍ | 584k/692k [00:27<00:05, 20.1kKB/s]
 85%|████████▍ | 586k/692k [00:27<00:05, 19.0kKB/s]
 85%|████████▍ | 588k/692k [00:27<00:05, 18.6kKB/s]
 85%|████████▌ | 591k/692k [00:27<00:05, 19.9kKB/s]
 86%|████████▌ | 593k/692k [00:27<00:05, 18.9kKB/s]
 86%|████████▌ | 595k/692k [00:27<00:04, 20.2kKB/s]
 86%|████████▋ | 597k/692k [00:27<00:04, 19.1kKB/s]
 87%|████████▋ | 600k/692k [00:27<00:05, 18.5kKB/s]
 87%|████████▋ | 602k/692k [00:28<00:04, 19.9kKB/s]
 87%|████████▋ | 604k/692k [00:28<00:04, 18.9kKB/s]
 88%|████████▊ | 607k/692k [00:28<00:04, 20.2kKB/s]
 88%|████████▊ | 609k/692k [00:28<00:04, 19.2kKB/s]
 88%|████████▊ | 611k/692k [00:28<00:04, 18.5kKB/s]
 89%|████████▊ | 614k/692k [00:28<00:03, 19.9kKB/s]
 89%|████████▉ | 616k/692k [00:28<00:04, 18.9kKB/s]
 89%|████████▉ | 618k/692k [00:28<00:03, 20.2kKB/s]
 90%|████████▉ | 620k/692k [00:29<00:03, 19.1kKB/s]
 90%|████████▉ | 622k/692k [00:29<00:03, 19.5kKB/s]
 90%|█████████ | 624k/692k [00:29<00:03, 18.7kKB/s]
 90%|█████████ | 626k/692k [00:29<00:03, 19.8kKB/s]
 91%|█████████ | 628k/692k [00:29<00:03, 18.7kKB/s]
 91%|█████████ | 631k/692k [00:29<00:03, 20.0kKB/s]
 91%|█████████▏| 633k/692k [00:29<00:03, 19.0kKB/s]
 92%|█████████▏| 635k/692k [00:29<00:02, 19.8kKB/s]
 92%|█████████▏| 637k/692k [00:29<00:02, 18.7kKB/s]
 92%|█████████▏| 639k/692k [00:30<00:02, 18.6kKB/s]
 93%|█████████▎| 642k/692k [00:30<00:02, 20.1kKB/s]
 93%|█████████▎| 644k/692k [00:30<00:02, 18.5kKB/s]
 93%|█████████▎| 646k/692k [00:30<00:02, 18.8kKB/s]
 94%|█████████▎| 648k/692k [00:30<00:02, 18.6kKB/s]
 94%|█████████▍| 651k/692k [00:30<00:02, 19.1kKB/s]
 94%|█████████▍| 653k/692k [00:30<00:02, 18.8kKB/s]
 95%|█████████▍| 655k/692k [00:30<00:01, 19.4kKB/s]
 95%|█████████▍| 657k/692k [00:30<00:01, 19.2kKB/s]
 95%|█████████▌| 660k/692k [00:31<00:01, 19.4kKB/s]
 96%|█████████▌| 662k/692k [00:31<00:01, 19.8kKB/s]
 96%|█████████▌| 664k/692k [00:31<00:01, 19.5kKB/s]
 96%|█████████▌| 666k/692k [00:31<00:01, 20.6kKB/s]
 97%|█████████▋| 668k/692k [00:31<00:01, 19.6kKB/s]
 97%|█████████▋| 671k/692k [00:31<00:01, 20.7kKB/s]
 97%|█████████▋| 673k/692k [00:31<00:00, 19.8kKB/s]
 98%|█████████▊| 675k/692k [00:31<00:00, 20.8kKB/s]
 98%|█████████▊| 677k/692k [00:31<00:00, 19.9kKB/s]
 98%|█████████▊| 680k/692k [00:32<00:00, 21.1kKB/s]
 98%|█████████▊| 682k/692k [00:32<00:00, 20.1kKB/s]
 99%|█████████▉| 684k/692k [00:32<00:00, 21.5kKB/s]
 99%|█████████▉| 687k/692k [00:32<00:00, 19.6kKB/s]
100%|█████████▉| 689k/692k [00:32<00:00, 20.4kKB/s]
100%|█████████▉| 692k/692k [00:32<00:00, 22.5kKB/s]
100%|██████████| 692k/692k [00:32<00:00, 21.2kKB/s]

  0%|          | 0.00/97.5k [00:00<?, ?KB/s]
  0%|          | 30.0/97.5k [00:00<07:56, 205KB/s]
  0%|          | 77.0/97.5k [00:00<05:56, 274KB/s]
  0%|          | 156/97.5k [00:00<04:35, 353KB/s] 
  0%|          | 263/97.5k [00:00<03:19, 488KB/s]
  0%|          | 400/97.5k [00:00<02:31, 643KB/s]
  1%|          | 623/97.5k [00:00<01:43, 934KB/s]
  1%|          | 944/97.5k [00:01<01:12, 1.34kKB/s]
  1%|▏         | 1.38k/97.5k [00:01<00:51, 1.86kKB/s]
  2%|▏         | 2.00k/97.5k [00:01<00:34, 2.76kKB/s]
  2%|▏         | 2.44k/97.5k [00:01<00:30, 3.11kKB/s]
  4%|▎         | 3.63k/97.5k [00:01<00:18, 5.14kKB/s]
  5%|▍         | 4.62k/97.5k [00:01<00:14, 6.33kKB/s]
  7%|▋         | 7.03k/97.5k [00:01<00:08, 10.4kKB/s]
  9%|▊         | 8.33k/97.5k [00:01<00:08, 11.0kKB/s]
 11%|█▏        | 11.1k/97.5k [00:02<00:05, 14.7kKB/s]
 13%|█▎        | 12.6k/97.5k [00:02<00:05, 14.3kKB/s]
 16%|█▌        | 15.8k/97.5k [00:02<00:04, 18.9kKB/s]
 18%|█▊        | 17.7k/97.5k [00:02<00:04, 17.2kKB/s]
 22%|██▏       | 21.2k/97.5k [00:02<00:03, 20.2kKB/s]
 24%|██▍       | 23.4k/97.5k [00:02<00:03, 20.6kKB/s]
 28%|██▊       | 27.4k/97.5k [00:02<00:02, 24.4kKB/s]
 31%|███       | 29.9k/97.5k [00:02<00:02, 24.0kKB/s]
 35%|███▍      | 34.1k/97.5k [00:02<00:02, 28.9kKB/s]
 38%|███▊      | 37.3k/97.5k [00:03<00:02, 29.7kKB/s]
 41%|████▏     | 40.3k/97.5k [00:03<00:01, 29.3kKB/s]
 46%|████▋     | 45.3k/97.5k [00:03<00:01, 35.3kKB/s]
 50%|█████     | 48.9k/97.5k [00:03<00:01, 32.7kKB/s]
 56%|█████▌    | 54.2k/97.5k [00:03<00:01, 34.0kKB/s]
 61%|██████    | 59.5k/97.5k [00:03<00:01, 34.8kKB/s]
 65%|██████▍   | 63.0k/97.5k [00:03<00:01, 33.5kKB/s]
 68%|██████▊   | 66.4k/97.5k [00:03<00:01, 27.7kKB/s]
 71%|███████   | 69.3k/97.5k [00:04<00:01, 27.5kKB/s]
 76%|███████▌  | 73.6k/97.5k [00:04<00:00, 30.9kKB/s]
 79%|███████▉  | 76.9k/97.5k [00:04<00:00, 25.9kKB/s]
 82%|████████▏ | 79.6k/97.5k [00:04<00:00, 25.6kKB/s]
 84%|████████▍ | 82.4k/97.5k [00:04<00:00, 25.2kKB/s]
 87%|████████▋ | 85.0k/97.5k [00:04<00:00, 24.9kKB/s]
 90%|████████▉ | 87.5k/97.5k [00:04<00:00, 23.9kKB/s]
 93%|█████████▎| 91.0k/97.5k [00:04<00:00, 25.6kKB/s]
 98%|█████████▊| 95.4k/97.5k [00:05<00:00, 30.0kKB/s]
100%|██████████| 97.5k/97.5k [00:05<00:00, 19.1kKB/s]
------------------------------ Captured log setup ------------------------------
INFO     recommenders.datasets.download_utils:download_utils.py:38 Downloading http://snap.stanford.edu/data/amazon/productGraph/categoryFiles/reviews_Movies_and_TV_5.json.gz
INFO     recommenders.datasets.download_utils:download_utils.py:38 Downloading http://snap.stanford.edu/data/amazon/productGraph/categoryFiles/meta_Movies_and_TV.json.gz
INFO     root:amazon_reviews.py:399 start reviews preprocessing...
INFO     root:amazon_reviews.py:386 start meta preprocessing...
INFO     root:amazon_reviews.py:419 start create instances...
INFO     root:amazon_reviews.py:356 creating item2cate dict
INFO     root:amazon_reviews.py:367 getting sampled data...
INFO     root:amazon_reviews.py:460 start data processing...
INFO     root:amazon_reviews.py:202 data generating...
INFO     root:amazon_reviews.py:79 vocab generating...
INFO     root:amazon_reviews.py:148 start valid negative sampling
INFO     root:amazon_reviews.py:170 start test negative sampling
----------------------------- Captured stdout call -----------------------------
WARNING:tensorflow:From /mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/recommenders/models/deeprec/models/sequential/sli_rec.py:61: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell)`, which is equivalent to this API
------------------------------ Captured log call -------------------------------
WARNING  tensorflow:deprecation.py:50 From /mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/recommenders/models/deeprec/models/sequential/sli_rec.py:61: dynamic_rnn (from tensorflow.python.ops.rnn) is deprecated and will be removed in a future version.
Instructions for updating:
Please use `keras.layers.RNN(cell)`, which is equivalent to this API
________________________ test_sum_component_definition _________________________

sequential_files = ('/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/tests/resources/deeprec/slirec', '/mnt/azureml/cr/j/e49a1d...ab.pkl', '/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/tests/resources/deeprec/slirec/category_vocab.pkl')
deeprec_config_path = PosixPath('/mnt/azureml/cr/j/e49a1dc1154c4ee3b42372de2b064c9c/exe/wd/recommenders/models/deeprec/config')

    @pytest.mark.gpu
    def test_sum_component_definition(sequential_files, deeprec_config_path):
        yaml_file_sum = os.path.join(deeprec_config_path, "sum.yaml")
        data_path, user_vocab, item_vocab, cate_vocab = sequential_files

        # SUM model
        hparams_sum = prepare_hparams(
            yaml_file_sum,
            train_num_ngs=4,
            embed_l2=0.0,
            layer_l2=0.0,
            learning_rate=0.001,
            epochs=1,
            MODEL_DIR=os.path.join(data_path, "model"),
            SUMMARIES_DIR=os.path.join(data_path, "summary"),
            user_vocab=user_vocab,
            item_vocab=item_vocab,
            cate_vocab=cate_vocab,
            need_sample=True,
        )

>       model_sum = SUMModel(hparams_sum, SequentialIterator)
E       NameError: name 'SUMModel' is not defined

tests/unit/recommenders/models/test_deeprec_model.py:324: NameError

How do we replicate the issue?

See PR gate failure: https://github.com/recommenders-team/recommenders/actions/runs/6499567929/job/17653180645?pr=2017

Expected behavior (i.e. solution)

We need to put them back to work with the newest TF

Other Comments

miguelgfierro commented 1 year ago

@Leavingseason do you know a fix for these issues with the new TF?

FYI @SimonYansenZhao @loomlike @anargyri

miguelgfierro commented 1 year ago

Possible solution: https://github.com/recommenders-team/recommenders/pull/1988/commits/18c89aad1867db98717d5ada89cba7c4430d565c

We would need to add these versions to the setup

2.8.4
2.9.3
>=2.10.1
"tensorflow==2.8.4,==2.9.3,>=2.10.1",