Potential bug

[nilips70]

Hi, I just tried reproducing your results in strats_notebook.ipynb and I got an AttributeError when running:

model, fore_model = build_strats(D, fore_max_len, V, d, N, he, dropout, forecast=True)

Apparently there is a problem with the Transform layer.

[sindhura97]

Hi, can you please paste the error you are getting?

[nilips70]

Running this:

D, fore_max_len, V, d, N, he, dropout = 2, 880, 130, 50, 2, 4, 0.2
model, fore_model =  build_strats(D, fore_max_len, V, d, N, he, dropout, forecast=True)

raises this error:

AttributeError Traceback (most recent call last) /Users/Python codes/STraTS-main/test.ipynb Cell 11 in <cell line: 2>() 1 D, fore_max_len, V, d, N, he, dropout = 2, 880, 10, 50, 2, 4, 0.2 ----> 2 model, fore_model = build_strats(D, fore_max_len, V, d, N, he, dropout, forecast=True)

/Users/Python codes/STraTS-main/test.ipynb Cell 11 in build_strats(D, max_len, V, d, N, he, dropout, forecast) 172 mask = Lambda(lambda x:K.clip(x,0,1))(varis) # b, L 173 # mask = Lambda(lambda x:K.concatenate((K.ones_like(x)[:,0:1], x), axis=-1))(mask) # b, L+1 --> 174 cont_emb = Transformer(N, he, dk=None, dv=None, dff=None, dropout=dropout)(comb_emb, mask=mask) 175 attn_weights = Attention(2d)(cont_emb, mask=mask) 176 fused_emb = Lambda(lambda x:K.sum(x[0]x[1], axis=-2))([cont_emb, attn_weights])

File /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/keras/utils/traceback_utils.py:67, in filter_traceback..error_handler(*args, **kwargs) 65 except Exception as e: # pylint: disable=broad-except 66 filtered_tb = _process_traceback_frames(e.traceback) ---> 67 raise e.with_traceback(filtered_tb) from None 68 finally: 69 del filtered_tb

File /var/folders/3m/_t8llt6n10z5xzvm7vxh37nw0000gp/T/autograph_generated_filerc1iw7kh.py:88, in outer_factory..inner_factory..tfcall(self, x, mask, mask_value) 86 A = ag.Undefined('A') 87 k = ag.Undefined('k') ---> 88 ag.for_stmt(ag__.converted_call(ag.ld(range), (ag__.ld(self).N,), None, fscope), None, loop_body_1, get_state_1, set_state_1, ('A', 'x'), {'iterate_names': 'i'}) 89 try: ...

Call arguments received by layer "transformer_3" (type Transformer): • x=tf.Tensor(shape=(None, 880, 50), dtype=float32) • mask=tf.Tensor(shape=(None, 880), dtype=float32) • mask_value=-1e-30

[sindhura97]

The notebook is running on my end. I just updated the notebook in the repo to the one with the outputs/results.

Did you by chance skip pasting a part of the error trace that shows which line inside the Transformer layer is causing the error? Can you try using tensorflow '1.15.0' of which keras is a part of? Please share your notebook if the error persists.

[Christoper-Harvey]

I am receiving the exact same error as nilips70. Same stack trace. I tried on TF 1.15.0 and TF 2.3

[sindhura97]

Not sure what is causing this. In the above trace, this path "../lib/python3.10/site-packages/keras/" makes it seem like you might be using keras that is not part of the tensorflow package. Have you by chance changed tensorflow.keras in the code to just keras? I can try to take a look at your notebook if you email it to me at sindhut@vt.edu.

[sindhura97]

This might be helpful. Just added a .yml file for the environment I used. If you are using conda, you can go to the directory containing strats.yml and run these commands to create the conda environment and add it to the jupyter notebook. conda env create -f strats.yml conda activate strats conda install -c anaconda ipykernel python -m ipykernel install --user --name=strats

Then, you can open a new notebook with strats environment and run the code here.

[Christoper-Harvey]

Okay so I am running on a windows machine (which might be part of the problem). The yaml you posted only works on linux. strats_universal_yaml_file.txt I updated the yaml to work on all os's.

That being said. I was getting an hd5 error on loading up jupyter when importing tf so I ran: conda install jupyterlab which updated a lot of the ipykernel dependencies which allowed me to run it. I'm not sure what is happening to cause this issue. If @nilips70 is also on a windows machine that might be the problem.

Running the yml file I attached here and installing jupyterlab seems to have fixed the issue and the model is now running.

Thanks, Chris

[nilips70]

@sindhura97 @Christoper-Harvey Thank you both. I am on Mac and I can't setup your suggested configurations on my system. I am getting different errors. Could you please share a configuration with as few as possible dependencies just to get the code working on Google Colab?

[Christoper-Harvey]

@nilips70 if you download and save the 'strats_universal_yaml_file.txt' and rename it as strats.yml and then run:

conda env create -f strats.yml

using the txt file I linked then it should tell you what packages are not available on your system then you just remove those dependencies. Neither of us have macOS so it would be hard for us to tell you what is available on mac.

[nilips70]

After fiddling a lot, I managed to get past that error. Thank you @Christoper-Harvey. The model is really cool, I would have deployed it in Google Colab so everyone could try it without bothering about the configuration issues.

[Aminsn]

Hi All, the the error is caused by the tf_utils.smart_cond function used in the Transformer layer. The function cannot be accessed from the "tensorflow.python.keras.utils" if you are using Tensorflow 2. So if you want to run the code using the newest versions of Tensorflow you should add that function manually (unless the exact function is available in TF2 which I couldn't find). To do that just add the below code to the beginning of the script:

from tensorflow.python.framework import smart_cond as smart_module
from tensorflow.python.ops import control_flow_ops, variables

def smart_cond(pred, true_fn=None, false_fn=None, name=None):
  if isinstance(pred, variables.Variable):
    return control_flow_ops.cond(
        pred, true_fn=true_fn, false_fn=false_fn, name=name)
  return smart_module.smart_cond(
      pred, true_fn=true_fn, false_fn=false_fn, name=name)

and replace all the tf_utils.smart_cond with smart_cond and the script should work with no problem with Tensorflow2. This way you can run it on Google Colab right away without the need for downgrading the already installed Tensorflow.