Describe the bug\
The unit-test fuse.dl.tests.test_cat.TestCrossAttentionTransformerEncoder test_all_contexts randomly fails:
======================================================================
ERROR [2.241s]: test_all_contexts (fuse.dl.tests.test_cat.TestCrossAttentionTransformerEncoder)
test cross attention transformer for each of the three context options: "seq_a", "seq_b" and "both"
----------------------------------------------------------------------
Describe the bug\ The unit-test
fuse.dl.tests.test_cat.TestCrossAttentionTransformerEncoder test_all_contexts
randomly fails:FuseMedML version\ Latest
To reproduce\ ?
Additional context\ Full traceback on slack.