Have troubles running test for test_smart_clutrr.py

[alex4321]

Python: 3.8.5
Torch: 1.7.1
Cuda: 11.0 (yet it was disabled through CUDA_VISIBLE_DEVICES)

I got few problems with data types (torch Embeddings required LongTensor instead of IntTensor). Yet when I specified tensor types explicitly - you can see https://github.com/alex4321/ctp - I got problems running test_smart_clutrr_v5 in test_smart_clutrr.py:

(lines 515-520)

np.testing.assert_allclose(inf0_np, [0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], rtol=1e-1, atol=1e-1)
np.testing.assert_allclose(inf1_np, [1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], rtol=1e-1, atol=1e-1)
np.testing.assert_allclose(inf2_np, [1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], rtol=1e-1, atol=1e-1)
np.testing.assert_allclose(inf3_np, [1, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0], rtol=1e-1, atol=1e-1)
np.testing.assert_allclose(inf4_np, [1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0], rtol=1e-1, atol=1e-1)

But I got next values:

INF0 tensor([3.5844e-08, 1.0600e-08, 6.5727e-09, 1.8932e-08, 7.4809e-09, 7.8440e-09,
        8.1592e-09, 2.4311e-08, 1.3718e-08, 4.8176e-07, 1.1065e-08, 2.0157e-07])
INF1 tensor([9.9900e-01, 1.3420e-06, 1.2384e-06, 3.3722e-06, 1.8620e-06, 6.9090e-06,
        1.5691e-06, 4.7638e-06, 4.3208e-06, 1.7816e-06, 1.1471e-06, 4.2599e-07])
INF2 tensor([9.9900e-01, 7.9380e-06, 1.4266e-06, 3.9714e-06, 3.3722e-06, 9.3712e-06,
        6.9090e-06, 4.7638e-06, 5.5142e-06, 2.6228e-06, 1.9439e-06, 1.1219e-06])
INF3 tensor([9.9900e-01, 7.9380e-06, 2.6228e-06, 5.5142e-06, 3.3722e-06, 9.3712e-06,
        6.9090e-06, 4.7638e-06, 5.5142e-06, 2.6228e-06, 1.9439e-06, 1.3885e-06])
INF4 tensor([9.9900e-01, 7.9380e-06, 2.6228e-06, 5.5142e-06, 3.9714e-06, 9.3712e-06,
        6.9090e-06, 4.7638e-06, 5.5142e-06, 2.6228e-06, 1.9439e-06, 1.3885e-06])

So only first two assertions passes.

(test_classic.py works fine with the same data, so I guess it should be some problem with kbcr.clutrr.models.smart.NeuralKB, kbcr.clutrr.models.smart.Hoppy? Also, previous tests test_smart_clutrr_v1 - test_smart_clutrr_v4 passed succesfully too)

[alex4321]

Also, I'm getting error with tests\test_cli.py:

@pytest.mark.light
    def test_complex_cli_v1():
        env = os.environ.copy()
        env['PYTHONPATH'] = '.'

        cmd = [
            sys.executable,
            './bin/kbc-cli.py',
            '--train', 'data/wn18rr/dev.tsv',
            '--dev', 'data/wn18rr/dev.tsv',
            '--test', 'data/wn18rr/test.tsv',
            '-m', 'complex',
            '-k', '100',
            '-b', '100',
            '-e', '1',
            '--N3', '0.0001',
            '-l', '0.1',
            '-V', '1',
            '-o', 'adagrad',
            '-B', '2000'
        ]

        p = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE, cwd=os.getcwd())
        out, err = p.communicate()
        out_str = out.decode("utf-8")
        err_str = err.decode("utf-8")
        assert p.returncode == 0, f"Process exited with {p.returncode} code\nOutput: {out_str}\nError: {err_str}"

        sys.stdout = sys.stderr

        lines = out_str.split("\n")

        sanity_check_flag_1 = False

        for line in lines:
            if 'Batch 1/31' in line:
                value = float(line.split()[5])
                np.testing.assert_allclose(value, 18.311768, atol=1e-3, rtol=1e-3)
            if 'Batch 10/31' in line:
                value = float(line.split()[5])
                np.testing.assert_allclose(value, 18.273418, atol=1e-3, rtol=1e-3)
            if 'Final' in line and 'dev results' in line:
                value = float(line.split()[4])
>               np.testing.assert_allclose(value, 0.116451, atol=1e-3, rtol=1e-3)
E               AssertionError: 
E               Not equal to tolerance rtol=0.001, atol=0.001
E               
E               Mismatched elements: 1 / 1 (100%)
E               Max absolute difference: 0.312262
E               Max relative difference: 2.68148835
E                x: array(0.428713)
E                y: array(0.116451)

tests\test_cli.py:55: AssertionError

All other test passes