Test failed due to Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

ShixiangWang commented 2 years ago

Any suggestion?

$ repairsig -c test_data/BRCA_T.conf -J 2 -O output_T
[2021-12-16 14:29:57] INFO:repairsig.resources: Initializing tensors.
[2021-12-16 14:29:57] INFO:repairsig.resources: Successfully read M with shape [3,560,96]
[2021-12-16 14:29:57] INFO:repairsig.resources: Successfully read P with shape [9,96]
[2021-12-16 14:29:57] INFO:repairsig.resources: Tensor initializtion complete.
[2021-12-16 14:29:57] INFO:repairsig.resources: Final tensor shapes are [dimensions with a "+" refer to the fixed (left) and the to be inferred component (right)]:
[2021-12-16 14:29:57] INFO:repairsig.resources: M: [560, 96, 3]
[2021-12-16 14:29:57] INFO:repairsig.resources: P: [9 + 0,  96]
[2021-12-16 14:29:57] INFO:repairsig.resources: W: [9, 3]
[2021-12-16 14:29:57] INFO:repairsig.resources: A: [560, 9]
[2021-12-16 14:29:57] INFO:repairsig.resources: Q: [0 + 2,  96]
[2021-12-16 14:29:57] INFO:repairsig.resources: R: [2, 3]
[2021-12-16 14:29:57] INFO:repairsig.resources: D: [560, 2]
2021-12-16 14:29:57.210000: I tensorflow/compiler/jit/xla_cpu_device.cc:41] Not creating XLA devices, tf_xla_enable_xla_devices not set
2021-12-16 14:29:57.210563: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE4.1 SSE4.2 AVX AVX2 AVX512F FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2021-12-16 14:29:57.215194: I tensorflow/core/common_runtime/process_util.cc:146] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
[2021-12-16 14:29:57] INFO:repairsig.optimizer: Successfully initialized optimizer.
[2021-12-16 14:29:57] INFO:repairsig.optimizer: Model definition complete, commencing optimization
[2021-12-16 14:29:57] INFO:repairsig.optimizer: ### New optimization: iterations 5000 step size 50.0 optimizer adam
Epoch 0 loss 43136.3516
WARNING:tensorflow:AutoGraph could not transform <function Optimizer.optimizeModel.<locals>.doOpt at 0x7f9e51aa6320> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module 'gast' has no attribute 'Index'
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
[2021-12-16 14:29:57] WARNING:tensorflow: AutoGraph could not transform <function Optimizer.optimizeModel.<locals>.doOpt at 0x7f9e51aa6320> and will run it as-is.
Please report this to the TensorFlow team. When filing the bug, set the verbosity to 10 (on Linux, `export AUTOGRAPH_VERBOSITY=10`) and attach the full output.
Cause: module 'gast' has no attribute 'Index'
To silence this warning, decorate the function with @tf.autograph.experimental.do_not_convert
Traceback (most recent call last):
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 447, in converted_call
    converted_f = _convert_actual(target_entity, program_ctx)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 284, in _convert_actual
    transformed, module, source_map = _TRANSPILER.transform(entity, program_ctx)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/pyct/transpiler.py", line 286, in transform
    return self.transform_function(obj, user_context)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/pyct/transpiler.py", line 470, in transform_function
    nodes, ctx = super(PyToPy, self).transform_function(fn, user_context)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/pyct/transpiler.py", line 363, in transform_function
    result = self.transform_ast(node, context)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 252, in transform_ast
    node = self.initial_analysis(node, ctx)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 239, in initial_analysis
    node = qual_names.resolve(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/pyct/qual_names.py", line 252, in resolve
    return QnResolver().visit(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 326, in generic_visit
    value = self.visit(value)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 326, in generic_visit
    value = self.visit(value)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 326, in generic_visit
    value = self.visit(value)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 326, in generic_visit
    value = self.visit(value)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 335, in generic_visit
    new_node = self.visit(old_value)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 335, in generic_visit
    new_node = self.visit(old_value)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/pyct/qual_names.py", line 222, in visit_Attribute
    node = self.generic_visit(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 335, in generic_visit
    new_node = self.visit(old_value)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/ast.py", line 271, in visit
    return visitor(node)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/pyct/qual_names.py", line 232, in visit_Subscript
    if not isinstance(s, gast.Index):
AttributeError: module 'gast' has no attribute 'Index'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/wangsx/miniconda3/envs/methy/bin/repairsig", line 33, in <module>
    sys.exit(load_entry_point('repairsig', 'console_scripts', 'repairsig')())
  File "/home/wangsx/proj/RepairSig/src/repairsig/repairsig.py", line 105, in run
    main(sys.argv[1:])
  File "/home/wangsx/proj/RepairSig/src/repairsig/repairsig.py", line 89, in main
    opt.optimize()
  File "/home/wangsx/proj/RepairSig/src/repairsig/optimizer.py", line 205, in optimize
    self.optimizeModel(self.compute_loss, epoch, stepsize, optimizer)
  File "/home/wangsx/proj/RepairSig/src/repairsig/optimizer.py", line 317, in optimizeModel
    doOpt()
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 828, in __call__
    result = self._call(*args, **kwds)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 871, in _call
    self._initialize(args, kwds, add_initializers_to=initializers)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 726, in _initialize
    *args, **kwds))
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 2969, in _get_concrete_function_internal_garbage_collected
    graph_function, _ = self._maybe_define_function(args, kwargs)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3361, in _maybe_define_function
    graph_function = self._create_graph_function(args, kwargs)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/eager/function.py", line 3206, in _create_graph_function
    capture_by_value=self._capture_by_value),
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 990, in func_graph_from_py_func
    func_outputs = python_func(*func_args, **func_kwargs)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/eager/def_function.py", line 634, in wrapped_fn
    out = weak_wrapped_fn().__wrapped__(*args, **kwds)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/framework/func_graph.py", line 973, in wrapper
    user_requested=True,
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 454, in converted_call
    return _fall_back_unconverted(f, args, kwargs, options, e)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 501, in _fall_back_unconverted
    return _call_unconverted(f, args, kwargs, options)
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/autograph/impl/api.py", line 478, in _call_unconverted
    return f(*args, **kwargs)
  File "/home/wangsx/proj/RepairSig/src/repairsig/optimizer.py", line 275, in doOpt
    while epoch < num_it and cont:
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 885, in __bool__
    self._disallow_bool_casting()
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 489, in _disallow_bool_casting
    "using a `tf.Tensor` as a Python `bool`")
  File "/home/wangsx/miniconda3/envs/methy/lib/python3.7/site-packages/tensorflow/python/framework/ops.py", line 476, in _disallow_when_autograph_enabled
    " indicate you are trying to use an unsupported feature.".format(task))
tensorflow.python.framework.errors_impl.OperatorNotAllowedInGraphError: using a `tf.Tensor` as a Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature.

ShixiangWang commented 2 years ago

The conda list result

$ conda list
# packages in environment at /home/wangsx/miniconda3/envs/methy:
#
# Name                    Version                   Build  Channel
_libgcc_mutex             0.1                        main  
_openmp_mutex             4.5                       1_gnu  
_tflow_select             2.3.0                       mkl  
absl-py                   0.13.0           py37h06a4308_0  
aiohttp                   3.8.1            py37h7f8727e_0  
aiosignal                 1.2.0              pyhd3eb1b0_0  
astor                     0.8.1            py37h06a4308_0  
astunparse                1.6.3                      py_0  
async-timeout             4.0.1              pyhd3eb1b0_0  
asynctest                 0.13.0                     py_0  
attrs                     21.2.0             pyhd3eb1b0_0  
blas                      1.0                         mkl  
blinker                   1.4              py37h06a4308_0  
bottleneck                1.3.2            py37heb32a55_1  
brotli                    1.0.9                he6710b0_2  
brotlipy                  0.7.0           py37h27cfd23_1003  
c-ares                    1.17.1               h27cfd23_0  
ca-certificates           2021.10.26           h06a4308_2  
cachetools                4.2.2              pyhd3eb1b0_0  
certifi                   2021.10.8        py37h06a4308_0  
cffi                      1.14.6           py37h400218f_0  
charset-normalizer        2.0.4              pyhd3eb1b0_0  
click                     8.0.3              pyhd3eb1b0_0  
configargparse            1.4                pyhd3eb1b0_0  
cryptography              3.4.8            py37hd23ed53_0  
cycler                    0.11.0             pyhd3eb1b0_0  
dataclasses               0.8                pyh6d0b6a4_7  
dbus                      1.13.18              hb2f20db_0  
expat                     2.4.1                h2531618_2  
fontconfig                2.13.1               h6c09931_0  
fonttools                 4.25.0             pyhd3eb1b0_0  
freetype                  2.11.0               h70c0345_0  
frozenlist                1.2.0            py37h7f8727e_0  
gast                      0.4.0              pyhd3eb1b0_0  
giflib                    5.2.1                h7b6447c_0  
glib                      2.69.1               h5202010_0  
google-auth               1.33.0             pyhd3eb1b0_0  
google-auth-oauthlib      0.4.4              pyhd3eb1b0_0  
google-pasta              0.2.0              pyhd3eb1b0_0  
grpcio                    1.42.0           py37hce63b2e_0  
gst-plugins-base          1.14.0               h8213a91_2  
gstreamer                 1.14.0               h28cd5cc_2  
h5py                      2.10.0           py37hd6299e0_1  
hdf5                      1.10.6               hb1b8bf9_0  
icu                       58.2                 he6710b0_3  
idna                      3.3                pyhd3eb1b0_0  
importlib-metadata        4.8.2            py37h06a4308_0  
intel-openmp              2021.4.0          h06a4308_3561  
jpeg                      9d                   h7f8727e_0  
keras-preprocessing       1.1.2              pyhd3eb1b0_0  
kiwisolver                1.3.1            py37h2531618_0  
lcms2                     2.12                 h3be6417_0  
ld_impl_linux-64          2.35.1               h7274673_9  
libffi                    3.3                  he6710b0_2  
libgcc-ng                 9.3.0               h5101ec6_17  
libgfortran-ng            7.5.0               ha8ba4b0_17  
libgfortran4              7.5.0               ha8ba4b0_17  
libgomp                   9.3.0               h5101ec6_17  
libpng                    1.6.37               hbc83047_0  
libprotobuf               3.17.2               h4ff587b_1  
libstdcxx-ng              9.3.0               hd4cf53a_17  
libtiff                   4.2.0                h85742a9_0  
libuuid                   1.0.3                h7f8727e_2  
libwebp                   1.2.0                h89dd481_0  
libwebp-base              1.2.0                h27cfd23_0  
libxcb                    1.14                 h7b6447c_0  
libxml2                   2.9.12               h03d6c58_0  
lz4-c                     1.9.3                h295c915_1  
markdown                  3.3.4            py37h06a4308_0  
matplotlib                3.5.0            py37h06a4308_0  
matplotlib-base           3.5.0            py37h3ed280b_0  
mkl                       2021.4.0           h06a4308_640  
mkl-service               2.4.0            py37h7f8727e_0  
mkl_fft                   1.3.1            py37hd3c417c_0  
mkl_random                1.2.2            py37h51133e4_0  
multidict                 5.1.0            py37h27cfd23_2  
munkres                   1.1.4                      py_0  
ncurses                   6.3                  h7f8727e_2  
numexpr                   2.7.3            py37h22e1b3c_1  
numpy                     1.21.2           py37h20f2e39_0  
numpy-base                1.21.2           py37h79a1101_0  
oauthlib                  3.1.1              pyhd3eb1b0_0  
olefile                   0.46                     py37_0  
openssl                   1.1.1l               h7f8727e_0  
opt_einsum                3.3.0              pyhd3eb1b0_1  
packaging                 21.3               pyhd3eb1b0_0  
pandas                    1.3.4            py37h8c16a72_0  
pcre                      8.45                 h295c915_0  
pillow                    8.4.0            py37h5aabda8_0  
pip                       21.2.2           py37h06a4308_0  
protobuf                  3.17.2           py37h295c915_0  
pyasn1                    0.4.8              pyhd3eb1b0_0  
pyasn1-modules            0.2.8                      py_0  
pycparser                 2.21               pyhd3eb1b0_0  
pyjwt                     2.1.0            py37h06a4308_0  
pyopenssl                 21.0.0             pyhd3eb1b0_1  
pyparsing                 3.0.4              pyhd3eb1b0_0  
pyqt                      5.9.2            py37h05f1152_2  
pysocks                   1.7.1                    py37_1  
python                    3.7.11               h12debd9_0  
python-dateutil           2.8.2              pyhd3eb1b0_0  
python-flatbuffers        2.0                pyhd3eb1b0_0  
pytz                      2021.3             pyhd3eb1b0_0  
qt                        5.9.7                h5867ecd_1  
readline                  8.1                  h27cfd23_0  
repairsig                 1.0.post1.dev1+g37204f7.d20211216           dev_0    <develop>
requests                  2.26.0             pyhd3eb1b0_0  
requests-oauthlib         1.3.0                      py_0  
rsa                       4.7.2              pyhd3eb1b0_1  
scipy                     1.7.1            py37h292c36d_2  
setuptools                58.0.4           py37h06a4308_0  
sip                       4.19.8           py37hf484d3e_0  
six                       1.16.0             pyhd3eb1b0_0  
sqlite                    3.36.0               hc218d9a_0  
tensorboard               2.4.0              pyhc547734_0  
tensorboard-plugin-wit    1.6.0                      py_0  
tensorflow                2.4.1           mkl_py37h2d14ff2_0  
tensorflow-base           2.4.1           mkl_py37h43e0292_0  
tensorflow-estimator      2.6.0              pyh7b7c402_0  
termcolor                 1.1.0            py37h06a4308_1  
tk                        8.6.11               h1ccaba5_0  
tornado                   6.1              py37h27cfd23_0  
typing-extensions         3.10.0.2             hd3eb1b0_0  
typing_extensions         3.10.0.2           pyh06a4308_0  
urllib3                   1.26.7             pyhd3eb1b0_0  
werkzeug                  2.0.2              pyhd3eb1b0_0  
wheel                     0.37.0             pyhd3eb1b0_1  
wrapt                     1.13.3           py37h7f8727e_2  
xz                        5.2.5                h7b6447c_0  
yarl                      1.6.3            py37h27cfd23_0  
zipp                      3.6.0              pyhd3eb1b0_0  
zlib                      1.2.11               h7b6447c_3  
zstd                      1.4.9                haebb681_0

drivenbyentropy commented 2 years ago

Thank you for reporting this.

I suspect this is related to this issue. Could you please try downgrading gast to version 0.3.3 and let us know if this fixes the issue?

Thanks!

ShixiangWang commented 2 years ago

@drivenbyentropy Thanks, I will try and see how it works :)

ShixiangWang commented 2 years ago

@drivenbyentropy It works now. Could you put this info to README or dependencies?

Also I have some questions:

Could I input any matrix with different dimension? Or just 96 mutation catalog is valid?
When I specify N or J for numbers of signatures, would it infer the best signature number as the paper describe? I mean if I set 10 to N, if it will find the best value range from 1 to 10.

ShixiangWang commented 2 years ago

Also got an error when run:

$ repairsig -M test_data/BRCA_T_template.txt -O output_T2 -N 10 -J 0
usage: repairsig [-h] -M MATRIX [MATRIX ...] [-P PRIMARY] [-N NUMPRI]
                 [-Q SECONDARY] [-J NUMSEC] [-l] [-d DELIMITER] [-O OUT]
                 [-v {CRITICAL,ERROR,WARNING,INFO,DEBUG}]
                 [-e EPOCHS [EPOCHS ...]] [-s STEPSIZES [STEPSIZES ...]]
                 [-o {adadelta,adagrad,adam,adamax,nadam,rmaprop,sgd} [{adadelta,adagrad,adam,adamax,nadam,rmaprop,sgd} ...]]
                 [-c CONFIG]
repairsig: error: argument -J/--numsec: 0 is an invalid int value. Must be >= 0

ShixiangWang commented 2 years ago

And I cannot understand the rule to set proper values to the following parameters. Could you put some guide in README. Thanks :).

https://github.com/ncbi/RepairSig/blob/37204f7f0e1b34d26b78509be996609cade760f2/test_data/BRCA_T.conf#L8-L10

ncbi / RepairSig

Test failed due to Python `bool` is not allowed: AutoGraph did convert this function. This might indicate you are trying to use an unsupported feature. #1