search

biocore / songbird

Vanilla regression methods for microbiome differential abundance analysis
BSD 3-Clause "New" or "Revised" License
54 stars 25 forks source link

ValueError: initial_value must have a shape specified: Tensor("random_normal:0", shape=(2, ?), dtype=float32) #146

Open BrettLoman opened 3 years ago

BrettLoman commented 3 years ago

Hello,

I am using Songbird stand alone, and it has worked perfectly until I ran into this issue using a very specific .biom file as input. This .biom file is a subset of samples from a "parent" .biom file, but when I use the parent .biom file or other subsets of samples in this same file songbird runs to completion without this error (and I've used it many times at this point).

Any chance of figuring out what the issue is here? (see error message below)

Thank you!!! Brett

OMP: Info #212: KMP_AFFINITY: decoding x2APIC ids. OMP: Info #210: KMP_AFFINITY: Affinity capable, using global cpuid leaf 11 info OMP: Info #154: KMP_AFFINITY: Initial OS proc set respected: 10-29 OMP: Info #156: KMP_AFFINITY: 20 available OS procs OMP: Info #158: KMP_AFFINITY: Nonuniform topology OMP: Info #179: KMP_AFFINITY: 2 packages x 14 cores/pkg x 1 threads/core (20 total cores) OMP: Info #214: KMP_AFFINITY: OS proc to physical thread map: OMP: Info #171: KMP_AFFINITY: OS proc 10 maps to package 0 core 13 OMP: Info #171: KMP_AFFINITY: OS proc 11 maps to package 0 core 14 OMP: Info #171: KMP_AFFINITY: OS proc 12 maps to package 0 core 16 OMP: Info #171: KMP_AFFINITY: OS proc 13 maps to package 0 core 17 OMP: Info #171: KMP_AFFINITY: OS proc 14 maps to package 0 core 18 OMP: Info #171: KMP_AFFINITY: OS proc 15 maps to package 0 core 20 OMP: Info #171: KMP_AFFINITY: OS proc 16 maps to package 0 core 21 OMP: Info #171: KMP_AFFINITY: OS proc 17 maps to package 0 core 22 OMP: Info #171: KMP_AFFINITY: OS proc 18 maps to package 0 core 24 OMP: Info #171: KMP_AFFINITY: OS proc 19 maps to package 0 core 25 OMP: Info #171: KMP_AFFINITY: OS proc 20 maps to package 0 core 26 OMP: Info #171: KMP_AFFINITY: OS proc 21 maps to package 0 core 28 OMP: Info #171: KMP_AFFINITY: OS proc 22 maps to package 0 core 29 OMP: Info #171: KMP_AFFINITY: OS proc 23 maps to package 0 core 30 OMP: Info #171: KMP_AFFINITY: OS proc 24 maps to package 1 core 0 OMP: Info #171: KMP_AFFINITY: OS proc 25 maps to package 1 core 1 OMP: Info #171: KMP_AFFINITY: OS proc 26 maps to package 1 core 2 OMP: Info #171: KMP_AFFINITY: OS proc 27 maps to package 1 core 4 OMP: Info #171: KMP_AFFINITY: OS proc 28 maps to package 1 core 5 OMP: Info #171: KMP_AFFINITY: OS proc 29 maps to package 1 core 6 OMP: Info #250: KMP_AFFINITY: pid 17161 tid 17161 thread 0 bound to OS proc set 10 WARNING:tensorflow:From /gpfs0/export/apps/opt/Songbird/1.0.3/bin/songbird:191: The name tf.Session is deprecated. Please use tf.compat.v1.Session instead.

2020-12-14 10:27:59.621442: I tensorflow/core/platform/cpu_feature_guard.cc:145] This TensorFlow binary is optimized with Intel(R) MKL-DNN to use the following CPU instructions in performance critical operations: SSE4.1 SSE4.2 AVX AVX2 FMA To enable them in non-MKL-DNN operations, rebuild TensorFlow with the appropriate compiler flags. 2020-12-14 10:27:59.668857: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2794880000 Hz 2020-12-14 10:27:59.672470: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x55999e7fdae0 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2020-12-14 10:27:59.673363: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2020-12-14 10:27:59.676186: I tensorflow/core/common_runtime/process_util.cc:115] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance. WARNING:tensorflow:From /gpfs0/export/apps/opt/Songbird/1.0.3/bin/songbird:194: The name tf.set_random_seed is deprecated. Please use tf.compat.v1.set_random_seed instead.

WARNING:tensorflow:From /gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/songbird/multinomial.py:70: multinomial (from tensorflow.python.ops.random_ops) is deprecated and will be removed in a future version. Instructions for updating: Use tf.random.categorical instead. WARNING:tensorflow:From /gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/songbird/multinomial.py:81: The name tf.random_normal is deprecated. Please use tf.random.normal instead.

Traceback (most recent call last): File "/gpfs0/export/apps/opt/Songbird/1.0.3/bin/songbird", line 225, in songbird() File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/click/core.py", line 829, in call return self.main(args, kwargs) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, ctx.params) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(args, kwargs) File "/gpfs0/export/apps/opt/Songbird/1.0.3/bin/songbird", line 196, in multinomial model(session, trainX, trainY, testX, testY) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/songbird/multinomial.py", line 81, in call tf.random_normal([self.p, self.D-1]), name='qB') File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 258, in call return cls._variable_v1_call(*args, kwargs) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 219, in _variable_v1_call shape=shape) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 197, in previous_getter = lambda kwargs: default_variable_creator(None, *kwargs) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/tensorflow_core/python/ops/variable_scope.py", line 2519, in default_variable_creator shape=shape) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 262, in call return super(VariableMetaclass, cls).call(args, kwargs) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 1688, in init shape=shape) File "/gpfs0/export/apps/opt/Songbird/1.0.3/lib/python3.7/site-packages/tensorflow_core/python/ops/variables.py", line 1853, in _init_from_args self._initial_value) ValueError: initial_value must have a shape specified: Tensor("random_normal:0", shape=(2, ?), dtype=float32) Job Statistics for 76837: JobID User Start End Elapsed MaxRSS TotalCPU State Exit NodeList ReqTRES 

       76837     brl003 2020-12-14T10:27:55 2020-12-14T10:28:00   00:00:05             00:04.216     FAILED  1:0   r1pl-hpcf-h03     billing=20,cpu=20,mem=855000M,node=1 
 76837.batch            2020-12-14T10:27:55 2020-12-14T10:28:00   00:00:05          0  00:04.214     FAILED  1:0   r1pl-hpcf-h03                                          
76837.extern            2020-12-14T10:27:55 2020-12-14T10:28:00   00:00:05          0  00:00.001  COMPLETED  0:0   r1pl-hpcf-h03

CPU Efficiency: 4.00% of 00:01:40 core-walltime

mortonjt commented 3 years ago

Hi @BrettLoman thank you for raising this issue. These sorts of issues can arise if an empty dataset is inputted. This can happen for the following reasons