Open seandrews opened 3 years ago
Hello,
This file was written before the lastest Tensorflow addons update, however, it's not much different now..!! Basically, GroupNormalization layer uses "N" groups to normalize its input. The number of groups should be a multiplier of the number of input units, which is 100 in this example. By default, the tensorflow_addons library uses n_groups=32
as a default number of groups in the GroupNormalization layer, which is not a multiplier of 100.
For example, if you use a GRU with 100 units, then acceptable "n_groups" are for example: 2, 5, 10, 20, 25, 50, 100
That being said, you could try to add the following argument:
GroupNormalization(n_groups=25)
This should be working.
Hi,
Thanks for the quick reply... I adjusted units from 25 to 100 and added n_groups=25. Did not appear to work so I looked up GroupNomalization and changed n_groups to groups. There appears to be less errors but still get the following...
Please help.
Thanks Stephen
C:\Users\Stephen\venv\Scripts\python.exe
C:/Users/Stephen/PycharmProjects/Stocks-Prediction/tesla/GRU.py
2021-10-19 00:52:33.515528: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cudart64_110.dll
2021-10-19 00:52:37.473526: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library nvcuda.dll
2021-10-19 00:52:38.674207: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with
properties:
pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1650 Ti computeCapability:
7.5
coreClock: 1.485GHz coreCount: 16 deviceMemorySize: 4.00GiB
deviceMemoryBandwidth: 178.84GiB/s
2021-10-19 00:52:38.674556: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cudart64_110.dll
2021-10-19 00:52:38.684265: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cublas64_11.dll
2021-10-19 00:52:38.684464: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cublasLt64_11.dll
2021-10-19 00:52:38.690455: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cufft64_10.dll
2021-10-19 00:52:38.692069: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library curand64_10.dll
2021-10-19 00:52:38.695225: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cusolver64_11.dll
2021-10-19 00:52:38.698834: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cusparse64_11.dll
2021-10-19 00:52:38.699796: I
tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully
opened dynamic library cudnn64_8.dll
2021-10-19 00:52:38.700025: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu
devices: 0
2021-10-19 00:52:38.700396: I
tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary
is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the
following CPU instructions in performance-critical operations: AVX AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate
compiler flags.
2021-10-19 00:52:38.701888: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with
properties:
pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1650 Ti computeCapability:
7.5
coreClock: 1.485GHz coreCount: 16 deviceMemorySize: 4.00GiB
deviceMemoryBandwidth: 178.84GiB/s
2021-10-19 00:52:38.702269: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu
devices: 0
2021-10-19 00:52:39.167333: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect
StreamExecutor with strength 1 edge matrix:
2021-10-19 00:52:39.167527: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0
2021-10-19 00:52:39.167632: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N
2021-10-19 00:52:39.167881: I
tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow
device (/job:localhost/replica:0/task:0/device:GPU:0 with 2143 MB memory)
-> physical GPU (device: 0, name: NVIDIA GeForce GTX 1650 Ti, pci bus id:
0000:01:00.0, compute capability: 7.5)
WARNING:tensorflow:Please add keras.layers.InputLayer
instead of
keras.Input
to Sequential model. keras.Input
is intended to be used by
Functional model.
Traceback (most recent call last):
File "C:\Users\Stephen\PycharmProjects\Stocks-Prediction\tesla\GRU.py",
line 34, in
On Tue, Oct 19, 2021 at 12:26 AM Vasileios Kochliaridis < @.***> wrote:
Hello,
This file was written before the lastest Tensorflow addons update! Basically, GroupNormalization layer uses "N" groups to normalize its input. The number of groups should be a multiplier of the number of input units. By default, the tensorflow_addons library uses n_groups=32 as a default number of groups in the GroupNormalization layer, which is not a multiplier of 100.
For example, if you use a GRU with 100 units, then acceptable "n_groups" are for example: 2, 5, 10, 20, 25, 50, 100
That being said, you could try to add the following argument:
GroupNormalization(n_groups=25)
This should be working.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kochlisGit/Stocks-Prediction/issues/1#issuecomment-945670133, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHSCKLCLYZMBSZSQHMRWA2DUHP767ANCNFSM5GGJB5QQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
It looks like there is a mismatch error with the version of keras and tensorflow_addons modules that you have installed in your device. The best solution here is to update both tensorflow & tensorflow_addons versions in your machine.
However, If this is not possible, try replacing Yogi with "adam" algorithm
optimizers='adam'
, which is supported just fine!
Google's YOGI is based on adam's algorithm and it's only 1 line of code that it differs from the implementation of adam, so there won't be much difference anyway.
Hi Vasileios,
Finally got it to go... created a new environment and reloaded all the components. Then I found that I needed to download the CUDNN .dll to put in the CUDA Toolkit directory.
Now everything appears to work fine.
Thanks Stephen
On Tue, Oct 19, 2021 at 1:59 AM Vasileios Kochliaridis < @.***> wrote:
It looks like there is a mismatch error with the version of keras and tensorflow_addons modules that you have installed in your device. The best solution here is to update both tensorflow & tensorflow_addons versions in your machine.
However, If this is not possible, try replacing Yogi with "adam" algorithm optimizers='adam' , which is supported just fine! Google's YOGI is based on adam's algorithm and it's only 1 line of code that it differs from the implementation of adam, so there won't be much difference anyway.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/kochlisGit/Stocks-Prediction/issues/1#issuecomment-945740353, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHSCKLD2RFD6NKHLUT5Z3QTUHQK27ANCNFSM5GGJB5QQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
C:\Users\Stephen\venv\Scripts\python.exe C:/Users/Stephen/PycharmProjects/Stocks-Prediction/tesla/BGRU_LSTM.py 2021-10-18 23:27:59.979789: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll 2021-10-18 23:28:04.304261: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library nvcuda.dll 2021-10-18 23:28:05.505473: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1650 Ti computeCapability: 7.5 coreClock: 1.485GHz coreCount: 16 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 178.84GiB/s 2021-10-18 23:28:05.505823: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudart64_110.dll 2021-10-18 23:28:05.514343: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublas64_11.dll 2021-10-18 23:28:05.514544: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cublasLt64_11.dll 2021-10-18 23:28:05.520595: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cufft64_10.dll 2021-10-18 23:28:05.522458: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library curand64_10.dll 2021-10-18 23:28:05.525636: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusolver64_11.dll 2021-10-18 23:28:05.529390: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cusparse64_11.dll 2021-10-18 23:28:05.530409: I tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library cudnn64_8.dll 2021-10-18 23:28:05.530640: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0 2021-10-18 23:28:05.531024: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX AVX2 To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. 2021-10-18 23:28:05.532899: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1733] Found device 0 with properties: pciBusID: 0000:01:00.0 name: NVIDIA GeForce GTX 1650 Ti computeCapability: 7.5 coreClock: 1.485GHz coreCount: 16 deviceMemorySize: 4.00GiB deviceMemoryBandwidth: 178.84GiB/s 2021-10-18 23:28:05.533461: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1871] Adding visible gpu devices: 0 2021-10-18 23:28:05.990579: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1258] Device interconnect StreamExecutor with strength 1 edge matrix: 2021-10-18 23:28:05.990779: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1264] 0 2021-10-18 23:28:05.990884: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1277] 0: N 2021-10-18 23:28:05.991112: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1418] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 2143 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce GTX 1650 Ti, pci bus id: 0000:01:00.0, compute capability: 7.5) WARNING:tensorflow:Please add
model = rnn_model.build_model(input_shape, layers)
File "C:\Users\Stephen\PycharmProjects\Stocks-Prediction\tesla\rnn_model.py", line 12, in build_model
model.add(layer)
File "C:\Users\Stephen\venv\lib\site-packages\tensorflow\python\training\tracking\base.py", line 522, in _method_wrapper
result = method(self, *args, kwargs)
File "C:\Users\Stephen\venv\lib\site-packages\keras\engine\sequential.py", line 223, in add
output_tensor = layer(self.outputs[0])
File "C:\Users\Stephen\venv\lib\site-packages\keras\engine\base_layer.py", line 945, in call
return self._functional_construction_call(inputs, args, kwargs,
File "C:\Users\Stephen\venv\lib\site-packages\keras\engine\base_layer.py", line 1083, in _functional_construction_call
outputs = self._keras_tensor_symbolic_call(
File "C:\Users\Stephen\venv\lib\site-packages\keras\engine\base_layer.py", line 816, in _keras_tensor_symbolic_call
return self._infer_output_signature(inputs, args, kwargs, input_masks)
File "C:\Users\Stephen\venv\lib\site-packages\keras\engine\base_layer.py", line 856, in _infer_output_signature
outputs = call_fn(inputs, *args, *kwargs)
File "C:\Users\Stephen\venv\lib\site-packages\keras\engine\functional.py", line 1442, in call
return getattr(self._module, self._method_name)(args, kwargs)
File "C:\Users\Stephen\venv\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 1023, in call
self._maybe_build(inputs)
File "C:\Users\Stephen\venv\lib\site-packages\tensorflow\python\keras\engine\base_layer.py", line 2625, in _maybe_build
self.build(input_shapes) # pylint:disable=not-callable
File "C:\Users\Stephen\venv\lib\site-packages\tensorflow_addons\layers\normalizations.py", line 110, in build
self._check_size_of_dimensions(input_shape)
File "C:\Users\Stephen\venv\lib\site-packages\tensorflow_addons\layers\normalizations.py", line 236, in _check_size_of_dimensions
raise ValueError(
ValueError: Number of groups (32) must be a multiple of the number of channels (100).
keras.layers.InputLayer
instead ofkeras.Input
to Sequential model.keras.Input
is intended to be used by Functional model. Traceback (most recent call last): File "C:\Users\Stephen\PycharmProjects\Stocks-Prediction\tesla\BGRU_LSTM.py", line 32, in