Tensorflow issue / GPU?

Hey Phill,

I logged this support ticket with Tensorflow, https://github.com/tensorflow/tensorflow/issues/59419

I seem to be getting it running PCA_nesq and Anomaly_nesq on ARM chip VPS's, but I can run a lot of the PCA classes and Anomaly_profit runs fine too. I just wondered if any of the nesq code needs a GPU to run?

It's a little complicated. Anomaly (base class) is the overall framework for anomaly detection algorithms, and nseq (number of sequential ups/downs) is the approach used to tag buys/sells that are used to train the algorithms. Some of those algorithms (AnomalyDetector_.py) are indeed based on Tensorflow. Which detection algorithm is used is controlled by the variable 'classifiertype' in Anomaly.py (or the specific Anomaly.py subclass). You must have picked up a version where I was playing with the Tensorflow variants. The default in the latest versions is IFOR (Isolation Forest), which has consistently performed the best, and does not use Tensorflow at all. Having said that, it works for me so it's probably a package issue

Can you send me the output that you see please? That should tell me which combination you are using.

Thanks,

Phil

On Tue, Jan 24, 2023 at 5:22 PM tom s @.***> wrote:

Hey Phill,

I logged this support ticket with Tensorflow, tensorflow/tensorflow#59419 https://github.com/tensorflow/tensorflow/issues/59419

I seem to be getting it running PCA_nesq and Anomaly_nesq on ARM chip VPS's, but I can run a lot of the PCA classes and Anomaly_profit runs fine too. I just wondered if any of the nesq code needs a GPU to run?

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/5, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4X57MNODI45ZXVDAFETTWUB56DANCNFSM6AAAAAAUFYIBJY . You are receiving this because you are subscribed to this thread.Message ID: @.***>

This is the latest, it's from a VPS that is running Anomaly_profit with IFOR and that is running fine so I'm struggling to see why one causes the error and one doesn't. I've gone down the path of installing tensorflow_io separately but that doesn't seem to be the cause. If there's nothing that jumps out I'll throw it back at Tensorflow and see if they have any ideas.

E tensorflow/tsl/lib/monitoring/collection_registry.cc:81] Cannot register 2 metrics with the same name: /tensorflow/core/bfc_allocator_delay Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: /home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/init.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths: ['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: caused by: ['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6Status22MaybeAddSourceLocationENS_14SourceLocationE'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}") Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: /home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/init.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths: ['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: caused by: ['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']

On Wed, Jan 25, 2023 at 3:09 PM Nateemma @.***> wrote:

It's a little complicated. Anomaly (base class) is the overall framework for anomaly detection algorithms, and nseq (number of sequential ups/downs) is the approach used to tag buys/sells that are used to train the algorithms. Some of those algorithms (AnomalyDetector_.py) are indeed based on Tensorflow. Which detection algorithm is used is controlled by the variable 'classifiertype' in Anomaly.py (or the specific Anomaly.py subclass). You must have picked up a version where I was playing with the Tensorflow variants. The default in the latest versions is IFOR (Isolation Forest), which has consistently performed the best, and does not use Tensorflow at all. Having said that, it works for me so it's probably a package issue

Can you send me the output that you see please? That should tell me which combination you are using.

Thanks,

Phil

On Tue, Jan 24, 2023 at 5:22 PM tom s @.***> wrote:

Hey Phill,

I logged this support ticket with Tensorflow, tensorflow/tensorflow#59419 https://github.com/tensorflow/tensorflow/issues/59419

I seem to be getting it running PCA_nesq and Anomaly_nesq on ARM chip VPS's, but I can run a lot of the PCA classes and Anomaly_profit runs fine too. I just wondered if any of the nesq code needs a GPU to run?

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/5, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ABD4X57MNODI45ZXVDAFETTWUB56DANCNFSM6AAAAAAUFYIBJY

. You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/5#issuecomment-1402990370, or unsubscribe https://github.com/notifications/unsubscribe-auth/AXXUNNPNRSPY6GCRGDHXISLWUCDMXANCNFSM6AAAAAAUFYIBJY . You are receiving this because you authored the thread.Message ID: @.***>

I’ll take a look tomorrow . The only thing I can think of is that there is some generic code at the top of the files that sets random seeds and logging levels. It should be possible to comment that out in the main files.

You probably have a package version issue, but I don’t understand why it doesn’t affect all of the strategies in the same way…

Cheers

Phil

On Tue, Jan 24, 2023 at 6:38 PM tom s @.***> wrote:

This is the latest, it's from a VPS that is running Anomaly_profit with IFOR and that is running fine so I'm struggling to see why one causes the error and one doesn't. I've gone down the path of installing tensorflow_io separately but that doesn't seem to be the cause. If there's nothing that jumps out I'll throw it back at Tensorflow and see if they have any ideas.

E tensorflow/tsl/lib/monitoring/collection_registry.cc:81] Cannot register 2 metrics with the same name: /tensorflow/core/bfc_allocator_delay Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]:

/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/init.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: caused by:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6Status22MaybeAddSourceLocationENS_14SourceLocationE'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}") Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]:

/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/init.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: caused by:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']

On Wed, Jan 25, 2023 at 3:09 PM Nateemma @.***> wrote:

It's a little complicated. Anomaly (base class) is the overall framework for anomaly detection algorithms, and nseq (number of sequential ups/downs) is the approach used to tag buys/sells that are used to train the algorithms. Some of those algorithms (AnomalyDetector_.py) are indeed based on Tensorflow. Which detection algorithm is used is controlled by the variable 'classifiertype' in Anomaly.py (or the specific Anomaly.py subclass). You must have picked up a version where I was playing with the Tensorflow variants. The default in the latest versions is IFOR (Isolation Forest), which has consistently performed the best, and does not use Tensorflow at all. Having said that, it works for me so it's probably a package issue

Can you send me the output that you see please? That should tell me which combination you are using.

Thanks,

Phil

On Tue, Jan 24, 2023 at 5:22 PM tom s @.***> wrote:

Hey Phill,

I logged this support ticket with Tensorflow, tensorflow/tensorflow#59419 https://github.com/tensorflow/tensorflow/issues/59419

I seem to be getting it running PCA_nesq and Anomaly_nesq on ARM chip VPS's, but I can run a lot of the PCA classes and Anomaly_profit runs fine too. I just wondered if any of the nesq code needs a GPU to run?

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/5, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ABD4X57MNODI45ZXVDAFETTWUB56DANCNFSM6AAAAAAUFYIBJY

. You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub <https://github.com/nateemma/strategies/issues/5#issuecomment-1402990370 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AXXUNNPNRSPY6GCRGDHXISLWUCDMXANCNFSM6AAAAAAUFYIBJY

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/5#issuecomment-1403017973, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4X54DDDVKXA5O2HSPEJLWUCG2PANCNFSM6AAAAAAUFYIBJY . You are receiving this because you commented.Message ID: @.***>

OK, I modified 3 files: Anomaly.py, Anomaly_nseq.py and ClassifierSklearn.py - I removed tensorflow references and modified the nseq criteria a little bit (for other reasons). They should be on github by now.

The other possibility is that this is caused by freqai. You may have to reinstall freqtrade (sh setup.sh -u)

Cheers,

Phil

On Tue, Jan 24, 2023 at 7:57 PM Phil Price @.***> wrote:

I’ll take a look tomorrow . The only thing I can think of is that there is some generic code at the top of the files that sets random seeds and logging levels. It should be possible to comment that out in the main files.

You probably have a package version issue, but I don’t understand why it doesn’t affect all of the strategies in the same way…

Cheers

Phil

On Tue, Jan 24, 2023 at 6:38 PM tom s @.***> wrote:

This is the latest, it's from a VPS that is running Anomaly_profit with IFOR and that is running fine so I'm struggling to see why one causes the error and one doesn't. I've gone down the path of installing tensorflow_io separately but that doesn't seem to be the cause. If there's nothing that jumps out I'll throw it back at Tensorflow and see if they have any ideas.

E tensorflow/tsl/lib/monitoring/collection_registry.cc:81] Cannot register 2 metrics with the same name: /tensorflow/core/bfc_allocator_delay Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]:

/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/init.py:98: UserWarning: unable to load libtensorflow_io_plugins.so: unable to open file: libtensorflow_io_plugins.so, from paths:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: caused by:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io_plugins.so: undefined symbol: _ZN3tsl6Status22MaybeAddSourceLocationENS_14SourceLocationE'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: warnings.warn(f"unable to load libtensorflow_io_plugins.so: {e}") Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]:

/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/init.py:104: UserWarning: file system plugins are not loaded: unable to open file: libtensorflow_io.so, from paths:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so'] Jan 25 01:14:07 instance-20221222-1614 freqtrade_ft7b[319018]: caused by:

['/home/ubuntu/freqtrade/.env/lib/python3.10/site-packages/tensorflow_io/python/ops/libtensorflow_io.so: undefined symbol: _ZTVN10tensorflow13GcsFileSystemE']

On Wed, Jan 25, 2023 at 3:09 PM Nateemma @.***> wrote:

It's a little complicated. Anomaly (base class) is the overall framework for anomaly detection algorithms, and nseq (number of sequential ups/downs) is the approach used to tag buys/sells that are used to train the algorithms. Some of those algorithms (AnomalyDetector_.py) are indeed based on Tensorflow. Which detection algorithm is used is controlled by the variable 'classifiertype' in Anomaly.py (or the specific Anomaly.py subclass). You must have picked up a version where I was playing with the Tensorflow variants. The default in the latest versions is IFOR (Isolation Forest), which has consistently performed the best, and does not use Tensorflow at all. Having said that, it works for me so it's probably a package issue

Can you send me the output that you see please? That should tell me which combination you are using.

Thanks,

Phil

On Tue, Jan 24, 2023 at 5:22 PM tom s @.***> wrote:

Hey Phill,

I logged this support ticket with Tensorflow, tensorflow/tensorflow#59419 https://github.com/tensorflow/tensorflow/issues/59419

I seem to be getting it running PCA_nesq and Anomaly_nesq on ARM chip VPS's, but I can run a lot of the PCA classes and Anomaly_profit runs fine too. I just wondered if any of the nesq code needs a GPU to run?

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/5, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ABD4X57MNODI45ZXVDAFETTWUB56DANCNFSM6AAAAAAUFYIBJY

. You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/nateemma/strategies/issues/5#issuecomment-1402990370>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AXXUNNPNRSPY6GCRGDHXISLWUCDMXANCNFSM6AAAAAAUFYIBJY

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/nateemma/strategies/issues/5#issuecomment-1403017973, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABD4X54DDDVKXA5O2HSPEJLWUCG2PANCNFSM6AAAAAAUFYIBJY . You are receiving this because you commented.Message ID: @.***>

I had a similar problem a few time ago I solved it by changing to CPU usage

import tensorflow as tf
tf.config.set_visible_devices([], 'GPU')

now it works fine and no warning messages imagen

good job friend @nateemma

Interesting, did you change that in all "import tensorflow as tf" references or just those strategy files? I just did global search and there's 293 instances in 244 files in freqtrade. Wondering if I should change them all.

only those that are related to the strategy in classifiers like ClassifierKerasBinary.py In the tensorflow repository they have useful tips 31135

nateemma / strategies

Tensorflow issue / GPU? #5