MycroftAI / mycroft-precise

A lightweight, simple-to-use, RNN wake word listener
Apache License 2.0
842 stars 227 forks source link

Test after train.. problem... #208

Open daniele777 opened 3 years ago

daniele777 commented 3 years ago

TRAIN :

(.venv) dani@NB-Daniele:/mnt/c/Users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev$ precise-train -e 50 tars- start.net tars-start/ Using TensorFlow backend. WARNING:tensorflow:From /mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/.venv/lib/python3.7/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version. Instructions for updating: Colocations handled automatically by placer. WARNING:tensorflow:From /mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/.venv/lib/python3.7/site-packages/keras/backend/tensorflow_backend.py:3138: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version. Instructions for updating: Please use rate instead of keep_prob. Rate should be set to rate = 1 - keep_prob. Data: Loading wake-word... Loading not-wake-word... Loading wake-word... Loading not-wake-word... Inputs shape: (10, 29, 13) Outputs shape: (10, 1) Test inputs shape: (5, 29, 13) Test outputs shape: (5, 1)


Layer (type) Output Shape Param #

net (GRU) (None, 20) 2040


dense_1 (Dense) (None, 1) 21

Total params: 2,061 Trainable params: 2,061 Non-trainable params: 0


Train on 10 samples, validate on 5 samples 2021-05-20 11:51:48.868410: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2021-05-20 11:51:48.933481: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz 2021-05-20 11:51:48.938174: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x349df40 executing computations on platform Host. Devices: 2021-05-20 11:51:48.939065: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): , Epoch 1/50 10/10 [==============================] - 0s 40ms/step - loss: 0.1834 - acc: 0.6000 - val_loss: 0.1735 - val_acc: 0.8000 Epoch 2/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0476 - acc: 0.9000 - val_loss: 0.1760 - val_acc: 0.8000 Epoch 3/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0932 - acc: 0.9000 - val_loss: 0.1251 - val_acc: 0.8000 Epoch 4/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0419 - acc: 1.0000 - val_loss: 0.1146 - val_acc: 1.0000 Epoch 5/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0231 - acc: 1.0000 - val_loss: 0.1090 - val_acc: 1.0000 Epoch 6/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0531 - acc: 1.0000 - val_loss: 0.0883 - val_acc: 1.0000 Epoch 7/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0837 - acc: 0.9000 - val_loss: 0.0742 - val_acc: 1.0000 Epoch 8/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0532 - acc: 0.9000 - val_loss: 0.0794 - val_acc: 1.0000 Epoch 9/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0093 - acc: 1.0000 - val_loss: 0.0780 - val_acc: 1.0000 Epoch 10/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0587 - acc: 1.0000 - val_loss: 0.0596 - val_acc: 1.0000 Epoch 11/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0259 - acc: 1.0000 - val_loss: 0.0517 - val_acc: 1.0000 Epoch 12/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0685 - acc: 1.0000 - val_loss: 0.0441 - val_acc: 1.0000 Epoch 13/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0641 - acc: 0.9000 - val_loss: 0.0345 - val_acc: 1.0000 Epoch 14/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0512 - acc: 0.9000 - val_loss: 0.0317 - val_acc: 1.0000 Epoch 15/50 10/10 [==============================] - 0s 3ms/step - loss: 0.1730 - acc: 0.9000 - val_loss: 0.0256 - val_acc: 1.0000 Epoch 16/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0216 - acc: 1.0000 - val_loss: 0.0237 - val_acc: 1.0000 Epoch 17/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0129 - acc: 1.0000 - val_loss: 0.0229 - val_acc: 1.0000 Epoch 18/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0090 - acc: 1.0000 - val_loss: 0.0222 - val_acc: 1.0000 Epoch 19/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0567 - acc: 0.8000 - val_loss: 0.0219 - val_acc: 1.0000 Epoch 20/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0516 - acc: 1.0000 - val_loss: 0.0192 - val_acc: 1.0000 Epoch 21/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0307 - acc: 0.9000 - val_loss: 0.0187 - val_acc: 1.0000 Epoch 22/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0372 - acc: 0.9000 - val_loss: 0.0186 - val_acc: 1.0000 Epoch 23/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0240 - acc: 1.0000 - val_loss: 0.0182 - val_acc: 1.0000 Epoch 24/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0027 - acc: 1.0000 - val_loss: 0.0181 - val_acc: 1.0000 Epoch 25/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0155 - acc: 1.0000 - val_loss: 0.0158 - val_acc: 1.0000 Epoch 26/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0240 - acc: 1.0000 - val_loss: 0.0147 - val_acc: 1.0000 Epoch 27/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0085 - acc: 1.0000 - val_loss: 0.0142 - val_acc: 1.0000 Epoch 28/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0147 - acc: 1.0000 - val_loss: 0.0141 - val_acc: 1.0000 Epoch 29/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0087 - acc: 1.0000 - val_loss: 0.0142 - val_acc: 1.0000 Epoch 30/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0301 - acc: 0.9000 - val_loss: 0.0152 - val_acc: 1.0000 Epoch 31/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0170 - acc: 1.0000 - val_loss: 0.0144 - val_acc: 1.0000 Epoch 32/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0180 - acc: 1.0000 - val_loss: 0.0136 - val_acc: 1.0000 Epoch 33/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0093 - acc: 1.0000 - val_loss: 0.0129 - val_acc: 1.0000 Epoch 34/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0286 - acc: 1.0000 - val_loss: 0.0097 - val_acc: 1.0000 Epoch 35/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0058 - acc: 1.0000 - val_loss: 0.0093 - val_acc: 1.0000 Epoch 36/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0264 - acc: 1.0000 - val_loss: 0.0070 - val_acc: 1.0000 Epoch 37/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0016 - acc: 1.0000 - val_loss: 0.0070 - val_acc: 1.0000 Epoch 38/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0035 - acc: 1.0000 - val_loss: 0.0068 - val_acc: 1.0000 Epoch 39/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0049 - acc: 1.0000 - val_loss: 0.0065 - val_acc: 1.0000 Epoch 40/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0139 - acc: 1.0000 - val_loss: 0.0056 - val_acc: 1.0000 Epoch 41/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0062 - acc: 1.0000 - val_loss: 0.0055 - val_acc: 1.0000 Epoch 42/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0028 - acc: 1.0000 - val_loss: 0.0052 - val_acc: 1.0000 Epoch 43/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0156 - acc: 1.0000 - val_loss: 0.0052 - val_acc: 1.0000 Epoch 44/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0195 - acc: 1.0000 - val_loss: 0.0047 - val_acc: 1.0000 Epoch 45/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0027 - acc: 1.0000 - val_loss: 0.0043 - val_acc: 1.0000 Epoch 46/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0030 - acc: 1.0000 - val_loss: 0.0041 - val_acc: 1.0000 Epoch 47/50 10/10 [==============================] - 0s 2ms/step - loss: 7.9729e-04 - acc: 1.0000 - val_loss: 0.0041 - val_acc: 1.0000 Epoch 48/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0061 - acc: 1.0000 - val_loss: 0.0041 - val_acc: 1.0000 Epoch 49/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0025 - acc: 1.0000 - val_loss: 0.0038 - val_acc: 1.0000 Epoch 50/50 10/10 [==============================] - 0s 2ms/step - loss: 0.0021 - acc: 1.0000 - val_loss: 0.0039 - val_acc: 1.0000

TEST (.venv) dani@NB-Daniele:/mnt/c/Users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev$ precise-test tars-start.net tars-start/ Loading wake-word... Loading not-wake-word... 2021-05-20 11:53:05.242040: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2021-05-20 11:53:05.248528: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 2904000000 Hz 2021-05-20 11:53:05.252104: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x43832f0 executing computations on platform Host. Devices: 2021-05-20 11:53:05.252860: I tensorflow/compiler/xla/service/service.cc:158] StreamExecutor device (0): , Using TensorFlow backend. Traceback (most recent call last): File "/mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/.venv/bin/precise-test", line 33, in sys.exit(load_entry_point('mycroft-precise', 'console_scripts', 'precise-test')()) File "/mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/precise/scripts/base_script.py", line 49, in run_main script.run() File "/mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/precise/scripts/test.py", line 52, in run predictions = Listener.find_runner(args.model)(args.model).predict(inputs) File "/mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/precise/network_runner.py", line 85, in init self.model = load_precise_model(model_name) File "/mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/precise/model.py", line 54, in load_precise_model return load_keras().models.load_model(model_name) File "/mnt/c/users/daniele.angelini/mycroft-precise-dev/mycroft-precise-dev/.venv/lib/python3.7/site-packages/keras/models.py", line 242, in load_model model_config = json.loads(model_config.decode('utf-8')) AttributeError: 'str' object has no attribute 'decode'

Result of test sholuld be ok .???

JoySkipper commented 3 years ago

So I know this is a bit old but I'm having the same issue. Looks to be for me (as far as my amateur mind can tell) because precise only works for python versions <=3.6 and my raspberry pi 4 came with python 3.7 installed and thus built the venv on 3.7.

Thus, str.decode() which is a python2 skeleton that was finally dropped in 3.7, so it no longer is necessary and breaks the code. Removing the '.decode('utf-8') fixes that line, but then I run into the same problem again...and again...etc.

So instead of updating my entire local repo to be compatible with python 3.7 which could be an endless rabbit hole that'd break as soon as there's another update, I tried to re-clone the repo and run setup.sh using a build of python3.6 that I'd put onto my pi instead. I'll update here if this is successful.

Looks like the dev is working on uploading the tensorflow backend at the moment (the source of all the warnings in the top). Hopefully their next step would be to make precise compatible with 3.7? I'm tempted to try and do so myself but it's probably a lot hairier to do than it appears at first glance and I'm afraid my hunches are way off base. @krisgesling am I on the right track or do I have no idea what I'm doing? I don't mind working on updating compatibility to 3.7 to help out if I'm on the right track here, but I'm not confident enough that I am at the moment and don't want to put all that work in if it doesn't make sense or isn't actually the source of the problem.

JoySkipper commented 3 years ago

Update on building with Python 3.6. It worked, but it did take quite a bit of fiddling to get working. Mainly the issue revolved around installing the dependencies such as scipy which was weirdly difficult. I had to install a lot of scipy's dependencies by hand even though pip should have taken care of that for me.

Anyways, after installing precise I had to upgrade tensorflow to 1.14.0 even though it warns that precise only works for tensorflow as 1.13. When I ran it with 1.13 I got this error and didn't have the know-how to fix it :

https://github.com/tensorflow/tensorflow/issues/23386#issuecomment-434752741

I basically upgraded to 1.14 as a blind guess and it fixed the issue I was having. Point is precise training, testing, and listening is working now for me so perhaps I had the right idea with the python version issue.

el-tocino commented 3 years ago

You can use the latest tf1.x version for precise (.3, at any rate)