Open sushil-bharati opened 4 years ago
Here is the error report:
/usr/local/lib/python3.6/dist-packages/numpy/core/_methods.py:26: RuntimeWarning: invalid value encountered in reduce return umr_maximum(a, axis, None, out, keepdims) /usr/local/lib/python3.6/dist-packages/numpy/core/_methods.py:29: RuntimeWarning: invalid value encountered in reduce return umr_minimum(a, axis, None, out, keepdims) {"simapp_exception": {"version": "1.0", "date": "2019-08-30 04:29:37.384401", "function": "training_worker", "message": "NaN detected in loss function, aborting training. Job failed!", "exceptionType": "training_worker.exceptions", "eventType": "system_error", "errorCode": "503"}}
This problem occurs when reward is too low!? This is what I've found after some play around. Cannot be concluded.
Here is the error report: