search

johnolafenwa / DeepStack

The World's Leading Cross Platform AI Engine for Edge Devices
Apache License 2.0
675 stars 105 forks source link

WIN Version of Deepstack- Timeout error #77

Closed balucanb closed 3 years ago

balucanb commented 3 years ago

John Been trying to figure out what is causing this error- HTTPClient timeout at 65 seconds (Max=65 set in 'HTTPClientTimeoutSeconds' in aitool.settings.json): A task was canceled. [TaskCanceledException] Mod: d__48 Line:1575

Been working with the AI-Tool guys but I think it is coming from the Deepstack side? I am running the newest version CPU.2021.01.beta

Tinbum1 commented 3 years ago

My Deepstack windows GPU (same version number) has been running pretty good but yesterday it started to go wrong as well. It seems to run for a few minutes than stops.

Deepstack returned 'error occured on the server - RuntimeError('CUDA the launch timed out and was terminated')' - http status code 'InternalServerError' (500) in 203ms: Internal Server Error

Traceback (most recent call last): File "C:\DeepStack\intelligencelayer\shared\detection.py", line 96, in objectdetection det = detector.predict(img_path, threshold) File "C:\DeepStack\intelligencelayer\shared.\process.py", line 61, in predict pred, confidence, 0.45, classes=None, agnostic=False File "C:\DeepStack\intelligencelayer\shared.\utils\general.py", line 794, in non_max_suppression x = x[xc[xi]] # confidence RuntimeError: copy_if failed to synchronize: cudaErrorLaunchTimeout: the launch timed out and was terminated

Traceback (most recent call last): File "C:\DeepStack\intelligencelayer\shared\detection.py", line 96, in objectdetection det = detector.predict(img_path, threshold) File "C:\DeepStack\intelligencelayer\shared.\process.py", line 52, in predict img = torch.from_numpy(img).to(self.device) RuntimeError: CUDA error: the launch timed out and was terminated

Tinbum1 commented 3 years ago

Turned out mine was because i have added some cameras to the Nvidia GPU and it just couldn't cope even though all the memory and utilization figures looked ok.

johnolafenwa commented 3 years ago

Hello @balucanb and @Tinbum1 , the timeout can be due to many issues, most of which have been fixed in the latest update, see https://forum.deepstack.cc/t/deepstack-february-2021-release-fixes-for-jetson-windows-native-and-docker-versions/724

If this issue continues after installing the latest versions above, please reopen this issue.