Open kierzniak opened 2 years ago
Interesting edge case @kierzniak. I am not sure why CVAT returns an error once nuclio-local-storage-reader
is down, because nuclio-local-storage-reader
is used for function deployment and status tracking.
The 6h is hard-coded, but again, not sure how it effects CVAT in term of cancelling the procedure entirely.
In CVAT logs I'm just getting "500 Server Error: Internal Server Error for url: http://nuclio:8070/api/function_invocations" maybe CVAT should try execute function again instead of stopping process. I will open issue in CVAT repo.
Interesting. I've been trying to reproduce but no success yet. Nuclio dashboard ensure container is running before execute any commands over nuclio-local-storage-reader
.
Perhaps it has been fixed by latest versions. Id ask you check against 1.8.8 and see? anyway, introducing features on 1.5.x is something we will not do in the foreseen future.
Hi, I'm using Nucltio wih CVAT and I have a problem with automatic annotation using nuclio + yolo-v3-tf. Nuclio is deployed to dedicated server using Docker.
The problem is that my 1.5h video can't finish annotating. That long video should be processed in about 9h but after 6h I get an error that taks can't be finished.
After investigating logs this message caught my attention: "Failed to read functions from a local store". Apparently "nuclio-local-storage-reader" container is restarting after 6h Nuclio is returning error to CVAT and CVAT is stopping process. How I can reconfigure "nuclio-local-storage-reader" to restart after e.g. 24h? Or maybe problem is with CVAT which should handle this error differently?