search

microsoft / farmvibes-ai

FarmVibes.AI: Multi-Modal GeoSpatial ML Models for Agriculture and Sustainability
https://microsoft.github.io/farmvibes-ai/
MIT License
680 stars 118 forks source link

Indice's status stays as 'queued' when run for 109 indices #112

Closed click2cloud-Nagaraj closed 12 months ago

click2cloud-Nagaraj commented 1 year ago

Hello team, While running the workflow to compute 109 indices, spaceeye is computed as expected but no other indice comes into running and remain as queued. And we tried cancelling this workflow, which had no effect on the workflow and it stays in running with the timer clicking.

PFA log files, which include :

  1. Orchestrator logs.
  2. Worker logs.
  3. Screenshot for the workflow progress.

109_indices_logs.zip

robertomest commented 1 year ago

Hello @click2cloud-Nagaraj. I have been investigating issues with the status update of very large workflows like this. We have improved our implementation and the cluster should handle scenarios like this much better. The fix will be available on our next release.

rafaspadilha commented 1 year ago

Hi, @click2cloud-Nagaraj. Are you still experiencing this issue?

rafaspadilha commented 12 months ago

Closing this issue for now. @click2cloud-Nagaraj, feel free to reopen it if the problem persists.