equinor / flotilla

Flotilla is the main point of access for operators to interact with multiple robots in a facility.
Eclipse Public License 2.0
16 stars 38 forks source link

Localization mission not added, because of lingering missions in Mission Queue #1446

Closed tsundvoll closed 8 months ago

tsundvoll commented 8 months ago

Describe the bug The robot was available and not localized. There was a lingering mission in the Mission Queue (for reasons described a bit below). Then two new missions was added to the Mission Queue and nothing happened (no localization mission was started and the flow seems frozen) and the robot remains Available and currentArea: null.

From the logs in backend:

2024-03-01 16:02:49 - warn: Api.Services.MapService[0]
      => SpanId:bba25d7e3bfeac95, TraceId:ad972f88ef6dd7917953b77b359c04fc, ParentId:0000000000000000 => ConnectionId:0HN1PVM2QHN8V => RequestPath:/missions RequestId:0HN1PVM2QHN8V:000006F1 => Api.Controllers.MissionSchedulingController.Create (api)
      Unable to find a map for mission '(null)'

To Reproduce Steps to reproduce the behavior:

  1. Make isar-robot fail every step (manually edit the code in robotinterface implementation in isar-robot)
  2. Schedule two Echo mission at a time for isar-robot
  3. Observe that the localization mission fails and the missions in queue are cancelled
  4. (Do this many times until you) Observe that a mission seem to be added to the queue after the localization step was failing and the first mission was cancelled (the second mission by chance took some time to add)

Expected behavior When adding a new mission to the queue; if the robot is not localized and Available, a localization mission should always be started.

Screenshots

image

oysand commented 8 months ago

Worked when scheduling a batch of missions. Then when scheduling a new batch again, the system did not schedule a localization mission and was held up.

oysand commented 8 months ago

Bug was that the _scheduleLocalizationSemaphore did not release after the first MissionService.PendingLocalizationMissionRunExists(missionRun.Robot.Id)) returned true as it returned outside of the try block with finally to release the semaphore.

Added the following to fix it: image