search

afrl-rq / OpenUxAS-SoI

Project for multi-UAV cooperative decision making
Other
71 stars 38 forks source link

Cannot handle more than 15 tasks #61

Open kfu0115 opened 6 years ago

kfu0115 commented 6 years ago

I am running a performance analysis. I have two vehicles and point search tasks. It take about a minute to for task assignment when I have 15 point search tasks. But when I have 16 tasks, it takes forever to run (more than 24 hours and I don't think it's actually doing anything at the time, could be an memory allocation issue) Do you know why is that the case? is there a way I can debug it? (I am using visual studio)

Here is an example: image

If you can't see the screenshot, basically the screen just stays idle. I think it went into that state after executing a line in either RouteAggregator or AssignmentTreeBranchandBound.

Thanks

jonsteingass commented 6 years ago

If you're interested in debugging UxAS with Visual Studio you can attach Visual Studio to a running process after launching UxAS, or you can add the command line arguments to the uxas project through its properties page.

To attach a process to Visual Studio (UxAS must be running):

To add command line arguments in Visual Studio:

Let me know if this helps. I can add pictures if that's more useful.

kfu0115 commented 6 years ago

I've been debugging using Visual Studio. Attaching to a process will be too slow to catch the steps before it went into the infinite running state. What I do to debug the process is change replacing the scenario files. And in my automation request message, I gradually add more points to the scenario. (also, I need to change the time running UxAS) I created some tracer to monitor all functions in RouteAggregator and AssignmentTreeBranchAnndBound. It seems it never when to the stage of "BuildMatrixRequests"

On Tue, May 29, 2018 at 7:20 AM, jonsteingass notifications@github.com wrote:

If you're interested in debugging UxAS with Visual Studio you can attach Visual Studio to a running process after launching UxAS, or you can add the command line arguments to the uxas project through its properties page.

To attach a process to Visual Studio (UxAS must be running):

  • Click the "Attach to Process..." option in the debug drop-down menu
  • Select UxAS from the list of available processes

To add command line arguments in Visual Studio:

  • Right-click the uxas project from the UxAS solution in the solution explorer
  • Select Properties from the popup menu
  • In the Configuration Properties tree, navigate to the Debugging node
  • Add the -cfgPath parameter to the Command Arguments and have it point to your UxAS configuration file

Let me know if this helps. I can add pictures if that's more useful.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/afrl-rq/OpenUxAS/issues/61#issuecomment-392794305, or mute the thread https://github.com/notifications/unsubscribe-auth/AkQdppAtLdea_YQ2Tq2MTyBawZYMrsR0ks5t3VkvgaJpZM4UOc_f .

kfu0115 commented 6 years ago

I think the problem is it went into the infinite loop and never got a chance to get out unless it receive a termination message. I have no idea how to solve it and why it is the case though.

So, after executing CheckAllRoutePlans() from RouteAggregratorService, it went to LmcpObjectNetworkClientBase::executeNetworkClient() nad trapped in the while loop as shown in the picture forever. The m_networkClientTypeName in the line UXAS_LOG_DEBUG_VERBOSE_MESSAGING(m_networkClientTypeName, "::executeNetworkClient completed calling m_lmcpObjectMessageReceiverPipe.getNextMessageObject()"); is CmasiPointSerachTaskService.

And the if statement is always false since receivedLmcpMessage is empty. Does it mean that CmasiPointSerchTaskService is not sending anything ? The same config file works before I add the 16th point to the file.

Can you give me some suggestion for that?

I also attached my configuration file image

image

SteveRas commented 6 years ago

Can you post the config and message files that you are using. I've seen problems like this before, usually something that I did. I'd like to track this down

kfu0115BAE commented 6 years ago

FilesUxAs.zip Here are the files you need to run the scenario.

kfu0115 commented 6 years ago

Posted them in a zip file. Please let me know if there is any problem running it!

SteveRas commented 6 years ago

Based on your inputs, I was able to run UxAS and recreate your issue. It turns out that there is a ‘feature’ in the PointSearchTask that causes the number of routes UxAS needs to calculate to increase exponentially with the number of PointSearchTasks. In general, when planning routes the heading of the vehicle and the task are used as constraints. If the heading to the PointSearchTask is unconstrained, 16 discrete headings are used. This allows the vehicle to plan approaches to the PointSearchTask from any of these headings. To implement this, the PointSearchTask generates 16 options. For planning purposes, each option is essentially a PointSearchTask.

An input to the assignment algorithm is a cost matrix that contains the cost for each vehicle to travel to the start of each task option and the cost to travel from the end of each task option to the start of every other task option. The cost is calculated by planning routes.

TotalRoutes = NumberTaskOptions * (NumberVehicles + (NumberTaskOptions - 1))

For your case: NumberVehicles = 2 NumberTasks = 16 NumberOptionsPerTask = 16 NumberTaskOptions = 16 * 16 = 256

TotalRoutes = 256 * (2 + (256 - 1)) = 65,792

This means that UxAS needs to calculate 65,792 routes in order to calculate an assignment. Either something is going wrong in the ‘RouteAgregator’ or it is taking a very long time to calculate those routes.

You can still use the same PointSearchTasks for your experiment if reduce the number of route required by adding a (or a few) wedges to the ‘ViewAngleList’ of each of the PointsearchTasks, e.g.

<ViewAngleList>
    <Wedge Series="CMASI">
        <AzimuthCenterline>90.0</AzimuthCenterline>
    </Wedge>
</ViewAngleList>

My Troubleshooting Order: -first use the console to see if the is any helpful information -Check the saved messages. ’datawork/Savedmessages/ ‘ These are the best troubleshooting aid to track down where the problem is occurring. -Turn off/on services in the config file. Sometime this helps to isolate the problematic service. -Attach a debugger

derekkingston commented 6 years ago

You may also be running into a size limitation in LMCP. No array/vector can be longer than (2^16-1). If you have 16 point search tasks each with 16 options then you would overflow the message size field. We are working to reduce the default number of options for point search task to make this a bit more tractable for large numbers of tasks.