Identify entities to log during deployment

[marc-hanheide]

We need to decide what we want to log (and how) during the deployment. I suggest a specific component to be developed that logs into mongodb everything AAF deems relevant. Hence we need to know what we want to log, and what research questions we want to answer. Let's use this issue to collect what we want to log, by editing or commenting on this entry. @denisehe is the person to tell us.

The idea is to generate logs that can easily be translated into ELAN annotations.

Things to log (all including time stamp):

• [ ] All scheduled tasks (including start and end time, and meta data about the task), and all executed tasks (including start and end time, and success), including "source" of the task (routine, staff interface, robot interface)
• [ ] current waypoint
• [ ] currently displayed page (logging the /strands_webserver/display_*/page topics)
• [ ] detected humans (with ID, start and end time of their appearance)
• [ ] speech output (start and end time)
• [ ] navigation recoveries
• [ ] errors in roslog
• [ ] system restarts
• [ ] Circle Card readings

Besides, we shall record the screen (screen saver needs to be disabled) at very low framerate (once every 3 seconds) using avconv -f x11grab and the "interaction camera" (webcam to be provided by @mzillich) at 0.5Hz (see https://github.com/strands-project/aaf_deployment/issues/16).

[marc-hanheide]

I tag @mudrole1 @hawesie @yianni @Jailander @cburbridge @ToMadoRe @denisehe @cdondrup @nilsbore @mzillich as they are the AAF core team and should follow this discussion.

[marc-hanheide]

assigned to @denisehe to share her view on what to log first

[cdondrup]

Using the standard people perception with the dedicated logging component stores x y coordinates of the person, a uuid, the current robot pose, and the timestamps for each message. If this is enough we can just run that. Storing the robot pose might be a bit of an overhead in case it is logged constantly anyway. The benefit is that this puts it all in one message and makes it very easy to parse.

[marc-hanheide]

That's a good start. It does go into mongodb, right? For later, we'd need annotations with a start and end date of a person by ID. That can of course be created from your data, so this should be sufficient as far as persons are concerned. BTW, we agree that everything is time-logged using rostime, yes?

[cdondrup]

Everything goes into mongodb, yes. I guess rostime is the cheapest way of doing this since most messages have header any way. Isn't ros using the system time? What would be the alternative?

[marc-hanheide]

rostime is system time AFAIK, but I have seen logs, where people log the time since start or likewise. We must make sure everything is synchronised. We might even consider "watermarking" the videos we record from head camera etc to include the rostime, so we can align them. At least using -timestamp for ffmpeg set to the correct time whenever we start recording something (and using it in the filename as well).

[kunzel]

Regarding the detected humans, we plan to transform a sequence of poses for one person into a trajectory data structure (https://github.com/strands-project/strands_perception_people/blob/indigo-devel/human_trajectory/msg/Trajectory.msg). It works already offline. It think, @ferdianjovan will also look into an online version. @cdondrup @marc-hanheide Please let us know if you would like to add anything to the Trajectory.msg.

[marc-hanheide]

Yes, we want that :+1:

[cdondrup]

Looks exactly like what we want.

[denisehe]

Hey,

I think we can discuss some details during our todays hangouts meeting!

[marc-hanheide]

I think we'd need a written schedule. But that could come out of today's hangout. An idea for @hawesie why not use Google calendar to"program" it?

[denisehe]

Loging data that would be of relevance for AFF:

Bellbot:

• Frequency of use (how often is the service used during Y2?) [log data]
• user feedback to short questionnaire implemented on robot [log data/questionnaire]
• Number of errors occurring during tour [log data]

Info Terminal:

• Frequency of use (over time) (log data)
• what information is used most (log data)

Walking group:

• error-frequency
• how often/long is the robot used during the waiting phase in the beginning (freuquencies, durations of use)

Loging data that would be of relevance G4S

• frequency/duraitons of ID-checks
• error-frequency

AND: VERY IMPORTANT: how can these date be extracted from the overall loging file? is it possible that you implement the thing that it really gives a nice outputfile containing frequ./duraitons streight away?

[denisehe]

ah and in case of infoterminal: where was the infoterminal used (what area of the buliding

and in case of bellbot where from did the service start and where did it go

[denisehe]

required vido logs from the small newly mounted cam:

1) AAF: a) video record the walking group b) video record when infoterminal is used c) video record when bellbot is used

2) G4S: video record the ID scan

[hawesie]

I would like to agree a common core set of logs for both AAF and G4S so we make sure we can compare deployments in a number of ways.

For the record, the task executor logs lots of information about tasks which can be used to reconstruct robot behaviour. Check message store collection task_events to see this. These events are also published when the system is running.

[hawesie]

@marc-hanheide can we assign someone to create the correct config for mongodb_log for most of the things in your list above?

[marc-hanheide]

Yes, but whom?

[hawesie]

Me maybe? Or someone at your place? Or @nilsbore as he's got experience with the logging.

[marc-hanheide]

@nilsbore might be a good person for this, indeed. He volunteered to be at the AAF pre-deployment.

[nilsbore]

I guess this would mostly be poking people to log stuff in their systems / identify which topics should be logged?

[marc-hanheide]

I was actually thinking to have a kind of aggregator node, a bit like the marathon logger.

[hawesie]

I think it should be as simple as a launch file which launches mongodb_log with the correct topics.

[marc-hanheide]

Also possible, then we'll have to log a lot and do post-mortem analysis to extract meaningful stuff. If we are not logging too much that should be feasible. FYI: I started playing with extracting and visualising logs from message_store into ELAN. This could be developed into a post-mortem analysis tool to extract meaningful annotations for @denisehe and others at AAF. Right now it's more a sandpit to play with: https://github.com/strands-project/elan_analyser But here is a sample output. Imagine the different tiers to be related to person presence (and annotating their distance or the number of people etc.), the video being the one recorded from the front camera and the annotations being temporally aligned with the video (obviously). Then we'd have different tier annotating the current waypoint, the current task, etc.

[hawesie]

Adding @PDuckworth to the conversation as he would like to see robot poses logged throughout the runs.

[denisehe]

hey, I trust your judgement in this reagard @marc-hanheide - I have no pre-experience with ELAN but if we can easily trasphere information to SPSS it should be fine :-)

[cdondrup]

I think the logging of people should be done by #85 This will log all the poses of detected people into the datacentre. Only problem, it doesn't give you a start and end time without looking at all the detections with same uuid and finding the min and max time.

[marc-hanheide]

@cdondrup @hawesie @ferdianjovan How is the logging done at G4S? We should be doing the very same thing here.

[hawesie]

There is not a standard approach. I thought (as above) we assigned @nilsbore to get some logging going. My assumption is that it will be one or more instances of mongodb_log plus a list of topics. And there will be a shared list of topics, plus potentially different additions for each site.

[marc-hanheide]

Yes, sure. I meant more specifically about the people tracks.

[nilsbore]

@hawesie I am sorry, between different paper deadlines this has slipped my mind. I can assemble a list at the AAF pre-deployment next week but unfortunately I don't have any time before that.

[nilsbore]

Ok, I had a first look at topics to log during AAF deployment: https://github.com/strands-project/aaf_deployment/wiki/Topics-to-log-during-deployments. Please could @bfalacerda have a look at the monitored nav topics to see if they make sense. @Jailander could you have a look at topological nav topics? And @mudrole1 , I could not find so many topics to log from the scheduler, could you also have a look?

Otherwise, I will document each group of topics in the document to describe what they are actually publishing. Will try to get that done today.

[hawesie]

@nilsbore /task_executor/events is already logged in the node itself, as are scheduling problems but the executor.

[nilsbore]

@hawesie So we shouldn't log anything from the executor here?

[marc-hanheide]

Are these logged via mongodb_log? I want it all in one Format/place to ease later analysis.

[hawesie]

@marc-hanheide everything I mentioned above is done by the message store which is exactly what mongodb_log does anyway.

@nilsbore I think it's all covered unless @marc-hanheide or @mudrole1 think of other things.

[nilsbore]

So I'm running into a bit of a problem as mentioned on gitter. If @hawesie could have a look at this I would be grateful. You can try by using my logged_topics_list branch: https://github.com/nilsbore/aaf_deployment/tree/logged_topics_list . Run using roslaunch aaf_logging topics.launch. This doesn't work for me, but if I change to only log 5 topics (up until /goal), it works. It doesn't seem to be a limit on the number of topics that can be logged as other variations with more topics also work. It's hard to say what is causing this. I see the exact same behaviour when running and providing the topics to log via the command line: rosrun mongodb_log mongodb_log.py /rosout_filtered /tf /robot_pose /cmd_vel /goal vs rosrun mongodb_log mongodb_log.py /rosout_filtered /tf /robot_pose /cmd_vel /goal /mileage .

[nilsbore]

So me and @marc-hanheide found the source of this. If you look at https://github.com/strands-project/mongodb_store/blob/hydro-devel/mongodb_log/scripts/mongodb_log.py#L435 , it seems like mongodb_log is blocking other topics from subscribing until the previous ones have been announced and mongodb_log can get their type.

[nilsbore]

https://github.com/strands-project/mongodb_store/pull/125 addresses the mentioned issue.

[nilsbore]

@hawesie The next problem is that the same node name is assigned to all the worker nodes launched in mongodb_log.py when launched via roslaunch, as detailed in https://github.com/strands-project/mongodb_store/issues/116 . Is there a plan for fixing this or should we launch this from a bash script?

[hawesie]

I think that's easy to fix. At the moment all these issues are a bit spread around, so please make me an ordered list of things you'd like me to fix + deadlines.

[nilsbore]

Ok, cool. The things that are left is this + the issue you opened. I'll drop some comments there.