Running in an error while trying to import edges

[julianofischer]

Hi, I am using the following command in order to import hyccups contact trace (ONE format).

ditl graphs ie --orig-time-unit s --dest-time-unit s --format one hyccups.jar output_one_set_init.txt

I converted myself the hyccups trace to ONE format. The file is attached: output_one_set_init.txt

I suspect that the error is due to connections inconsistencies since I do not filtered the original trace. However, I need further investigation. Could someone help me? Anyway, I intend to post the solution here, in order to help to solve future issues.

[jwhitbeck]

I got the following stack trace when attempting to reproduce.

ditl graphs ie --orig-time-unit s --dest-time-unit s --format one hyccups.jar output_one_set_init.txt

java.lang.NullPointerException
    at ditl.graphs.AdjacencyMap.remove(AdjacencyMap.java:98)
    at ditl.graphs.AdjacencyMap$Edges.remove(AdjacencyMap.java:599)
    at ditl.graphs.AdjacencyMap.remove(AdjacencyMap.java:91)
    at ditl.graphs.AdjacencySet.remove(AdjacencySet.java:84)
    at ditl.graphs.EdgeTrace$Updater.handleEvent(EdgeTrace.java:59)
    at ditl.graphs.EdgeTrace$Updater.handleEvent(EdgeTrace.java:39)
    at ditl.StatefulWriter.write(StatefulWriter.java:47)
    at ditl.Writer.append(Writer.java:97)
    at ditl.graphs.ONEContacts.fromONE(ONEContacts.java:52)
    at ditl.graphs.cli.ImportEdges.run(ImportEdges.java:84)
    at ditl.cli.App.exec(App.java:113)
    at ditl.cli.CLI.startApp(CLI.java:129)
    at ditl.cli.CLI.parseArgs(CLI.java:80)
    at ditl.cli.CLI.main(CLI.java:158)
java.lang.NullPointerException

Looking at the code from AdjacencyMap.java:

final Map<Integer, T> from_map = map.get(id1);
final T obj = from_map.remove(id2);

By design, most of the ditl code will fail loudly if the input traces are inconsistent in some way. In this case, it looks you are removing an edge that doesn't exist. I see two solutions:

• add some adhoc logging to that section of code to figure out exactly what edge is causing this, and then fix it in the original trace; or
• swallow the error entirely to get a clean ditl trace (at the cost of some information loss from the original trace)

[julianofischer]

Hello again, John, how are you? I am trying to solve this problem. First of all, you are right. The error is due a try to remove an edge which doesn't exist. Following is a snippet of the connection trace.

322575 CONN 52 4 up
322576 CONN 52 4 down
322585 CONN 52 4 up
322586 CONN 52 4 down
322595 CONN 52 4 up
322596 CONN 4 52 up
322596 CONN 52 4 down
322597 CONN 4 52 down      #the error occurs here

As you can see above, the error occurs when trying to remove the connection 4 -> 52 at the instant 322597.

My question is: DITL is considering 4 -> 52 and 52 -> 4 as the same connection? I know all connections in ONE are bidirectional so that it considers 4->52 equals to 52->4. However, I was thinking DITL were able to deal with unidirectional connections. Am I mistaken?

[jwhitbeck]

Hi Juliano,

DITL has two different types of graph traces:

• Edges (bidirectional)
• Arcs (unidirectional)

If you need unidirectional connections, you should use "arcs".

[julianofischer]

Hello @jwhitbeck, thank you for your attention. I would like to import arcs from other traces. Regarding edges, it is possible to import from 2 formats: crawdad and one. However, regarding arcs, it is not possible to import from one format, maybe because there are no unidirectional connections in one.

I could not figure it out what is the structure of this format. I guess it is something like below

TIMESTAMP
FROM TO UP/DOWN
FROM TO UP/DOWN
FROM TO UP/DOWN
FROM TO UP/DOWN
TIMESTAMP
FROM TO UP/DOWN
FROM TO UP/DOWN
...

Could you kindly confirm if I am right?

[julianofischer]

while ((line = br.readLine()) != null) {
            final String[] elems = line.split("[ \t]+");
            final Integer id1 = idGen.getInternalId(elems[0]);
            final Integer id2 = idGen.getInternalId(elems[1]);
            final long begin = (long) (Double.parseDouble(elems[2]) * timeMul) + offset;
            final long end = (long) (Double.parseDouble(elems[3]) * timeMul) + offset;
            arcWriter.queue(begin, new ArcEvent(id1, id2, ArcEvent.Type.UP));
            arcWriter.queue(end, new ArcEvent(id1, id2, ArcEvent.Type.DOWN));
        }

Based on this code I think the right format is: FROM TO BEGIN END

Right?

[jwhitbeck]

That seems right :)