GraphOne sample code. Error while archiving.

[pradeepfn]

I want to run an iterative graph processing workload with GraphOne static_view API. I came up with a basic code piece to get things working.

In this sample code,

• I insert edge pairs. (write)
• get_out_degree of a vertex (read).

However, when the inserted edge count exceeds BATCH_SIZE ( 1 << 16) GraphOne archive thread segfaults. edge_sharding.h:459

You can find my sample code at - https://github.com/pradeepfn/GraphOne/blob/master/server.cpp

I have not done any modifications to GraphOne code. Modifications are only in CmakeList file and server.cpp file. You can run above by building and running (no args) graphone_server cmake target.

thanks.

[pradeep-k]

I was also trying to write a similar file to allow users to develop with GraphOne. I will take this file and will share the new version tomorrow morning after fixing the issues.

Meanwhile, batch_update() is broken, and right now only pgraph->batch_edge() works.

[pradeepfn]

Thanks!. Looking forward to the fixed code. Please feel free to use our code as an example on the project page. In the example code, we are not creating the persist thread for debugging reasons. ( https://github.com/pradeepfn/GraphOne/blob/d037a1fc53c73286532ab3027ee8441a289daddb/server.cpp#L94).

But we like to have both archive and persist threads in our working example.

[pradeep-k]

example.cpp was added in master branch. You can just rename example.cpp to main.cpp, and can use the current CMake infrastructure.

[pradeepfn]

Thanks. The example code works!. I have a follow-up question. As per my understanding durable GraphOne in the same code enabled by changing the code to;

g->set_odir("/dev/shm/"); g->create_threads(true, true);

The example seems to be working fine with the changes and data/metadata files get created in /dev/shm/

Can you please confirm. Are there any gotchas when using GraphOne in persist/durable mode. ?

[pradeepfn]

Apparently, I was wrong. Can't see any data files in /dev/shm/. The example works with the above changes, but I am not sure if the edge list gets persisted.

[pradeep-k]

It works for me when I do the above two changes

pkumar@mira:~/src/GraphOne/build$ ./graphone32
Threads Count = 19
enterting w_func
----------------
Archived_edges = 0
Non-archived edges : 0
Degree Out of 1 = 0
----------------
Archived_edges = 0
Non-archived edges : 4
Degree Out of 1 = 0
----------------
Archived_edges = 4
Non-archived edges : 65533
Degree Out of 1 = 1
----------------
Archived_edges = 65537
Non-archived edges : 65536
Degree Out of 1 = 84
----------------
Archived_edges = 131073
Non-archived edges : 65536
Degree Out of 1 = 147
----------------
Archived_edges = 262144
Non-archived edges : 0
Degree Out of 1 = 271
----------------
pkumar@mira:~/src/GraphOne/build$ ls -lh /dev/shm/
total 2.0M
-rwx------ 1 pkumar pkumar 2.0M Jan  6 07:25 friend.elog
-rw-rw-r-- 1 pkumar pkumar    0 Jan  6 07:25 friend.snap
-rwx------ 1 pkumar pkumar    0 Jan  6 07:25 friend.str
-rw-rw-r-- 1 pkumar pkumar    0 Jan  6 07:25 gtype.vtable

[pradeep-k]

Please make sure that you give the last slash in /dev/shm/ or try with any writable directory. The output directory must exist. GraphOne will not create directory.

Also, there is function in test/plain_to_edge.h named as waitfor_archived_durable() that may be called at the end in example.cpp where waitfor_archive() is called. But irrespective, files are created during file_open() call, and are written latter in the durable thread.

Do let me know, if it works.

[pradeepfn]

Thanks for the valuable help!. Sample code works.!