Write the .dag file to a memory based file, which should greatle improve read/write times. Only noticeable on large datasets I imagine.
This should be easy given rclone. I'll test next time I have a giant dataset to work with.
Can the .dag file be skipped entirely? Specically, can the flat file be skipped, and can the data be streamed from the database directly to the cytoscape client?
I feel like the answer is yes, but also, I'm still debugging a fair amount, and having the flat file to poke around in is helpful.
I have two ideas come to mind.
Write the
.dag
file to a memory based file, which should greatle improve read/write times. Only noticeable on large datasets I imagine.Can the
.dag
file be skipped entirely? Specically, can the flat file be skipped, and can the data be streamed from the database directly to the cytoscape client?