When testing new SPT graph plugins, I've found myself using the cg-gnn workflow not to train a model and get importance scores, but to use the first half of the pipeline to create graph datasets that fit the requirements of models not yet included in SPT.
I'd like to modularize the cg-gnn workflow such that the first 3 processes for graph dataset creation can be called without invoking the cg-gnn model training portion, and potentially vice versa. Nextflow does support importing processes from other .nf files, so this ought to be not too big of a lift. I think the only significant changes we'd need to make are
Split cggnn.nf into graph-export.nf and train.nf. We could even break it out so that there are process-specific .nf files like graph-export.nf, train-cg-gnn.nf, and upload-importances.nf that are imported by .nf files that define a specific workflow like cg-gnn.nf, export-graphs.nf, and extract-train-upload-cg-gnn.nf.
Edit spt workflow configure such that it copies all the relevant .nf files when setting up a workflow.
This will facilitate #268 as well since it can recycle some of the processes used for the CG-GNN workflow.
When testing new SPT graph plugins, I've found myself using the cg-gnn workflow not to train a model and get importance scores, but to use the first half of the pipeline to create graph datasets that fit the requirements of models not yet included in SPT.
I'd like to modularize the cg-gnn workflow such that the first 3 processes for graph dataset creation can be called without invoking the cg-gnn model training portion, and potentially vice versa. Nextflow does support importing processes from other .nf files, so this ought to be not too big of a lift. I think the only significant changes we'd need to make are
cggnn.nfintograph-export.nfandtrain.nf. We could even break it out so that there are process-specific .nf files likegraph-export.nf,train-cg-gnn.nf, andupload-importances.nfthat are imported by .nf files that define a specific workflow likecg-gnn.nf,export-graphs.nf, andextract-train-upload-cg-gnn.nf.spt workflow configuresuch that it copies all the relevant .nf files when setting up a workflow.This will facilitate #268 as well since it can recycle some of the processes used for the CG-GNN workflow.