Using the the cluster settings as follows Athena distributes the sub-assemblies across the nodes of our grid engine very nicely.
"cluster_settings": { "cluster_type": "IPCluster", "processes": 318, "cluster_options": { "scheduler": "sge", "queue": "all.q", "extra_params": {"mem":16}
Where "mem":16 is 16GB of RAM.
However, Athena uses these same settings for the OLC/Flye step. This is problematic for two reasons. First, athena passes the processes value (here set to 318) to flye's thread argument and flye will reject values greater than 128. Second, the RAM demands of the single Flye step far exceed that of the individual sub-assemblies. Can there be distinct cluster settings for the sub-assemblies and the OLC steps? I see how to work around setting the thread value but not where the OLC submits the jobs to the cluster such as the setting could be changed. Happy to work with a little steer.
Using the the cluster settings as follows Athena distributes the sub-assemblies across the nodes of our grid engine very nicely.
"cluster_settings": { "cluster_type": "IPCluster", "processes": 318, "cluster_options": { "scheduler": "sge", "queue": "all.q", "extra_params": {"mem":16}Where "mem":16 is 16GB of RAM.However, Athena uses these same settings for the OLC/Flye step. This is problematic for two reasons. First, athena passes the processes value (here set to 318) to flye's thread argument and flye will reject values greater than 128. Second, the RAM demands of the single Flye step far exceed that of the individual sub-assemblies. Can there be distinct cluster settings for the sub-assemblies and the OLC steps? I see how to work around setting the thread value but not where the OLC submits the jobs to the cluster such as the setting could be changed. Happy to work with a little steer.