ArtsemiY
commented
1 year ago Dear @dacolombo , thanks for the feedback!
Unfortunately, the SLURM submission functionality has not been reviewed for a while and we need some time to review that. I will inspect your issue with the TomoBEAR binary SLURM submission, but meanwhile I can suggest you to try a workaround - prepare and submit to the SLURM job script with environment setup and TomoBEAR binary execution in the same way you run this binary interactively. Not elegant, but should work
Hope it helps!
Describe the bug
Hi all, after using local execution of TomoBEAR for the Ribosome (EMPIAR-10064) tutorial, I am trying to submit the last DynamoAlignment job to a Slurm cluster since it needs more resources. When running the following command:
The job is correctly submitted to the queue, but it fails with this error as soon as it starts:
Indeed, the directory TomoBEAR/for_redistribution_files_only is not present inside of the TomoBEAR installation directory. I have seen in the code that this directory is accessed during the compilation and initialisation: https://github.com/KudryashevLab/TomoBEAR/blob/8a940e3cd0d5990aa37ea3de763a37573d68ae23/environment/initializeEnvironment.m#L166 and https://github.com/KudryashevLab/TomoBEAR/blob/8a940e3cd0d5990aa37ea3de763a37573d68ae23/utilities/compileTomoBEAR.m#L116
But I can't find how these directory are created and what should be the content of the run_TomoBEAR.sh script inside of it. I have a run_TomoBEAR.sh script in the TomoBEAR installation directory, which also makes use of the script inside of
for_redistribution_files_only.Am I missing something? Is there some procedure to generate this run_TomoBEAR.sh script? What should be the content of this script?
From what I understand, TomoBEAR has been initialised correctly, with the only warning showing being the missing conda environment to be provided for some optional dependencies.
Project JSON file The project json file is the same present in the repository for the Ribosome (EMPIAR-10064) tutorial.
~/ribosome_empiar_10064_dynamo.json
```JSON { "general": { "project_name": "Ribosome", "project_description": "Ribosome EMPIAR 10064", "data_path": "~/tomobear_test/tests/", "processing_path": "~/tomobear_test/output", "expected_symmetrie": "C1", "apix": 2.62, "tilt_angles": [-60.0, -58.0, -56.0, -54.0, -52.0, -50.0, -48.0, -46.0, -44.0, -42.0, -40.0, -38.0, -36.0, -34.0, -32.0, -30.0, -28.0, -26.0, -24.0, -22.0, -20.0, -18.0, -16.0, -14.0, -12.0, -10.0, -8.0, -6.0, -4.0, -2.0, 0.0, 2.0, 4.0, 6.0, 8.0, 10.0, 12.0, 14.0, 16.0, 18.0, 20.0, 22.0, 24.0, 26.0, 28.0, 30.0, 32.0, 34.0, 36.0, 38.0, 40.0, 42.0, 44.0, 46.0, 48.0, 50.0, 52.0, 54.0, 56.0, 58.0, 60.0], "rotation_tilt_axis": -5, "gold_bead_size_in_nm": 9, "template_matching_binning": 8, "binnings": [2, 4, 8], "reconstruction_thickness": 1400, "as_boxes": false }, "MetaData": { }, "CreateStacks": { }, "DynamoTiltSeriesAlignment": { }, "DynamoCleanStacks": { }, "BatchRunTomo": { "skip_steps": [4], "ending_step": 6 }, "StopPipeline": { }, "BatchRunTomo": { "starting_step": 8, "ending_step": 8 }, "GCTFCtfphaseflipCTFCorrection": { }, "BatchRunTomo": { "starting_step": 10, "ending_step": 13 }, "BinStacks": { }, "Reconstruct": { }, "DynamoImportTomograms": { }, "EMDTemplateGeneration": { "template_emd_number": "3420", "flip_handedness": true }, "DynamoTemplateMatching": { }, "TemplateMatchingPostProcessing": { "cc_std": 2.5 }, "DynamoAlignmentProject": { "iterations": 3, "classes": 4, "use_symmetrie": false, "use_noise_classes": true }, "DynamoAlignmentProject": { "iterations": 3, "classes": 2, "use_noise_classes": true, "use_symmetrie": false, "selected_classes": [1] }, "DynamoAlignmentProject": { "iterations": 3, "classes": 4, "use_noise_classes": true, "use_symmetrie": false, "selected_classes": [1] }, "DynamoAlignmentProject": { "iterations": 3, "classes": 4, "use_symmetrie": false, "use_noise_classes": true, "selected_classes": [1] }, "DynamoAlignmentProject": { "iterations": 3, "classes": 4, "use_noise_classes": true, "use_symmetrie": false, "selected_classes": [1] }, "DynamoAlignmentProject": { "iterations": 3, "classes": 4, "use_noise_classes": true, "use_symmetrie": false, "selected_classes": [1] }, "DynamoAlignmentProject": { "iterations": 3, "classes": 3, "use_noise_classes": true, "use_symmetrie": false, "selected_classes": [1], "box_size": 1.10, "binning": 4 }, "StopPipeline": { }, "DynamoAlignmentProject": { "classes": 1, "iterations": 1, "use_noise_classes": false, "swap_particles": false, "use_symmetrie": false, "selected_classes": [1], "binning": 4, "threshold":0.8 }, "DynamoAlignmentProject": { "classes": 1, "iterations": 1, "use_noise_classes": false, "swap_particles": false, "use_symmetrie": false, "selected_classes": [1,2], "binning": 2, "dt_crop_in_memory": 0, "threshold":0.9 }, "BinStacks":{ "binnings": [1], "use_ctf_corrected_aligned_stack": false, "run_ctf_phaseflip": true }, "Reconstruct": { "reconstruct": "unbinned" }, "DynamoAlignmentProject": { "classes": 1, "iterations": 1, "use_noise_classes": false, "swap_particles": false, "use_symmetrie": false, "selected_classes": [1,2], "binning": 1, "threshold":1 } ```Error message
To Reproduce Steps to reproduce the behaviour:
TomoBEARor therun_TomoBEAR.shscript).Expected behavior The job is started locally in the node allocated through Slurm.