permission denied error

[Leovar101]

Hello, sorry in advance, since I do not know how to format requests here. I'm trying to run grenepipe as per guide on our cluster, and so far it always shuts down like this:

Job failed, going on with independent jobs. [Tue Jul 16 19:10:18 2024]

group job qc (jobs in lexicogr. order):

[Tue Jul 16 19:10:18 2024]
rule fastqc:
    input: /gpfs/bwfor/home/fr/fr_fr/fr_yf1009/GenoDat/P2021159DFGB5_1/DE40NGSUKBD139802_2.fq.gz
    output: qc/fastqc/D-1-R2_fastqc.html, qc/fastqc/D-1-R2_fastqc.zip
    log: logs/fastqc/D-1-R2.log
    jobid: 18
    benchmark: benchmarks/fastqc/D-1-R2.bench.log
    wildcards: sample=D, unit=1, id=R2

/bin/sh: /gpfs/bwfor/home/fr/fr_fr/fr_yf1009/software/grenepipe/profiles/slurm/slurm-submit.py: Permission denied Error submitting jobscript (exit code 126):

This then repeats a couple of times, always ending with the Permission denied warning. What Permission is denied though, and how can I address this? Do you have any idea what even is going on here? Thanks in advance, and please tell me what Information I can add to facilitate the progress. Best regards Yannick

[lczech]

Hi Yannick @Leovar101,

can you please post the full output log as an attached file here?

It looks right now like you are mixing parts of an older grenepipe version with the latest release v0.13.0 from yesterday (alternatively, I messed up some of the setup and did not fully migrate all files to the new slurm approach). In particular, in the new grenepipe, I needed to break backwards compatibility, due to updates in Snakemake. The error you are seeing is likely because the file grenepipe/profiles/slurm/slurm-submit.py does not exist any more - so, "permission denied" might just be telling you it is not there at all. Snakemake has changed the way it works with slurm, see my first attempt of describing this here. That documentation will likely be updated in the near future, once I have fully figured out how snakemake wants to interact with slurm these days... but it should be good enough to get you started.

So long Lucas

[Leovar101]

Hi Lucas, thx for the quick response. I assume you mean the snakemake log file? I've added the one I think is correct, please let me know if I should for a different log file. 2024-07-16T183845.236177.snakemake.log

Alright, I suppose since you were editing it yesterday, that I simply did not see the expanded note and documentation. I will see what I can do and read through the snakemake cluster execution page as well. Thank you so much for your help and work.

Yannick

[lczech]

Hi Yannick @Leovar101,

ah okay I see, you are using grenepipe v0.12.2. I somehow assumed that since you opened the issue one day after I released a new version v0.13.0, that your issue must be related to something that changed in that release. Interesting though that you still got v0.12.2 to work - there were many things that broke in the meantime due to conda and snakemake not being fully backwards compatible any more. If that version works for you, you can keep using it. But if you are just starting to use grenepipe anyway, it mgiht make sense to start using the new version, which updates a lot of the tools and snakemake. You'll have to re-create a new grenepipe conda environemnt, but other than that, it should work almost the same.

But for now, let's see if we can fix your issue. So, since you are using v0.12.2 there, forget my tip about the new cluster documentation, as that does not fully apply to this version, and isn't the issue here. I think, all you need to do is to make the slurm submission scripts executable.

From the main grenepipe directory, call

chmod +x profiles/*/*.py

and then try again. I think that should work.

On that note, to figure out how that issue happened in the first place: How did you get grenepipe onto the cluster that you are using? Via git, or download on your computer and then copy it over? If the latter, what type of computer are you using (Windows, Mac, Linux)? And did you unpack the downloaded zip file first and then move it? Because it seems that the file permissions that tell the cluster unix system that these slurm files can be executed got lost somewhere in that process, and hence you are seeing the permission denied error, because you need to give permission to those files to be executed first.

Cheers Lucas

[Leovar101]

Hi Lucas,

I just updated everything to the new version and created the environment. I'll see if I can get it to work now, and update accordingly. I did use a windows pc to download and unpack the file, and then copied it over to my cluster profile, yes.

Cheers Yannick

[lczech]

Hi Yannick,

ah nice, let me know how it goes with the new version ;-)

Also, yes, the unpacking on Windows explains what happens. Windows does not use file permissions the way Unix does, and so those got stripped there, and then were not available once transferred.

Closing the issue then, but feel free to comment or open a new one if you have any trouble with the new version!

Cheers Lucas