omerwe / polyfun

PolyFun (POLYgenic FUNctionally-informed fine-mapping)
MIT License
89 stars 22 forks source link

/tmp filled up quickly by genome wide fine mapping #40

Closed yyshi1 closed 3 years ago

yyshi1 commented 3 years ago

Hello,

Thank for create_finemapper_jobs.py, the genome wide finemapping become much easier. At the same time, I got email from system admin that I filled /tmp folder quickly with ld.npz files so nothing else will dispatch. I can clean up the /tmp folder after each job finished. I thought it might be a good idea to have codes to do clean up inside the finemapper.py. I used susie in the finemapper.py.

Best regards,

Yunling

omerwe commented 3 years ago

Hi Yunling,

Thanks for the bug report (I thought that the OS can automatically handle deleting files from /tmp, but I guess not...). I modified the code to explicitly delete temporary LD files after fine-mapping. Can you please git pull the latest code and let me know if there's a problem?

Best,

Omer

yyshi1 commented 3 years ago

Hi Omer,

Thank you for quickly updating the script. You are right that OS can clean up /tmp folder automatically. The problem is that whole genome fine mapping can quickly fill the folder before OS ‘s schedule clean up. For example, there are more than 250 jobs for chromosome 2.

I will download the script for testing and will let you know.

Yunling

Sent from my iPhone

On Mar 10, 2021, at 3:56 AM, Omer Weissbrod @.***> wrote:

 Hi Yunling,

Thanks for the bug report (I thought that the OS can automatically handle deleting files from /tmp, but I guess not...). I modified the code to explicitly delete temporary LD files after fine-mapping. Can you please git pull the latest code and let me know if there's a problem?

Best,

Omer

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.

yyshi1 commented 3 years ago

Hi Omer,

I tested the fixed script on chromosome 2 with 244 jobs submitted. The script works perfectly even although I asked system admin monitor /tmp directory and had no complain.

Thank you for fixing the script!

Yunling

Sent from my iPhone

On Mar 10, 2021, at 12:53 PM, Yunling Shi @.***> wrote:

Hi Omer,

Thank you for quickly updating the script. You are right that OS can clean up /tmp folder automatically. The problem is that whole genome fine mapping can quickly fill the folder before OS ‘s schedule clean up. For example, there are more than 250 jobs for chromosome 2.

I will download the script for testing and will let you know.

Yunling

Sent from my iPhone

On Mar 10, 2021, at 3:56 AM, Omer Weissbrod @.***> wrote:

 Hi Yunling,

Thanks for the bug report (I thought that the OS can automatically handle deleting files from /tmp, but I guess not...). I modified the code to explicitly delete temporary LD files after fine-mapping. Can you please git pull the latest code and let me know if there's a problem?

Best,

Omer

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or unsubscribe.