Closed Aikaterini-Chatzipli closed 4 years ago
Please update the package from GitHub
Was there a bug and have you fixed it now?
Thanks
There was initially a bug. But it should work now.
I updated the software but I am still getting the same error as before.
Hi Mark, I believe the issue the user is having came from the decomposition plot. Given that I don't know any other user having the issue, do you have any idea on the cause of the issue for this particular user?
Thanks, Mishu
From: Aikaterini-Chatzipli notifications@github.com Sent: Tuesday, June 23, 2020 3:18 PM To: AlexandrovLab/SigProfilerExtractor SigProfilerExtractor@noreply.github.com Cc: Islam, S m0islam@health.ucsd.edu; Comment comment@noreply.github.com Subject: Re: [AlexandrovLab/SigProfilerExtractor] Failing to run SigProfilerExtractor (#29)
I updated the software but I am still getting the same error as before.
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/AlexandrovLab/SigProfilerExtractor/issues/29#issuecomment-648457755, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFOQ2HLGZM7U4F3TQJGH3T3RYES3DANCNFSM4OC3OI4Q.
Hello Katerina,
It looks like ReportLab is unable to find the 'Arial Bold.ttf' font file. I would like to verify that the 'Arial Bold.ttf' font file exists and is accessible on your system.
Could you please check that the version you have downloaded has a 'Fonts' directory and that it contains 'Arial Bold.ttf'? I believe you can check it exists here:
ls /hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor/src/Fonts
If it does have a 'Fonts' directory, could you please check that the 'Arial Bold.ttf' file inside of it works? You can test it by running this code in a python instance (note: reportlab.rl_config.TTFSearchPath is where ReportLab is going to search for the font file):
import reportlab
from reportlab.pdfbase.ttfonts import TTFont
from reportlab.pdfbase import pdfmetrics
reportlab.rl_config.TTFSearchPath = ["/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor/src/Fonts"]
pdfmetrics.registerFont(TTFont('Arial-Bold', 'Arial Bold.ttf'))
Please let me know if you come across any issues.
Thanks, Mark
Hi Mark,
Thanks for you message. It seems I don't have the Fonts directory. How can I get it?
Thanks,
Katerina
The Fonts directory is included in the current version of the GitHub repository. Could you please update your environment's version of SigProfilerExtractor with the current version of SigProfilerExtractor on GitHub?
I did update the package from GitHub using the following line: "git clone https://github.com/AlexandrovLab/SigProfilerExtractor.git" Then, I activated the environment and run SigProfilerExtractor but I got the same error about the Fonts. Do you think the environment is not somehow updated? If so, what else should I do?
Thank you, Katerina
I believe that you are running SigProfilerExtractor from the virtual environment named "SigProfilerExtractor": /hps/anaconda3/envs/SigProfilerExtractor/
The virtual environment has the GitHub package SigProfilerExtractor installed at: /hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor
In order to update the version of SigProfilerExtractor that your virtual environment is using, you will need to update the SigProfilerExtractor library at: /hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor
This is a way to update the SigProfilerExtractor library in your virtual environment
After running: git clone https://github.com/AlexandrovLab/SigProfilerExtractor.git
You will want to copy the file over to your virtual environment's site-packages: cp -r SigProfilerExtract/SigProfilerExtractor /hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/
Please let me know if this works for you.
Thanks, Mark
Thank you very much, Mark. The Fonts issue is resolved but I am now getting a different error. Please read below. The run_subs_signatures.py I run contains your code and I attach it below too. Thanks.
from SigProfilerExtractor import sigpro as sig
subsVcfFolder='/hps/filt_vcfs/subs/'
toti = 50
output_folder='/hps/subsResults/'
sig.sigProfilerExtractor("vcf", output_folder, subsVcfFolder, reference_genome="GRCh38", minimum_signatures=1, maximum_signatures=8, nmf_replicates=toti)
Traceback (most recent call last):
File "run_subs_signatures.py", line 1, in
I am curious why your environment is not automatically installing the requirements when you install the SigProfilerExtractor tool. Anyway, just write "$pip install PyPDF2" in your terminal. That should solve the problem.
Yes, this is solved, thank you. I have got two more though, please see below. Is it supposed to find log files which do not exist?
os.remove(log_file)
FileNotFoundError: [Errno 2] No such file or directory: '/hps/filt_vcfs/subs/logs/SigProfilerMatrixGenerator_subs_GRCh382020-06-25.out'
os.remove(error_file)
FileNotFoundError: [Errno 2] No such file or directory: '/hps/filt_vcfs/subs/logs/SigProfilerMatrixGenerator_subs_GRCh382020-06-25.err'
Hi Erik, Could you please check if the issue the user is talking about is coming from MatrixGenerator.
Thanks, Mishu
From: Aikaterini-Chatzipli notifications@github.com Sent: Thursday, June 25, 2020 11:41 AM To: AlexandrovLab/SigProfilerExtractor SigProfilerExtractor@noreply.github.com Cc: Islam, S m0islam@health.ucsd.edu; Comment comment@noreply.github.com Subject: Re: [AlexandrovLab/SigProfilerExtractor] Failing to run SigProfilerExtractor (#29)
Yes, this is solved, thank you. I have got two more though, please see below. Is it supposed to find log files which do not exist?
os.remove(log_file)
FileNotFoundError: [Errno 2] No such file or directory: '/hps/filt_vcfs/subs/logs/SigProfilerMatrixGenerator_subs_GRCh382020-06-25.out'
os.remove(error_file)
FileNotFoundError: [Errno 2] No such file or directory: '/hps/filt_vcfs/subs/logs/SigProfilerMatrixGenerator_subs_GRCh382020-06-25.err'
— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/AlexandrovLab/SigProfilerExtractor/issues/29#issuecomment-649751764, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFOQ2HJJPLWSOCMW3W6OUZDRYOK57ANCNFSM4OC3OI4Q.
Hi,
Can I please ask that you update SigProfilerMatrixGenerator. This appears to be the result of a bug that was fixed some time ago. If you are still experiencing the problem please let me know and please include the version of SigProfilerMatrixGenerator that is installed on your system (you can type the command: pip show SigProfilerMatrixGenerator)!
Best, Erik
Thanks Erik! I updated SigProfilerMatrixGenerator and the version I am using is 1.1.16. It is very strange that the code still complains about the log files which do exist. Please read messages below. What would you advise me to do next?
Thank you,
Katerina
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor/sigpro.py", line 543, in sigProfilerExtractor data = datadump.SigProfilerMatrixGeneratorFunc(project_name, refgen, project, exome=exome, bed_file=None, chrom_based=False, plot=False, gs=False) File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerMatrixGenerator/scripts/SigProfilerMatrixGeneratorFunc.py", line 303, in SigProfilerMatrixGeneratorFunc os.remove(log_file) FileNotFoundError: [Errno 2] No such file or directory: '/hps/filt_vcfs/subs/logs/SigProfilerMatrixGenerator_subs_GRCh382020-06-28.out'
Can you please provide all of your commands and/or script that you are using to run the tools along with the directory from which you are running the code (type pwd)? We have not experienced this with any other users, so I imagine there is a simple solution relating to project directory structure or from where you are running the code.
Best, Erik
Here is how I updated SigProfilerMatrixGenerator
$ pip install SigProfilerMatrixGenerator --upgrade
Requirement already up-to-date: SigProfilerMatrixGenerator in /hps/anaconda3/lib/python3.7/site-packages (1.1.16) Requirement already satisfied, skipping upgrade: matplotlib>=2.2.2 in /hps/anaconda3/lib/python3.7/site-packages (from SigProfilerMatrixGenerator) (3.1.3) Requirement already satisfied, skipping upgrade: pandas>=0.23.4 in /hps/anaconda3/lib/python3.7/site-packages (from SigProfilerMatrixGenerator) (1.0.1) Requirement already satisfied, skipping upgrade: numpy>=1.14.3 in /hps/anaconda3/lib/python3.7/site-packages (from SigProfilerMatrixGenerator) (1.18.1) Requirement already satisfied, skipping upgrade: sigProfilerPlotting>=1.0.1 in /hps/anaconda3/lib/python3.7/site-packages (from SigProfilerMatrixGenerator) (1.1.6) Requirement already satisfied, skipping upgrade: statsmodels>=0.9.0 in /hps/anaconda3/lib/python3.7/site-packages (from SigProfilerMatrixGenerator) (0.11.0) Requirement already satisfied, skipping upgrade: scipy>=1.1.0 in /hps/anaconda3/lib/python3.7/site-packages (from SigProfilerMatrixGenerator) (1.4.1) Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /hps/anaconda3/lib/python3.7/site-packages (from matplotlib>=2.2.2->SigProfilerMatrixGenerator) (2.4.6) Requirement already satisfied, skipping upgrade: cycler>=0.10 in /hps/anaconda3/lib/python3.7/site-packages (from matplotlib>=2.2.2->SigProfilerMatrixGenerator) (0.10.0) Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /hps/anaconda3/lib/python3.7/site-packages (from matplotlib>=2.2.2->SigProfilerMatrixGenerator) (1.1.0) Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /hps/anaconda3/lib/python3.7/site-packages (from matplotlib>=2.2.2->SigProfilerMatrixGenerator) (2.8.1) Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /hps/anaconda3/lib/python3.7/site-packages (from pandas>=0.23.4->SigProfilerMatrixGenerator) (2019.3) Requirement already satisfied, skipping upgrade: patsy>=0.5 in /hps/anaconda3/lib/python3.7/site-packages (from statsmodels>=0.9.0->SigProfilerMatrixGenerator) (0.5.1) Requirement already satisfied, skipping upgrade: six in /hps/anaconda3/lib/python3.7/site-packages (from cycler>=0.10->matplotlib>=2.2.2->SigProfilerMatrixGenerator) (1.14.0) Requirement already satisfied, skipping upgrade: setuptools in /hps/anaconda3/lib/python3.7/site-packages (from kiwisolver>=1.0.1->matplotlib>=2.2.2->SigProfilerMatrixGenerator) (45.2.0.post20200210)
My working directory is /hps/code/ from where I call the script named run_subs_signatures.py which contains the lines below:
`from SigProfilerExtractor import sigpro as sig
subsVcfFolder='/hps/filt_vcfs/subs/'
toti = 50
output_folder='/hps/subsResults/'
sig.sigProfilerExtractor("vcf", output_folder, subsVcfFolder, reference_genome="GRCh38", minimum_signatures=1, maximum_signatures=8, nmf_replicates=toti)`
To run run_subs_signatures.py I do the following,
$ pwd /hps/code/ $ conda activate SigProfilerExtractor $ python run_subs_signatures.py
Hi Eric,
Can you see anything obviously wrong based on the information provided in my previous comment?
Thank you,
Katerina
I am unsure as to what is causing this issue, as we cannot recreate the error nor has it been reported elsewhere. Can I ask you to try running the matrixGenerator separately? You can do this quite easily from an interactive python session. If this works, you can then run the extractor using the final generated matrices, rather than starting from vcf files each time.
$python3
from SigProfilerMatrixGenerator.scripts import SigProfilerMatrixGeneratorFunc as matGen matrices = matGen.SigProfilerMatrixGeneratorFunc("hps", "GRCh38", "/hps/filt_vcfs/subs/")
If this give you an error still, you can simply comment out line 303, in SigProfilerMatrixGeneratorFunc:
Best, Erik
Thanks Erik! I got the log file error below after commenting out line 303 from SigProfilerMatrixGeneratorFunc.py. Looking through SigProfilerMatrixGeneratorFunc.py, I see the following:
300 if os.path.exists(error_file): 301 os.remove(error_file) 302 if os.path.exists(log_file): 303 # os.remove(log_file) 304 sys.stderr = open(error_file, 'w')
#########
Traceback (most recent call last):
File "
###########
Also I don't understand why the script looks into '/hps/filtvcfs/subs/logs/SigProfilerMatrixGenerator/hps/_GRCh382020-07-08.err'
It should be looking into something more like '/hps/filt_vcfs/subs/logs/SigProfilerMatrixGenerator_subs_GRCh382020-06-28.err'
What do you think?
Thanks,
Katerina
You can also comment out "os.remove(errorfile)" as that is now giving you the issue. It appears from the error output that you are using "/hps/" as your project name, hence the tool is looking for a file called "SigProfilerMatrixGenerator/hps/_GRCh382020-07-08.err." You should use "subs" as your project name if you would like the file to find 'SigProfilerMatrixGenerator_subs_GRCh382020-06-28.err.'
Best, Erik
Hi Erik,
I am reopening this ticket as I encountered more problems while running SigProfilerExtractor.
I run SigProfilerExtractor from a fresh installation in a new environment. I got the errors below when I commented out lines 302 and 304 from SigProfilerMatrixGeneratorFunc.py as suggested before.
(SigProfilerExtractor) [ac630@compute-e-16-233 scripts]$ python run_subs_signatures.py
Traceback (most recent call last):
File "run_subs_signatures.py", line 4, in
File "/n/katerina/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerMatrixGenerator/scripts/SigProfilerMatrixGeneratorFunc.py", line 302
if os.path.exists(log_file):
^
IndentationError: expected an indented block
(SigProfilerExtractor) [ac630@compute-e-16-233 scripts]$ python run_subs_signatures.py
Traceback (most recent call last):
File "run_subs_signatures.py", line 4, in
File "/n/katerina/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerMatrixGenerator/scripts/SigProfilerMatrixGeneratorFunc.py", line 304
sys.stderr = open(error_file, 'w')
^
IndentationError: expected an indented block
If I don't comment out lines 302 and 304 from SigProfilerMatrixGeneratorFunc.py, SigProfilerExtractor additionally complains for the following:
Traceback (most recent call last):
File "
SigProfilerExtractor creates a logs directory in the input file directory. It generates the log files in there which later on reports as not found. I am not sure why it does that as I can see the files exist. Your input will be much appreciated.
Many thanks, Katerina
Hi Erik, Have been having the same problem as Katerina above and I have been trying to troubleshoot it to no avail. I have the newest versions of SigProfile so I'm not entirely sure why these errors are still popping up? please let me know how best to fix it. The error is that it is not recognizing the report (.err) file despite it existing in the exact same path it is looking for. Thanks Amatta
Hello SigProfilerExtractor developers,
I used SigProfilerExtractor for de novo extraction of mutational signatures on whole genome sequencing .vcf files. I run your python package and I got the error messages attached below which I think are related to the fonts of the output plots. Could you please advise me on how to fix this?
Many thanks,
Katerina
line 602, in open_for_read_by_name return open(name,mode) FileNotFoundError: [Errno 2] No such file or directory: 'Arial Bold.ttf' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/lib/utils.py", line 661, in open_for_read return getBytesIO(datareader(name) if name[:5].lower()=='data:' else urlopen(name).read()) File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/urllib/request.py", line 222, in urlopen return opener.open(url, data, timeout) File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/urllib/request.py", line 510, in open req = Request(fullurl, data) File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/urllib/request.py", line 328, in init self.full_url = url File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/urllib/request.py", line 354, in full_url self._parse() File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/urllib/request.py", line 383, in _parse raise ValueError("unknown url type: %r" % self.full_url) ValueError: unknown url type: 'Arial Bold.ttf' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/pdfbase/ttfonts.py", line 155, in TTFOpenFile f = open_for_read(fn,'rb') File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/lib/utils.py", line 663, in open_for_read raise IOError('Cannot open resource "%s"' % name) OSError: Cannot open resource "Arial Bold.ttf" During handling of the above exception, another exception occurred: Traceback (most recent call last): File "run_subs_signatures.py", line 4, in
from SigProfilerExtractor import sigpro as sig
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor/sigpro.py", line 33, in
from SigProfilerExtractor import subroutines as sub
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor/subroutines.py", line 22, in
from SigProfilerExtractor import PlotDecomposition as sp
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor/PlotDecomposition.py", line 25, in
from SigProfilerExtractor import PlotDecomposition_SBS96 as spd_96
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/SigProfilerExtractor/PlotDecomposition_SBS96.py", line 40, in
pdfmetrics.registerFont(TTFont('Arial-Bold', 'Arial Bold.ttf'))
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/pdfbase/ttfonts.py", line 1196, in init
self.face = TTFontFace(filename, validate=validate, subfontIndex=subfontIndex)
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/pdfbase/ttfonts.py", line 1090, in init
TTFontFile.init(self, filename, validate=validate, subfontIndex=subfontIndex)
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/pdfbase/ttfonts.py", line 457, in init
TTFontParser.init(self, file, validate=validate,subfontIndex=subfontIndex)
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/pdfbase/ttfonts.py", line 179, in init
self.readFile(file)
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/pdfbase/ttfonts.py", line 255, in readFile
self.filename, f = TTFOpenFile(f)
File "/hps/anaconda3/envs/SigProfilerExtractor/lib/python3.7/site-packages/reportlab/pdfbase/ttfonts.py", line 165, in TTFOpenFile
raise TTFError('Can\'t open file "%s"' % fn)
reportlab.pdfbase.ttfonts.TTFError: Can't open file "Arial Bold.ttf"