Open GoogleCodeExporter opened 9 years ago
It looks like you are missing the library libRblas.so. I would say try
installing it and see if the problem persists.
Original comment by alexge...@gmail.com
on 5 Aug 2011 at 6:57
Yes, I am trying to do it.
Federica
Original comment by federica...@gmail.com
on 8 Aug 2011 at 11:48
I found the missing library, it's at /applications/R-2.11.1/lib/libRblas.so
I added it to the library path:
export LD_LIBRARY_PATH=/applications/R-2.11.1/lib:$LD_LIBRARY_PATH
and that particular error is gone. We need to do this in all scripts before
running the R executable.
but now there are other errors, so I'm debugging those as we speak.
Alex
Original comment by alexge...@gmail.com
on 12 Aug 2011 at 9:16
Hi,
thank you for your help! SO, I added this path to my .bashrc file, my bash
profile and so on...but still I have a pending question. Probably is a naive
thing. I am using the local pipeline client right? And I am connected to
fgene3..but I am connected in bash or tsch? Because I am having the same error:
/applications/R-2.11.1/bin/exec/R: error while loading shared libraries:
libRblas.so: cannot open shared object file: No such file or directory
Error:
can not find program /applications/R-2.11.1/bin/exec/R at
/projects1/test_pipeline/NEW_ORGANIZATION/scripts/cnv-seq.pl line 124.
...so I was asking myself if I have to change something also in my .cshrc file.
Fed
Original comment by federica...@gmail.com
on 15 Aug 2011 at 7:43
To be safe, add it to .cshrc and .tcshrc as well:
setenv LD_LIBRARY_PATH /applications/R-2.11.1/lib:${LD_LIBRARY_PATH}
When i login, it reads .cshrc, not .bashrc. Have you tried this? Does it still
not work?
Original comment by alexge...@gmail.com
on 17 Aug 2011 at 9:24
ok, I have modified also the .cshrc... actually I don't have a .tcshrc.....BTW
I have the same error again:
/applications/R-2.11.1/bin/exec/R: error while loading shared libraries:
libRblas.so: cannot open shared object file: No such file or directory
Error:
can not find program /applications/R-2.11.1/bin/exec/R at
/projects1/test_pipeline/NEW_ORGANIZATION/scripts/cnv-seq.pl line 124.
Fe
Original comment by federica...@gmail.com
on 18 Aug 2011 at 5:16
Oh sorry I misunderstood. If you run this program from a shell session (not the
pipeline client), the environment variables will be set because .cshrc is read
when you log into the shell.
From pipeline, we need to set the environment variables through a wrapper
script. I've done that ( see attached). But again, there are new errors that
I'm working on right now. I'll update you with those soon.
Alex
Original comment by alexge...@gmail.com
on 18 Aug 2011 at 6:04
Attachments:
Oki, thank you so much!
fed
Original comment by federica...@gmail.com
on 18 Aug 2011 at 6:15
Yep, I have this one (donno if it can help):
Illegal division by zero at
/projects1/test_pipeline/NEW_ORGANIZATION/scripts/cnv-seq.pl line 168, <TEST>
line 1207.
Fed
Original comment by federica...@gmail.com
on 18 Aug 2011 at 6:34
Attachments:
I tried again after the update of teh pipeline, but still I have the same
error...
federica
Original comment by federica...@gmail.com
on 6 Sep 2011 at 8:01
I'm still working on debugging this, it's slow going though because my
connection to fgene1 gets dropped frequently.
Original comment by alexge...@gmail.com
on 6 Sep 2011 at 8:37
yep, I had the same problem from off campus when I was in Italy...do you want
me to signal this problem to our system admin? Can I do something to help you
out?
Fed
Original comment by federica...@gmail.com
on 6 Sep 2011 at 9:36
We will contact him.
Original comment by alexge...@gmail.com
on 7 Sep 2011 at 6:33
I am writing some technical emails to Bernard, so do you want me to write him
also about this one?
fed
Original comment by federica...@gmail.com
on 9 Sep 2011 at 5:34
sure, you can mention the occasional issues connecting to the server remotely
if you want. Something tells me he already knows though :)
Original comment by alexge...@gmail.com
on 9 Sep 2011 at 5:53
Now there are not errors...but it's running since 5 days, just to keep track of
it.
fed
Original comment by federica...@gmail.com
on 15 Sep 2011 at 5:37
Attachments:
same here, been going for almost two days now. Does this seem reasonable to
you?
Original comment by alexge...@gmail.com
on 15 Sep 2011 at 6:01
Actually no, it runs super fast usually...
Original comment by federica...@gmail.com
on 15 Sep 2011 at 6:04
Ok, I assume this is taking too long. I've been messing with the source for a
while to debug, and although I don't think I did anything intrusive, just in
case, I did a clean installation of the cnv-seq package to reverse all my
changes and see if we still get an infinite loop. If the failures from before
were a result of the strange server issues we were having, this might fix a lot.
I'm running it now, I'll let you know.
Alex
Original comment by alexge...@gmail.com
on 19 Sep 2011 at 6:27
Attachments:
Thanks, I am doing the same right away.
fed
Original comment by federica...@gmail.com
on 19 Sep 2011 at 6:46
Hi,
I ran the module, and this time the CNVseq run was ok. BTW, there is a problem
in the CNV-R connection on the system...as the cnv library is not present in
the server (see pasted error and pics:
File "<string>", line 1
print len(''chromosome=2,from=140036061,to=144238634''.split())
^
SyntaxError: invalid syntax
/projects1/test_pipeline/NEW_ORGANIZATION/scripts/cnv-seq-R.sh: line 13: [:
-gt: unary operator expected
Error in library(cnv) : there is no package called 'cnv'
Execution halted
Error in library(cnv) : there is no package called 'cnv'
Execution halted
cp: cannot stat `*.pdf': No such file or directory
Actually I have installed the package on my local mac with this procedure:
> install.packages("cnv")
> ./R CMD INSTALL /applications/CNVseq/cnv-seq/cnv/
Have you installed this package on your side? I remember that we had some
issues with this package on fgene...and here I think is the reason of the
failure..
Fed
Original comment by federica...@gmail.com
on 20 Sep 2011 at 12:09
Attachments:
More in detail here is what I did on my Unix local machine:
CNVseq: LOCAL MAC
-go to the folder with the cnv/folder and:
Federica-Torris-iMac:cnv_seq federicatorri$ R CMD INSTALL cnv/
-from GUI install the necessary library
-then run
./cnv-seq.pl --test JLK-227.hits --ref JAM-230.hits --genome human
I remember that on our server I was facing a lot of problems and that also with
Bernard finally the issue was not fixed..and then I ha dto run everything on my
local machine.
Sorry to bug you and hope it may help to see what happens on your Linux server
with the same issue.
Fed
Original comment by federica...@gmail.com
on 20 Sep 2011 at 12:18
Oh, excellent. I'm past the CNV-seq step as well. Fixing CNV-R shouldn't be too
complicated, looks like a simple python syntax error, probably my bad. I'll get
back to you soon.
Alex
Original comment by alexge...@gmail.com
on 20 Sep 2011 at 2:53
Hi Alex,
I want only to add something as I think I haven't explained everything properly
(sorry to re-bug you if I was clear instead :) ).
I fixed the problem going "around it" and using my local mac but CNVseq must
run on a linux server given the great amount of data. SO I explained to you how
I fixed it "locally" to see if you also can help me out (looking at how you
deal with it on your server) in understanding how the problem can be fixed also
on fgene, bc I have not been able to fix it to date by myself.
Thanks,
Fede
Original comment by federica...@gmail.com
on 20 Sep 2011 at 5:18
Yeah, so the CNV R library is missing on fgene1. We went through this on our
servers as well and I think installing CNV fixed the problem (looks like it
fixed the problem on your local machine). Should I email Bernard about
installing this package on fgene?
Is it just a simple `R CMD INSTALL /applications/CNVseq/cnv-seq/cnv/` ?
Original comment by alexge...@gmail.com
on 20 Sep 2011 at 6:35
Yep, I will email him about it cc-ing you. Please let me know also if you hear
from him abotu abyss or if I have to ask him again something.
fed
Original comment by federica...@gmail.com
on 20 Sep 2011 at 6:39
Oh no sorry, ABYSS is with Sam...:)
Original comment by federica...@gmail.com
on 20 Sep 2011 at 6:48
Hi Alex,
bernard pasted my this:
[root@fgene2 bin]# ./R CMD INSTALL /applications/CNVseq/cnv-seq/cnv/
* installing to library â/applications/R-2.11.1/libraryâ
* installing *source* package âcnvâ ...
** R
** preparing package for lazy loading
Loading required package: reshape
Loading required package: plyr
Attaching package: 'reshape'
The following object(s) are masked from 'package:plyr':
round_any
Loading required package: grid
Loading required package: proto
** help
No man pages found in package âcnvâ
*** installing help indices
** building package indices ...
** testing if installed package can be loaded
* DONE (cnv)
[root@fgene2 bin]#
I think now is ok also on our system..I will give a try.
I'll let you know.
Tks
Fed
Original comment by federica...@gmail.com
on 20 Sep 2011 at 9:26
The problem now is that CNV-seq was generating an unpredictable output .cnv
file, so the pipeline module couldn't pass it to CNVR and CNVR was failing
because of the missing file. I fixed CNV-seq to properly extract the output
filename. If you have an active session of this workflow right now, just
restart the CNV-seq module. I'm running it now.
btw the last time i ran cnv-seq it ran for several hours, but it completed. the
current run is taking over an hour as well. should we be concerned about this?
alex
Original comment by alexge...@gmail.com
on 21 Sep 2011 at 7:54
I have restarted it right now. About the timing I remember that on my local
machine was taking alot as well, even if less than days like that weird past
run..so I think that everything is ok..I am eager to see the output files, than
I;ll be able to tell you that is working ok.
Tks!
Fed
Original comment by federica...@gmail.com
on 21 Sep 2011 at 8:46
Hi, the run was successful this time...everything's green! It's great!
There is only something I need to change and discuss with you.
(1) First of all there is a duplication of the outputs:
[ftorri@fgene2 2011September21_13h36m46s971ms]$ ll -h
total 3.4G
-rw-rw-rw- 1 pipeline pipeline 155 Sep 21 19:37 CNV-R_1.OutputDataDescription-1
-rw-rw-rw- 1 pipeline pipeline 1.1K Sep 21 19:37 CNV-R_1.OutputPlot-1.pdf
-rw-rw-rw- 1 pipeline pipeline 111M Sep 21 19:33 CNV-Seq_1.OutputCNVFile-1.cnv
-rw-rw-rw- 1 pipeline pipeline 1.2G Sep 21 13:56 ExtractHits_1.OutputHits-1.hits
-rw-rw-rw- 1 pipeline pipeline 2.0G Sep 21 14:10 ExtractHits_2.OutputHits-1.hits
-rw-rw-rw- 1 pipeline pipeline 111M Sep 21 19:33
ExtractHits_2.OutputHits-1.hits-vs-ExtractHits_1.OutputHits-1.hits.log2-0.6.pval
ue-0.001.minw-4.cnv
-rw-rw-rw- 1 pipeline pipeline 29M Sep 21 14:26
ExtractHits_2.OutputHits-1.hits-vs-ExtractHits_1.OutputHits-1.hits.log2-0.6.pval
ue-0.001.minw-4.count
-rw-rw-rw- 1 pipeline pipeline 1.1K Sep 21 19:37 Rplots.pdf
drwxrwxrwx 2 pipeline pipeline 4.0K Sep 21 19:33 streams
The CNV-Seq_1.OutputCNVFile-1.cnv and the
ExtractHits_2.OutputHits-1.hits-vs-ExtractHits_1.OutputHits-1.hits.log2-0.6.pval
ue-0.001.minw-4.cnv are the SAME thing. So it's better that the .cnv file has
the name
:ExtractHits_2.OutputHits-1.hits-vs-ExtractHits_1.OutputHits-1.hits.log2-0.6.pva
lue-0.001.minw-4.cnv (bc incorporates the information about the comparison
between cases and controls). I think the issue is that with the pipeline we
have specified the output name as CNV-Seq_1.OutputCNVFile-1.cnv, while the
software per se is producing by default an output with the other name also.
(2) There is also something related to the logic of the last step (CNV-R). The
pdf plot is empty and the reason is that for the testing you use the interval
chromosome=2,from=140036061,to=144238634. We are demonstrating that the command
line has worked, and this is awesome! BTW the plot is empty bc probably in
this patient we don't have a deletion in this genomic interval. I describe you
the logical flow of what the user might want to do.
-automatically EXPORT ALL the picture of the CNVs in all the genome
-automatically EXPORT ALL the cnv in a specific chromosome
-EXPORT ONLY some CNVs he found interesting after having examinated the .cnv
file. So here the user might need to stop the pipeline run at the CNVseq step,
download the .cnv file, give a look at that and once he has a list of the
important events he would need to go back to the same workflow and run the
CNV-R step with the coordinates or the CNVid he wants to plot (in the .cnv file
the user can in fact find either the coordinates of the CNV or a specific ID
that is a number).
I am discovering in these days these new options of outputting. So in summary,
this a great success because this module was really hard to make it working,
tks for your help. I am attaching the updated version of the documentation
where I tried to explain everything. I am really interested in knowing what do
you think about those changes: do you think are too much complex and would
require too much work?
The other output of CNV-R module (production of the data description file) is
ok and perfect like that.
Thanks,
Federica
Original comment by federica...@gmail.com
on 22 Sep 2011 at 6:03
Attachments:
Yes, this completed for me too.
(1) This was a somewhat quick fix on my part. The reason is that the long
filename
(ExtractHits_2.OutputHits-1.hits-vs-ExtractHits_1.OutputHits-1.hits.log2-0.6.pva
lue-0.001.minw-4.cnv) is generated by CNV-seq at runtime, and not specified on
the command line beforehand. So Pipeline doesn't know what this filename will
be, and therefore can't access it. So as a workaround, after cnv-seq completes,
there is a cp step that copies any *.cnv file in the working directory to a
pipeline-generated filename (CNV-Seq_1.OutputCNVFile-1.cnv). Otherwise, the
next module won't be able to find the .cnv file.
If this is going to be an issue, I'm sure I could figure out a way for pipeline
to assemble the filename at runtime based on parameters. Let me know if you
think this is worthwhile.
(2) I just copied those arguments exactly from whatever documentation I was
reading at the time, so I'm not surprised it's irrelevant.
For the first case (full genome): is this just a matter of leaving out the
chromosome= and from=,to= flags? Is this the default behavior? Otherwise how do
you specify graphing the entire genome?
Second and third case: These will need to be run in separate pipeline
workflows. The process would involve completing the workflow, examining the
output, then doing a "copy with input" on the CNV-R module, pasting it into a
blank workflow, and running it with the desired new parameters.
The other options is to send the output of CNV-seq to several parallel
instances of CNV-R, each running a different set of parameters. You would have
to have a general idea of what areas you are interested in though. I can't
think of a more elegant solution at the moment.
I'm going to test the CNV-R module further to make sure it can generate a
proper pdf. Let me know if anything is unclear, or anything else I can put
together.
Alex
Original comment by alexge...@gmail.com
on 22 Sep 2011 at 7:01
Thank you for your quick reply! I really appreciate your suggestions.
(1) No I mean for the doubled name we can by now leave it like that (describing
btw what you told me, so the user won't be surprised).
(2) Actually I was testing the command lines but they are not working well ( I
mean the whole chr and whole genome)..so at this point maybe is better to make
things simple and to allow the user to export the regions he wants..that brings
us to the last point.
The idea to split the two workflows is the best one in my opinion ...maybe I
would keep them in the same canva like in the pic I attach here. The only thing
I would like to ask you is where the user can specify the genomic coordinates
or the CNVid he wants to visualize. I saw that in the parameter is explained an
example of the posisble arguments...but where is phisically the command line?
In any script? The thing that may be useful is a place where the user can enter
the parameters choosing the genomic location of the event he wants to visualize.
The command line may ask for regions: plot.cnv(data, chromosome=2,
from=140036061, to=144238634)
or CNvid: plot.cnv(data, CNV=4, upstream=4e+6, downstream=4e+6)
For the testing you are doing I suggest you to use the region chromosome 3 from
78154481 to 78160016 where actually there is a CNV in the tested case.
Fede
Original comment by federica...@gmail.com
on 22 Sep 2011 at 7:57
Attachments:
So all the arguments to plot.cnv (besides "data") are specified in the "Plot
Arguments" input parameter of the CNV-R module. You simply list the arguments
as they should be called in R. For example, if you want to call plot.cnv(data,
chromosome=2, from=140036061, to=144238634), you simply enter the string:
chromosome=2, from=140036061, to=144238634
The pipeline will automatically escape the '=' and whitespace, so you don't
need to worry about formatting, as long as they are specified with the proper R
syntax. (the description of this parameter says no spaces, but go ahead and use
spaces if you want, I'll remove this warning).
That being said, I just ran this with these parameters that you suggested and
got a failure:
Error in eval(expr, envir, enclos) : object 'position' not found
Calls: print ... lapply -> is.vector -> lapply -> FUN -> eval -> eval
In addition: Warning messages:
1: In max(na.omit(data$p.value)) :
no non-missing arguments to max; returning -Inf
2: In match.fun(get(".transform", .))(values) : NaNs produced
Execution halted
I'll investigate further, but does this tell you anything off the bat?
Alex
Original comment by alexge...@gmail.com
on 22 Sep 2011 at 10:12
Hi,
thanks. I am running it on another region I foudn to be affected by a deletion
in the sample. ("22" 49577649 49583184) I don't have idea about the reason of
this error you are getting, I have never had it..lets see what comes out with
my run.
fed
Original comment by federica...@gmail.com
on 22 Sep 2011 at 10:51
So this works for me now. Attached the output of my previous run with
chromosome=2,from=140036061,to=144238634 (same as before)
Original comment by alexge...@gmail.com
on 26 Sep 2011 at 6:32
Attachments:
Hi,
I am running CNVseq on a region with CNV..it is taking a lot (3 days ) and I
think it is stalling.I am n ot having nay error though. The module si exaclty
the same, I have only changed the interval as you can see from the shot. Have
you any news about your run?
Fed
Original comment by federica...@gmail.com
on 26 Sep 2011 at 6:34
Attachments:
just posted about this. CNV-seq tool 6 hours for me. Is the output stream
still reporting? If it's still doing work, it will print something like [1]
"cnv_id: 33634 of 33641" with the numbers incrementing. Otherwise, i would
restart it.
And if you didn't notice my post, check it out.
Original comment by alexge...@gmail.com
on 26 Sep 2011 at 6:41
Hi,
I have already restarted the module this morning as the output stream was not
reporting anymore. I hope it will end with a not daily timing...your plot is
perfect. Can you copy the working pipe in the library? SO I'm sure to have your
working copy.
tks
Federica
Original comment by federica...@gmail.com
on 26 Sep 2011 at 9:32
here it is. I don't have permissions to add to the pipeline library. i can put
it somewhere on fgene if you want though.
Original comment by alexge...@gmail.com
on 26 Sep 2011 at 9:41
Attachments:
I'll do that, no problem ...many thanks!
fed
Original comment by federica...@gmail.com
on 26 Sep 2011 at 9:49
Perfect, it ran for me as well with the plot.
Fed
Original comment by federica...@gmail.com
on 28 Sep 2011 at 12:16
Hi, when using the .cnv output from cnv-seq.pl and trying to plot with
library(cnv) from cnv-seq in R there appears an error like this. I am upset
because I have generated 7 files and some of them are working fine and some not
and they are generated in the same way. Here I attach the error in R cran:
> library(cnv)
> data <- read.delim("my_data.cnv")
> cnv.print(data)
cnv chromosome start end size log2 p.value
CNVR_1 chr Inf -Inf -Inf
CNVR_0 chrY chrX chrMT chr1 chr2 chr3 chr4 chr5 chr6
chr7 chr8
chr9 chr10 chr11 chr12 chr13 chr14 chr15 chr16 chr17 chr18
398
315321616 315321219 NA NA
Warning messages:
1: In min(sub$start) : no non-missing arguments to min; returning Inf
2: In min(sub$position) : no non-missing arguments to min; returning Inf
3: In max(sub$end) : no non-missing arguments to max; returning -Inf
4: In max(sub$position) : no non-missing arguments to max; returning -Inf
Thank you very much in advance!
Original comment by anna.pui...@gmail.com
on 25 Mar 2014 at 11:34
Hello, Anna
I had an exactly same problem with you..
Did you solve this problem?? I also could not use cnv.print.. (same error with
you) but, some of dataset works fine.. ;( too weird..
Somebody any helps??
Original comment by younhee...@gmail.com
on 3 Jun 2014 at 1:03
Original issue reported on code.google.com by
federica...@gmail.com
on 4 Aug 2011 at 11:30Attachments: