PaulaAlessio
commented
4 years ago I am afraid I am responsible forma that. I pushed some code and forgot to uncomment linea that I was commenting forma testing. Could you try again now?
Closed Vzplau closed 4 years ago
PaulaAlessio
commented
4 years ago I am afraid I am responsible forma that. I pushed some code and forgot to uncomment linea that I was commenting forma testing. Could you try again now?
Vzplau
commented
4 years ago Hi Paula,
No worries - thank you for letting me know, and for the quick reply. It still hasn't worked for me, unfortunately - what do you recommend?
The attached doc shows me building FastqPuri again, running the same code (cmake) and doing the build again - to no html generated report.
I tried also deleting the build folder and running cmake again to install a new build, but it didn't work either. I haven't included the code for this due to the amount of volume it would take up, but please let me know if this would help.
Fastqpuri no html report still, June 26 2020.docx
Thank you,
Victor
PaulaAlessio
commented
4 years ago Hi Victor,
Uhm, have you done a git pull in the first place? Can you do : git log in your repository and check that it looks like this:
commit 943eaa7e8fd6c21925325a700e15470fcd7081b0 (HEAD -> master, origin/master, origin/HEAD)
Author: perezp <pauperez@gmv.com>
Date: Fri Jun 26 09:20:30 2020 +0200
Uncomment commented lines
commit 62858391f0eb5fb2c1a0b9227ef77278b8c1c576
Author: Paula&Ale <paulaperezrubio@gmail.com>
Date: Thu Jun 18 23:37:39 2020 +0200
Update README.md
commit 8a3aaf915ca8f62d5a84be8067289ec418e60874
Author: perezp <pauperez@gmv.com>
Date: Thu Jun 18 23:34:28 2020 +0200
Issue #7: solved compilation error lacking R dependenciesThat is, the last commit message should be Uncomment commented lines. If you do not get that commit, run git pull on the repository.
I might have overlooked it, but I think the build process went OK. If there had been a problem with the R dependencies, the following message would have appeared:
-- WARNING: Package will be compiled but R script will not
be called. Something missing.
Check that you have installed:
- Rscript
- pandoc
- R packages: rmarkdown, knitr, pheatmap Make sure you do not get this while building the project.
Vzplau
commented
4 years ago Hi Paula,
Thanks for the quick response again. It still doesn't work for me; sorry about this.
I didn't do a git pull before your comment, but I did create a new git repository from scratch (using git init) and ran the git pull (https://github.com/jengelmann/FastqPuri.git), redid the cmake (to sudo make install), and tested a new Qreport. However, it still doesn't work:
root@ee6cc0c294bc:/tmp/example/Qreport_Sreport/run_test# Qreport -i ../test.fq.bz2 -l 51 -o my_test_output
Qreport from FastqPuri
- Input file: ../test.fq.bz2
- Read length: 51
- Number of tiles: 96
- Min quality: 27
- Qualities for properties plot: 27,33,37
- Output bin-file : my_test_output.bin
- Output html-file : my_test_output.html
- Output info-file: my_test_output.info
Starting Qreport at: Sat Jun 27 03:27:42 2020
- Reading a filtered file? no.
- Finished reading file.
WARNING: expected 96 tiles but found only 20.
- Writing data structure to file my_test_output.bin.
WARNING: html reports are NOT being generated.
Dependencies not fulfilled.
Finishing program at: Sat Jun 27 03:27:53 2020
Time elapsed: 5.092614 s.
Fastqpuri June 26 2020 pt.2, git pull.docx ^ Steps for doing the git init, git pull, getting the updated version of FastqPuri. I tested the Qreport again (refer to page 17-18 of the word doc), and entered 'git log' after to make sure I was up to date. The log for this can be referred to within pg. 18 of the word doc:
victorlau@Victors-MacBook-Air FastqPuri-master % git log
commit 0cfa4e0e1588f16dde93eecf189f4f80b9d51cba (HEAD -> master)
Author: Claudio Lottaz <claudio.lottaz@ur.de>
Date: Fri Jun 26 13:11:59 2020 +0200
Release 1.0.8 local installation
Changes to be committed:
modified: CMakeLists.txt
modified: README.md
commit 943eaa7e8fd6c21925325a700e15470fcd7081b0
Author: perezp <pauperez@gmv.com>
Date: Fri Jun 26 09:20:30 2020 +0200
Uncomment commented lines
I also checked to make sure that there were any warning messages (couldn't find nay), and also that the R dependencies were installed:
-- Check for pandoc: /usr/local/bin/pandoc
-- Check for Rscript: /usr/local/bin/Rscript
-- Check for gunzip: /usr/bin/gunzip
-- Check for gzip: /usr/bin/gzip
-- Found R_rmarkdown: TRUE
-- Found R_pheatmap: TRUE
-- Found R_knitr: TRUE What else can I try to solve this? Please let me know your thoughts.
Sincerely,
Victor
Vzplau
commented
4 years ago I did want to add something I'm not sure about as well; when I tried using the following command to restart a new git repository, (as the old one had trouble downloading and merging the files), I did the following:
sudo find . -type f | grep -i "\.git" | xargs rmI received this after (at the beginning of the pull), but I still got to the same comments and latest commits as mentioned in the above post ( commit 0cfa4e0e1588f16dde93eecf189f4f80b9d51cba (HEAD -> master) Author: Claudio Lottaz claudio.lottaz@ur.de Date: Fri Jun 26 13:11:59 2020 +0200).
Does this help?
victorlau@Victors-MacBook-Air FastqPuri-master % git pull https://github.com/jengelmann/FastqPuri.git
warning: Pulling without specifying how to reconcile divergent branches is
discouraged. You can squelch this message by running one of the following
commands sometime before your next pull:
git config pull.rebase false # merge (the default strategy)
git config pull.rebase true # rebase
git config pull.ff only # fast-forward only
You can replace "git config" with "git config --global" to set a default
preference for all repositories. You can also pass --rebase, --no-rebase,
or --ff-only on the command line to override the configured default per
invocation.
remote: Enumerating objects: 24, done.
remote: Counting objects: 100% (24/24), done.
remote: Compressing objects: 100% (18/18), done.
remote: Total 5322 (delta 10), reused 15 (delta 6), pack-reused 5298
Receiving objects: 100% (5322/5322), 90.27 MiB | 995.00 KiB/s, done.
Resolving deltas: 100% (3609/3609), done.
From https://github.com/jengelmann/FastqPuri
* branch HEAD -> FETCH_HEAD
error: The following untracked working tree files would be overwritten by merge:
CMakeLists.txt
Dockerfile
Doxyfile
Doxyfile.bak
LICENSE.txt
PANDOC.md
R/quality_report.Rmd
R/style.css
R/summary_filter_report.Rmd
R/summary_filter_reportDS.Rmd
R/summary_report.Rmd
R/utils.R
README.md
README_Qreport.md
README_Sreport.md
README_makeBloom.md
README_makeTree.md
README_trimFilter.md
README_trimFilterPE.md
Singularity
Singularity-Test
cmake/Rpkg_check.cmake
cmake/checkHeadersFunctions.cmake
config.h.in
doxygen_sqlite3.db
html/Lmer_8c.html
html/Lmer_8c__incl.map
html/Lmer_8c__incl.md5
html/Lmer_8c__incl.png
html/Lmer_8h.html
html/Lmer_8h__dep__incl.map
html/Lmer_8h__dep__incl.md5
html/Lmer_8h__dep__incl.png
html/Lmer_8h_source.html
html/Qreport_8c.html
html/Qreport_8c__incl.map
html/Qreport_8c__incl.md5
html/Qreport_8c__incl.png
html/Rcommand__Qreport_8c.html
html/Rcommand__Qreport_8c__incl.map
html/Rcommand__Qreport_8c__incl.md5
html/Rcommand__Qreport_8c__incl.png
html/Rcommand__Qreport_8h.html
html/Rcommand__Qreport_8h__dep__incl.map
html/Rcommand__Qreport_8h__dep__incl.md5
html/Rcommand__Qreport_8h__dep__incl.png
html/Rcommand__Qreport_8h_source.html
html/Rcommand__Sreport_8c.html
html/Rcommand__Sreport_8c__incl.map
html/Rcommand__Sreport_8c__incl.md5
html/Rcommand__Sreport_8c__incl.png
html/Rcommand__Sreport_8h.html
html/Rcommand__Sreport_8h__dep__incl.map
html/Rcommand__Sreport_8h__dep__incl.md5
html/Rcommand__Sreport_8h__dep__incl.png
html/Rcommand__Sreport_8h_source.html
html/Sreport_8c.html
html/Sreport_8c__incl.map
html/Sreport_8c__incl.md5
html/Sreport_8c__incl.png
html/adapters_8c.html
html/adapters_8c__incl.map
html/adapters_8c__incl.md5
html/adapters_8c__incl.png
html/adapters_8h.html
html/adapters_8h__dep__incl.map
html/adapters_8h__dep__incl.md5
html/adapters_8h__dep__incl.png
html/adapters_8h__incl.map
html/adapters_8h__incl.md5
html/adapters_8h__incl.png
html/adapters_8h_source.html
html/annotated.html
html/bc_s.png
html/bdwn.png
html/bloom_8c.html
html/bloom_8c__incl.map
html/bloom_8c__incl.md5
html/bloom_8c__incl.png
html/bloom_8h.html
html/bloom_8h__dep__incl.map
html/bloom_8h__dep__incl.md5
html/bloom_8h__dep__incl.png
html/bloom_8h__incl.map
html/bloom_8h__incl.md5
html/bloom_8h__incl.png
html/bloom_8h_source.html
html/city_8c.html
html/city_8c__incl.map
html/city_8c__incl.md5
html/city_8c__incl.png
html/city_8h.html
html/city_8h__dep__incl.map
html/city_8h__dep__incl.md5
html/city_8h__dep__incl.png
html/city_8h__incl.map
html/city_8h__incl.md5
html/city_8h__incl.png
html/city_8h_source.html
html/citycrc_8h.html
html/citycrc_8h__incl.map
html/citycrc_8h__incl.md5
html/citycrc_8h__incl.png
html/citycrc_8h_source.html
html/classes.html
html/closed.png
html/config_8h_source.html
html/defines_8h.html
html/defines_8h__dep__incl.map
html/defines_8h__dep__incl.md5
html/defines_8h__dep__incl.png
html/defines_8h__incl.map
html/defines_8h__incl.md5
html/defines_8h__incl.png
html/defines_8h_source.html
html/dir_000001_000000.html
html/dir_68267d1309a1af8e8297ef4c3efbcdba.html
html/dir_68267d1309a1af8e8297ef4c3efbcdba_dep.map
html/dir_68267d1309a1af8e8297ef4c3efbcdba_dep.md5
html/dir_68267d1309a1af8e8297ef4c3efbcdba_dep.png
html/dir_d44c64559bbebec7f509842c48db8b23.html
html/doc.png
html/doxygen.css
html/doxygen.png
html/dynsections.js
html/fa__read_8c.html
html/fa__read_8c__incl.map
html/fa__read_8c__incl.md5
html/fa__read_8c__incl.png
html/fa__read_8h.html
html/fa__read_8h__dep__incl.map
html/fa__read_8h__dep__incl.md5
html/fa__read_8h__dep__incl.png
html/fa__read_8h__incl.map
html/fa__read_8h__incl.md5
html/fa__read_8h__incl.png
html/fa__read_8h_source.html
html/files.html
html/folderclosed.png
html/folderopen.png
html/fopen__gen_8c.html
html/fopen__gen_8c__incl.map
html/fopen__gen_8c__incl.md5
html/fopen__gen_8c__incl.png
html/fopen__gen_8h.html
html/fopen__gen_8h__dep__incl.map
html/fopen__gen_8h__dep__incl.md5
html/fopen__gen_8h__dep__incl.png
html/fopen__gen_8h__inc
Aborting
victorlau@Victors-MacBook-Air FastqPuri-master % git init
Initialized empty Git repository in /Users/victorlau/Docker/dockerfiles/FastqPuri-master/.git/Hi Victor,
Go through the steps below
# Clone the repo again
git clone https://github.com/jengelmann/FastqPuri.git NewFastqPuri
# Build and install the project
cd NewFastqPuri
cmake -H. -Bbuild/
cd build
make
sudo make install
# Test local executables
cd ../examples/Qreport_Sreport
../../bin/Qreport -i test.fq.bz2 -l 51 -o myFirstTest
# If that works, check that the installation went OK by doing
Qreport -i test.fq.bz2 -l 51 -o myFirstTest And let me know whether it works
Vzplau
commented
4 years ago Hi Paula,
Thanks for writing the code, and sorry again for taking up your time. The new code (after cloning and running the direct instructions you've provided), I couldn't get either of the following to work:
../../bin/Qreport -i test.fq.bz2 -l 51 -o myFirstTest
#Critical error seems to be: The file '/tmp/FastqPuri_N1xR2r/quality_report.Rmd' does not exist.
Qreport -i test.fq.bz2 -l 51 -o myFirstTest
#Critical error appears to be: The file '/tmp/FastqPuri_Vc2GoU/quality_report.Rmd' does not exist. I'm not sure what the FastqPuri_XXXXXX folders are, but the quality_report.Rmd file is installed within the R folder (inside the newly cloned NewFastqPuri repository).
FastqPuri html issues Qreport, June 28 2020.docx
I've included the direct scripts below, of where this problem happened. The above word doc contains all my step-to-step work.
- Current working dir: /Users/victorlau/Docker/dockerfiles/FastqPuri-master/NewFastqPuri/examples/Qreport_Sreport
- Rmd-file used to generate HTML: ./R/quality_report.Rmd
Could not open file to be read
Could not open file to be read
Could not open file to be read
- Running command: /usr/local/bin/Rscript -e " inputfile = normalizePath('myFirstTest.bin', mustWork = TRUE) ; output = 'myFirstTest.html';output_file = gsub('.*/', '', output);path = gsub('[^/]+$', '', output);if (path != '') { output_file = paste0(normalizePath(path, mustWork=TRUE), '/', output_file); } else {output_file = paste0('/Users/victorlau/Docker/dockerfiles/FastqPuri-master/NewFastqPuri/examples/Qreport_Sreport', '/', output_file); };rmarkdown::render('/tmp/FastqPuri_N1xR2r/quality_report.Rmd', params = list(inputfile = inputfile, filter=0, version = '1.0.8'), output_file = output_file)"
Error in abs_path(input) :
The file '/tmp/FastqPuri_N1xR2r/quality_report.Rmd' does not exist.
Calls: <Anonymous> -> setwd -> dirname -> abs_pathvictorlau@Victors-MacBook-Air Qreport_Sreport % Qreport -i test.fq.bz2 -l 51 -o myFirstTest
Qreport from FastqPuri
- Input file: test.fq.bz2
- Read length: 51
- Number of tiles: 96
- Min quality: 27
- Qualities for properties plot: 27,33,37
- Output bin-file : myFirstTest.bin
- Output html-file : myFirstTest.html
- Output info-file: myFirstTest.info
Starting Qreport at: Sun Jun 28 16:04:23 2020
- Reading a filtered file? no.
- Finished reading file.
WARNING: expected 96 tiles but found only 20.
- Writing data structure to file myFirstTest.bin.
- Creating html output in file: myFirstTest.html
- Current working dir: /Users/victorlau/Docker/dockerfiles/FastqPuri-master/NewFastqPuri/examples/Qreport_Sreport
- Rmd-file used to generate HTML: ./R/quality_report.Rmd
Could not open file to be read
Could not open file to be read
Could not open file to be read
- Running command: /usr/local/bin/Rscript -e " inputfile = normalizePath('myFirstTest.bin', mustWork = TRUE) ; output = 'myFirstTest.html';output_file = gsub('.*/', '', output);path = gsub('[^/]+$', '', output);if (path != '') { output_file = paste0(normalizePath(path, mustWork=TRUE), '/', output_file); } else {output_file = paste0('/Users/victorlau/Docker/dockerfiles/FastqPuri-master/NewFastqPuri/examples/Qreport_Sreport', '/', output_file); };rmarkdown::render('/tmp/FastqPuri_Vc2GoU/quality_report.Rmd', params = list(inputfile = inputfile, filter=0, version = '1.0.8'), output_file = output_file)"
Error in abs_path(input) :
The file '/tmp/FastqPuri_Vc2GoU/quality_report.Rmd' does not exist.
Calls: <Anonymous> -> setwd -> dirname -> abs_path
In addition: Warning message:
In normalizePath(path, winslash = winslash, mustWork = mustWork) :
path[1]="/tmp/FastqPuri_Vc2GoU/quality_report.Rmd": No such file or directory
Execution halted
Something went wrong when executing R script.
Most probably, a html file will not be generated.
Exiting program.
victorlau@Victors-MacBook-Air Qreport_Sreport % Looking forward to your advice.
Sincerely,
Victor
Vzplau
commented
4 years ago Hi Paula,
Thanks for checking in again, and giving advice regarding tmp. Ichecked it and it does exist, but I was able to solve the problem by another means (but only on the NewFastqPuri git clone, rather than the original). I'll post the solution and close the issue, but I essentially cloned the R folder (from the NewFastqPuri files) into the current working directory (in this case, cd Qreport_Sreport).
I've been able to generate the html reports now! Thank you so much for your help - I'd like to provide a $5 e-gift card for a coffee place, as a way to say thanks; however, I don't know any good places in Germany. I'll probably transfer this by your institutional e-mail, after finding it - please let me know what coffee shop (or tea etc.) works best with you!
Best,Victor
On Mon, Jun 29, 2020 at 6:27 AM Paula&Ale notifications@github.com wrote:
You're running this on a Mac, right? Can you chek that the folder /tmp exists?
- if it doesn't, create it with user permissions
- if it does, can you check that it does not exist?
Do you have a Linux machine? (You can install a virtual machine maybe to test it) or use the docker image instead
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/jengelmann/FastqPuri/issues/9#issuecomment-651120201, or unsubscribe https://github.com/notifications/unsubscribe-auth/AQBQ56RCRR5EIKC2A7XLTYDRZCJE7ANCNFSM4OIZW4JA .
PaulaAlessio
commented
4 years ago Hi Victor,
Vzplau
commented
4 years ago Hi Paula,
Thanks for further defining the git clone, and yes - I've deleted it. I really appreciate the help.
The current solution works for Qreport for me, but I've run into the problem of trying to generate html reports and using the R dependencies for Sreport. I'm looking forward to what you're able to discover or solve, when you have the time.
(Ah, sounds good! Hope things resolve faster in Spain, and definitely agree for having coffee at home :) )
PaulaAlessio
commented
4 years ago Hi Victor,
The problem is that there is a part of the code that is wrongly assuming the OS to be a Unix-Linux. It tries to get the path of the .Rmd file by using the /proc/ directory that I guess doesn't exist in macOS. I will try to find a workaround for that so that it also works for macOS. Could you maybe try to install it on a Linux machine, or use the docker image?
PaulaAlessio
commented
4 years ago Hi Víctor,
I borrowed a Mac and made it work on it. I will clean the code and commit the changes tomorrow, together with a fix for openBSD (we want to test it on an openBSD as well). I will let you know when we push the changes.
PaulaAlessio
commented
4 years ago The code was relying on the existence of the /proc virtual filesystem, which is absent in macOS. Now, the new code is pushed, can you clone the repo again and give us feedback?
Cheers,
Paula
Vzplau
commented
4 years ago Hi Paula,
I'm sorry I missed this message, and have been slow on the rest! I've ran the initial FastqPuri (before you advised to clone the repository) by using docker build, but I thank you (and Luca and the team) for creating the proc changes.
Qreport is working wonderfully for me now, but I am having trouble generating Sreport. I will post the changes in a future comment.
Hope you're doing well.
Best,
Victor
Hi Victor,
The problem is that there is a part of the code that is wrongly assuming the OS to be a Unix-Linux. It tries to get the path of the .Rmd file by using the /proc/ directory that I guess doesn't exist in macOS. I will try to find a workaround for that so that it also works for macOS. Could you maybe try to install it on a Linux machine, or use the docker image?
Vzplau
commented
4 years ago Hi Paula,
Regarding Sreport, I am unable to generate a html report. I've attached the troubled code (and my major steps in bold) below:
2020:6:20 FastqPuri Sreport Issue.docx
I've tried accessing /usr/local/bin and found a Rscript file there, but executing it just runs up a completed process / info dump on terminal.
I think the issue is that a temporary "FastqPuri_(6 letter randomly generated combo)" folder is generated through each Rscript run in Sreport, but each temporary FastqPuri folder is only able to access the style.css, summary_report.Rmd, and utils.R folders. The other three files found in the R subfolder are missing.
Any recommendations for this? I could try rerunning the docker image, but I don't know if this would work. Will be looking forward to hearing from you, as usual.
Sincerely,
Victor
PaulaAlessio
commented
4 years ago I am confused. You're still having trouble with Sreport with the "usual " installation or you're talking about the docker image? My fix was meant to solve the problem for macOs, that is, for a "normal" installation without docker. Could you try that?
Vzplau
commented
4 years ago Sorry Paula - I've still been having issue with the 'usual' installation. The above document (2020:6:20 FastqPuri Sreport Issue.docx) documents my steps with the 'usual' installation as well.
I'll try re-running the docker image again, and let you know how that goes.
Best,
Victor
PaulaAlessio
commented
4 years ago I think you have to run Qreport in the first place. There's no summary report without Qreport... Could you try that? Should I send you the exact steps?
Vzplau
commented
4 years ago Hi Paula,
Thanks for helping ou, as usual.. I've ran Qreport prior to Sreport, but it didn't seem to work as per the documentation involved below:
2020.07.07 Sreport issues, even with Docker image build..docx
However, I tried deleting all the other Qreports and Sreports in the folder. This successfully created the Sreport, as shown below:
2020.07.07 Resolved Sreport when other Qreports and Sreports are deleted..docx
It works now! I think the code is all good, but let me know if I can clarify anything. Otherwise, I really appreciate the help - the offer for the coffee gift card still stands in the future, when you're able to go outside more :)
If it's all good, I'll proceed to close the thread.
Best wishes,
Victor
PaulaAlessio
commented
4 years ago The output seems OK to me, are you satisfied with the html? Does it look like the one in the example? If so, it is OK then,
Cheers
Paula
Vzplau
commented
4 years ago Hi Paula,
It looks like the one in the test example. Yes, I think it's all resolved. I appreciate your help so much, once again!
Well wishes,
Victor
Hello,
Hope everything is well. Sorry for asking about this - I've been able to generate Qreports with FastqPuri, but after doing the Qreport call, the following is generated: (I've been using Terminal on my MacOSX)
victorlau@d207-023-174-227 FastqPuri-master % docker run -v $PWD:/tmp -it fastqpuri root@649b4fb51c0b:/tmp# cd example/Qreport_Sreport/run_test
How should I proceed here, in order to generate the html report or fulfill the dependencies? I've thus far installed separate dependencies (in cmake, R, pandoc, and the three R packages).
Everything should be installed, as provided in the word doc documenting all of the downloading + dependencies installed with cmake. Please let me know what I can do to fix this problem.
Working through cmake, Fastqpuri.docx ^(This document shows the process of cmake working through the following code. I then tried redoing above Qreport call, but to no success in generating a html report.
$ cmake -H. -Bbuild/ [-DRSCRIPT=/path/to/my/R/bin/Rscript] [-DCMAKE_INSTALL_PREFIX=/path/to/my/root] ... $ cd build $ make $ sudo make install)
I do also apologize for my poorer understanding of coding; I've been working on self-learning the material. Thank you for the response in advance, and hope to hear soon~
Well wishes,
Victor