Closed jwhitaker17 closed 4 years ago
Dear Justine, please read the function documentation carefully, twice.
inside R using assigner::assignment_ngs
You're using the vcf file the wrong way, try following the example by using it directly. Thierry
Hi Thierry,
Thanks for getting back to me so quickly. If I'm understanding correctly, you're saying that I need to use the tidy_genomic_data function to convert/import my data. Is that correct? I've attempted to use it, but get the same error. Also, I get the same error when I try to use the example datasets. So, while I think I understand what you were saying about my data file, there seems to be a bigger problem.
I appreciate your help.
Justine Whitaker, PhD Assistant Professor 906 East 1st Street 229 Gouaux Hall Nicholls State University Thibodaux, LA 70301 985-493-2628
On Mon, Apr 6, 2020 at 4:48 PM Thierry Gosselin notifications@github.com wrote:
Closed #20 https://github.com/thierrygosselin/assigner/issues/20.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/thierrygosselin/assigner/issues/20#event-3207356370, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMGULSUYKB52254QZEM4Q6TRLJE4HANCNFSM4MCQJARA .
Could you send me via email the required files to reproduce the error? For me the example works
I'm not able to reproduce the error you have with your dataset.
Let's go over a couple of steps before thinking about the parallel processing problems of some PCS... (they don't make it easy to work with genomic datasets):
Plan A
make sure that it's not an install problem: reinstall assigner and make sure that gsi_sim
is installed properly using: assigner::install_gsi_sim()
if this doesn't fix the problem, try installing gsi_sim
separately. Instructions here. And then run this script:
gsi_sim -b justine_whitake_gsi_sim_input.txt --self-assign > justine_whitake_gsi_sim_output.txt
if you didn't change the name of the program:
gsi_sim-MINGW32_NT-6.1 -b justine_whitake_gsi_sim_input.txt --self-assign > justine_whitake_gsi_sim_output.txt
The input file was sent by email, to reproduce this is what I did:
require(radiator)
data <- radiator::read_rad(data = "STLtd.rad") #the data you sent
radiator::write_gsi_sim(data = data, filename = "justine_whitake_gsi_sim_input.txt")
If this is working the problem is probably the parallel processing within assigner.
If this is not working it's a problem with gsi_sim
on your computer.
Plan B
Eric Anderson as also done a tidy version of gsi_sim
called rubias.
Plan C Use the Cloud...
Plan D Use the folder I've sent yesterday. I can run something else if required.
Good luck Thierry
Hi Thierry,
I reinstalled assigner (per the instructions) and gsi_sim.
assigner::install_gsi_sim()Downloading file https://github.com/eriqande/gsi_sim/blob/080f462c8eff035fa3e9f2fdce26c3ac013e208a/gsi_sim-MINGW32_NT-6.1And copying to C:/Users/justi/Documents/R/win-library/3.6/assigner/bin/gsi_simtrying URL 'https://github.com/eriqande/gsi_sim/blob/080f462c8eff035fa3e9f2fdce26c3ac013e208a/gsi_sim-MINGW32_NT-6.1'Content type 'text/html; charset=utf-8' length unknowndownloaded 63 KB NULL
I tried running the practice data and got the same error.
When I tried to install gsi_sim separately, I couldn't compile it. I got the following error.
MINGW64 ~/gsi_sim (master) $ ./Compile_gsi_sim.sh Compiling up executable gsi_sim-MINGW64_NT-10.0-18362
./Compile_gsi_sim.sh: line 23: gcc: command not found
Best,
Justine Whitaker, PhD Assistant Professor 906 East 1st Street 229 Gouaux Hall Nicholls State University Thibodaux, LA 70301 985-493-2628
On Thu, Apr 9, 2020 at 8:54 AM Thierry Gosselin notifications@github.com wrote:
I'm not able to reproduce the error you have with your dataset.
Let's go over a couple of steps before thinking about the parallel processing problems of some PCS... (they don't make it easy to work with genomic datasets):
Plan A
-
make sure that it's not an install problem: reinstall assigner http://thierrygosselin.github.io/assigner/#installation and make sure that gsi_sim is installed properly using: assigner::install_gsi_sim()
if this doesn't fix the problem, try installing gsi_sim separately. Instructions here https://github.com/eriqande/gsi_sim. And then run this script:
gsi_sim -b justine_whitake_gsi_sim_input.txt --self-assign > justine_whitake_gsi_sim_output.txt
if you didn't change the name of the program:
gsi_sim-MINGW32_NT-6.1 -b justine_whitake_gsi_sim_input.txt --self-assign > justine_whitake_gsi_sim_output.txt
The input file was sent by email, to reproduce this is what I did:
require(radiator) data <- radiator::read_rad(data = "STLtd.rad") #the data you sent radiator::write_gsi_sim(data = data, filename = "justine_whitake_gsi_sim_input.txt")
If this is working the problem is probably the parallel processing within assigner. If this is not working it's a problem with gsi_sim on your computer.
Plan B
Eric Anderson as also done a tidy version of gsi_sim called rubias https://github.com/eriqande/rubias.
Plan C Use the Cloud https://thierrygosselin.github.io/RADseq_cloud_tutorial/ ...
Plan D Use the folder I've sent yesterday. I can run something else if required.
Good luck Thierry
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/thierrygosselin/assigner/issues/20#issuecomment-611540975, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMGULSVMY54ODQEV6T56KVTRLXHTFANCNFSM4MCQJARA .
Ok so definitely something with your computer and sadly I don't offer PC troubleshooting for 2 reasons:
Go for plan B, C or D above Best, Thierry
Got it. Thanks for all your help!
Justine Whitaker, PhD Assistant Professor 906 East 1st Street 229 Gouaux Hall Nicholls State University Thibodaux, LA 70301 985-493-2628
On Thu, Apr 9, 2020 at 2:56 PM Thierry Gosselin notifications@github.com wrote:
Ok so definitely something with your computer and sadly I don't offer PC troubleshooting for 2 reasons:
- life is too short ;)
- I don't have a PC, so usually if I really need one, I start an Amazon instance.
Go for plan B, C or D above Best, Thierry
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/thierrygosselin/assigner/issues/20#issuecomment-611723414, or unsubscribe https://github.com/notifications/unsubscribe-auth/AMGULST2KTWFZ7PV4POMM6DRLYR6NANCNFSM4MCQJARA .
Hi Thierry,
I'm gettting the error message below when attempting to run assignment_ngs in RStudio. I read through the older issues and found one similar, but there was no resolution posted. Based on that issue though, I ran detect Cores() , which shows 8 cores, and then tried running the code with parallel.core = 1. When I run assignment_ngs with more cores, I get more nodes producing more errors (5 vs. 1). Any help would be greatly appreciated. I attached the populations folder, but it would not let me attach the vcf. Thanks in advance!
Best, Justine Whitaker
STL.vcf2=read_vcf('STLpopulations.snps.vcf', strata = 'StLaw_popmapFINAL.txt', parallel.core = 1)
test2 <- assigner::assignment_ngs(data = STL.vcf2, assignment.analysis = "gsi_sim", markers.sampling = "ranked", thl = 0.2, iteration.method = 5, marker.number = c(100, 200, 300, 400, "all"), subsample = 20, iteration.subsample = 3, parallel.core = 1)
assigner::assignment_ngs ############################ ################################################################################ Execution date/time: 20200406@1332 Assignment analysis with gsi_sim Folder created: assignment_analysis_method_ranked_20200406@1332 Computation time, overall: 8 sec Calibrating REF/ALT alleles... number of REF/ALT switch = 21 Subsampling: selected using subsample size of: 20 Analyzing subsample: 1 Conducting Assignment analysis using Training, Holdout, Leave-one-out Using training samples to rank markers based on Fst Holdout samples saved in your folder Starting parallel computations, for progress monitor activity in folder... Error in checkForRemoteErrors(val) : one node produced an error: 'assignment_data_iteration_1_markers_100.output.txt' does not exist in current working directory ('C:/Users/justi/Documents/Fisheries Research Revisions/assignPOP/assignment_analysis_method_ranked_20200406@1332/subsample_1/assignment_1'). In addition: Warning messages: 1: In serialize(data, node$con) : 'package:stats' may not be available when loading 2: In serialize(data, node$con) : 'package:stats' may not be available when loading 3: In serialize(data, node$con) : 'package:stats' may not be available when loading 4: In serialize(data, node$con) : 'package:stats' may not be available when loading 5: In serialize(data, node$con) : 'package:stats' may not be available when loading 6: In serialize(data, node$con) : 'package:stats' may not be available when loading Computation time, overall: 118 sec ########################## assignment_ngs completed ############################
parallel::detectCores() [1] 8
ade4 1.7-15 2020-02-13 [1] CRAN (R 3.6.3)
adegenet 2.1.2 2020-01-20 [1] CRAN (R 3.6.3)
ape 5.3 2019-03-17 [1] CRAN (R 3.6.3)
assertthat 0.2.1 2019-03-21 [1] CRAN (R 3.6.3)
assigner 0.5.7 2020-04-03 [1] Github (thierrygosselin/assigner@1888222) assignPOP 1.1.9 2020-03-16 [1] CRAN (R 3.6.3)
backports 1.1.5 2019-10-02 [1] CRAN (R 3.6.1)
Biobase 2.46.0 2019-10-29 [1] Bioconductor
BiocGenerics 0.32.0 2019-10-29 [1] Bioconductor
Biostrings 2.54.0 2019-10-29 [1] Bioconductor
bitops 1.0-6 2013-08-17 [1] CRAN (R 3.6.0)
boot 1.3-24 2019-12-20 [2] CRAN (R 3.6.3)
broom 0.5.5 2020-02-29 [1] CRAN (R 3.6.3)
calibrate 1.7.5 2019-10-01 [1] CRAN (R 3.6.3)
callr 3.4.3 2020-03-28 [1] CRAN (R 3.6.3)
caret 6.0-86 2020-03-20 [1] CRAN (R 3.6.3)
class 7.3-15 2019-01-01 [2] CRAN (R 3.6.3)
classInt 0.4-2 2019-10-17 [1] CRAN (R 3.6.3)
cli 2.0.2 2020-02-28 [1] CRAN (R 3.6.3)
cluster 2.1.0 2019-06-19 [2] CRAN (R 3.6.3)
coda 0.19-3 2019-07-05 [1] CRAN (R 3.6.3)
codetools 0.2-16 2018-12-24 [2] CRAN (R 3.6.3)
colorspace 1.4-1 2019-03-18 [1] CRAN (R 3.6.1)
combinat 0.0-8 2012-10-29 [1] CRAN (R 3.6.0)
crayon 1.3.4 2017-09-16 [1] CRAN (R 3.6.3)
crosstalk 1.1.0.1 2020-03-13 [1] CRAN (R 3.6.3)
dartR 1.1.11 2019-02-07 [1] CRAN (R 3.6.3)
data.table 1.12.8 2019-12-09 [1] CRAN (R 3.6.3)
DBI 1.1.0 2019-12-15 [1] CRAN (R 3.6.3)
deldir 0.1-25 2020-02-03 [1] CRAN (R 3.6.2)
desc 1.2.0 2018-05-01 [1] CRAN (R 3.6.3)
devtools 2.2.2 2020-02-17 [1] CRAN (R 3.6.3)
digest 0.6.25 2020-02-23 [1] CRAN (R 3.6.3)
directlabels 2020.1.31 2020-02-01 [1] CRAN (R 3.6.3)
dismo 1.1-4 2017-01-09 [1] CRAN (R 3.6.3)
doParallel 1.0.15 2019-08-02 [1] CRAN (R 3.6.3)
dplyr 0.8.5 2020-03-07 [1] CRAN (R 3.6.3)
e1071 1.7-3 2019-11-26 [1] CRAN (R 3.6.3)
ellipse 0.4.1 2018-01-05 [1] CRAN (R 3.6.3)
ellipsis 0.3.0 2019-09-20 [1] CRAN (R 3.6.3)
expm 0.999-4 2019-03-21 [1] CRAN (R 3.6.3)
fansi 0.4.1 2020-01-08 [1] CRAN (R 3.6.3)
farver 2.0.3 2020-01-16 [1] CRAN (R 3.6.3)
fastmap 1.0.1 2019-10-08 [1] CRAN (R 3.6.3)
foreach 1.5.0 2020-03-30 [1] CRAN (R 3.6.3)
fs 1.4.0 2020-03-31 [1] CRAN (R 3.6.3)
gap 1.2.2 2020-02-02 [1] CRAN (R 3.6.3)
gdata 2.18.0 2017-06-06 [1] CRAN (R 3.6.2)
gdistance 1.3-1 2020-02-29 [1] CRAN (R 3.6.3)
gdsfmt 1.22.0 2019-10-29 [1] Bioconductor
generics 0.0.2 2018-11-29 [1] CRAN (R 3.6.3)
genetics 1.3.8.1.2 2019-04-22 [1] CRAN (R 3.6.3)
GenomeInfoDb 1.22.1 2020-03-27 [1] Bioconductor
GenomeInfoDbData 1.2.2 2020-04-03 [1] Bioconductor
GenomicRanges 1.38.0 2019-10-29 [1] Bioconductor
GGally 1.5.0 2020-03-25 [1] CRAN (R 3.6.3)
ggplot2 3.3.0 2020-03-05 [1] CRAN (R 3.6.3)
glue 1.3.2 2020-03-12 [1] CRAN (R 3.6.3)
gmodels 2.18.1 2018-06-25 [1] CRAN (R 3.6.3)
gower 0.2.1 2019-05-14 [1] CRAN (R 3.6.1)
gridExtra 2.3 2017-09-09 [1] CRAN (R 3.6.3)
gtable 0.3.0 2019-03-25 [1] CRAN (R 3.6.3)
gtools 3.8.1 2018-06-26 [1] CRAN (R 3.6.0)
GWASExactHW 1.01 2013-01-05 [1] CRAN (R 3.6.0)
hierfstat 0.04-22 2015-12-04 [1] CRAN (R 3.6.3)
hms 0.5.3 2020-01-08 [1] CRAN (R 3.6.3)
htmltools 0.4.0 2019-10-04 [1] CRAN (R 3.6.3)
htmlwidgets 1.5.1 2019-10-08 [1] CRAN (R 3.6.3)
httpuv 1.5.2 2019-09-11 [1] CRAN (R 3.6.3)
igraph 1.2.5 2020-03-19 [1] CRAN (R 3.6.3)
ipred 0.9-9 2019-04-28 [1] CRAN (R 3.6.3)
IRanges 2.20.2 2020-01-13 [1] Bioconductor
iterators 1.0.12 2019-07-26 [1] CRAN (R 3.6.3)
jsonlite 1.6.1 2020-02-02 [1] CRAN (R 3.6.3)
KernSmooth 2.23-16 2019-10-15 [2] CRAN (R 3.6.3)
knitr 1.28 2020-02-06 [1] CRAN (R 3.6.3)
labeling 0.3 2014-08-23 [1] CRAN (R 3.6.0)
later 1.0.0 2019-10-04 [1] CRAN (R 3.6.3)
lattice 0.20-38 2018-11-04 [2] CRAN (R 3.6.3)
lava 1.6.7 2020-03-05 [1] CRAN (R 3.6.3)
leaflet 2.0.3 2019-11-16 [1] CRAN (R 3.6.3)
LearnBayes 2.15.1 2018-03-18 [1] CRAN (R 3.6.0)
lifecycle 0.2.0 2020-03-06 [1] CRAN (R 3.6.3)
logistf 1.23 2018-07-19 [1] CRAN (R 3.6.3)
lubridate 1.7.4 2018-04-11 [1] CRAN (R 3.6.3)
magrittr 1.5 2014-11-22 [1] CRAN (R 3.6.3)
manipulateWidget 0.10.1 2020-02-24 [1] CRAN (R 3.6.3)
MASS 7.3-51.5 2019-12-20 [2] CRAN (R 3.6.3)
Matrix 1.2-18 2019-11-27 [2] CRAN (R 3.6.3)
memoise 1.1.0 2017-04-21 [1] CRAN (R 3.6.3)
memuse 4.1-0 2020-02-17 [1] CRAN (R 3.6.2)
mgcv 1.8-31 2019-11-09 [2] CRAN (R 3.6.3)
mice 3.8.0 2020-02-21 [1] CRAN (R 3.6.3)
mime 0.9 2020-02-04 [1] CRAN (R 3.6.2)
miniUI 0.1.1.1 2018-05-18 [1] CRAN (R 3.6.3)
mmod 1.3.3 2017-04-06 [1] CRAN (R 3.6.3)
ModelMetrics 1.2.2.2 2020-03-17 [1] CRAN (R 3.6.3)
munsell 0.5.0 2018-06-12 [1] CRAN (R 3.6.3)
mvtnorm 1.1-0 2020-02-24 [1] CRAN (R 3.6.2)
nlme 3.1-144 2020-02-06 [2] CRAN (R 3.6.3)
nnet 7.3-12 2016-02-02 [2] CRAN (R 3.6.3)
pbmcapply 1.5.0 2019-07-10 [1] CRAN (R 3.6.1)
pca3d 0.10.1 2019-12-14 [1] CRAN (R 3.6.3)
pegas 0.13 2020-03-10 [1] CRAN (R 3.6.3)
permute 0.9-5 2019-03-12 [1] CRAN (R 3.6.3)
pillar 1.4.3 2019-12-20 [1] CRAN (R 3.6.3)
pinfsc50 1.1.0 2016-12-02 [1] CRAN (R 3.6.0)
pkgbuild 1.0.6 2019-10-09 [1] CRAN (R 3.6.3)
pkgconfig 2.0.3 2019-09-22 [1] CRAN (R 3.6.3)
pkgload 1.0.2 2018-10-29 [1] CRAN (R 3.6.3)
plyr 1.8.6 2020-03-03 [1] CRAN (R 3.6.3)
png 0.1-7 2013-12-03 [1] CRAN (R 3.6.0)
PopGenReport 3.0.4 2019-02-04 [1] CRAN (R 3.6.3)
prettyunits 1.1.1 2020-01-24 [1] CRAN (R 3.6.3)
pROC 1.16.2 2020-03-19 [1] CRAN (R 3.6.3)
processx 3.4.2 2020-02-09 [1] CRAN (R 3.6.3)
prodlim 2019.11.13 2019-11-17 [1] CRAN (R 3.6.3)
promises 1.1.0 2019-10-04 [1] CRAN (R 3.6.3)
ps 1.3.2 2020-02-13 [1] CRAN (R 3.6.3)
purrr 0.3.3 2019-10-18 [1] CRAN (R 3.6.3)
quadprog 1.5-8 2019-11-20 [1] CRAN (R 3.6.1)
qvalue 2.18.0 2019-10-29 [1] Bioconductor
R.methodsS3 1.8.0 2020-02-14 [1] CRAN (R 3.6.2)
R.oo 1.23.0 2019-11-03 [1] CRAN (R 3.6.2)
R.utils 2.9.2 2019-12-08 [1] CRAN (R 3.6.3)
R6 2.4.1 2019-11-12 [1] CRAN (R 3.6.3)
radiator 1.1.5 2020-04-03 [1] Github (thierrygosselin/radiator@a13fe0a) randomForest 4.6-14 2018-03-25 [1] CRAN (R 3.6.3)
raster 3.0-12 2020-01-30 [1] CRAN (R 3.6.3)
RColorBrewer 1.1-2 2014-12-07 [1] CRAN (R 3.6.0)
Rcpp 1.0.4 2020-03-17 [1] CRAN (R 3.6.3)
RCurl 1.98-1.1 2020-01-19 [1] CRAN (R 3.6.2)
readr 1.3.1 2018-12-21 [1] CRAN (R 3.6.3)
recipes 0.1.10 2020-03-18 [1] CRAN (R 3.6.3)
remotes 2.1.1 2020-02-15 [1] CRAN (R 3.6.3)
reshape 0.8.8 2018-10-23 [1] CRAN (R 3.6.3)
reshape2 1.4.3 2017-12-11 [1] CRAN (R 3.6.3)
rgdal 1.4-8 2019-11-27 [1] CRAN (R 3.6.3)
rgl 0.100.50 2020-02-21 [1] CRAN (R 3.6.3)
RgoogleMaps 1.4.5.3 2020-02-12 [1] CRAN (R 3.6.3)
rlang 0.4.5 2020-03-01 [1] CRAN (R 3.6.3)
rpart 4.1-15 2019-04-12 [2] CRAN (R 3.6.3)
rprojroot 1.3-2 2018-01-03 [1] CRAN (R 3.6.3)
rrBLUP 4.6.1 2019-12-18 [1] CRAN (R 3.6.2)
rstudioapi 0.11 2020-02-07 [1] CRAN (R 3.6.3)
S4Vectors 0.24.3 2020-01-18 [1] Bioconductor
scales 1.1.0 2019-11-18 [1] CRAN (R 3.6.3)
SeqArray 1.26.2 2019-12-26 [1] Bioconductor
seqinr 3.6-1 2019-09-07 [1] CRAN (R 3.6.3)
SeqVarTools 1.24.1 2020-03-17 [1] Bioconductor
sessioninfo 1.1.1 2018-11-05 [1] CRAN (R 3.6.3)
sf 0.9-0 2020-03-24 [1] CRAN (R 3.6.3)
shiny 1.4.0.2 2020-03-13 [1] CRAN (R 3.6.3)
SNPRelate 1.20.1 2019-11-22 [1] Bioconductor
sp 1.4-1 2020-02-28 [1] CRAN (R 3.6.3)
spData 0.3.3 2020-02-11 [1] CRAN (R 3.6.3)
spdep 1.1-3 2019-09-18 [1] CRAN (R 3.6.3)
StAMPP 1.6.1 2020-03-20 [1] CRAN (R 3.6.3)
stringi 1.4.6 2020-02-17 [1] CRAN (R 3.6.2)
stringr 1.4.0 2019-02-10 [1] CRAN (R 3.6.3)
survival 3.1-8 2019-12-03 [2] CRAN (R 3.6.3)
testthat 2.3.2 2020-03-02 [1] CRAN (R 3.6.3)
tibble 2.1.3 2019-06-06 [1] CRAN (R 3.6.3)
tidyr 1.0.2 2020-01-24 [1] CRAN (R 3.6.3)
tidyselect 1.0.0 2020-01-27 [1] CRAN (R 3.6.3)
timeDate 3043.102 2018-02-21 [1] CRAN (R 3.6.2)
tree 1.0-40 2019-03-01 [1] CRAN (R 3.6.3)
units 0.6-6 2020-03-16 [1] CRAN (R 3.6.3)
UpSetR 1.4.0 2019-05-22 [1] CRAN (R 3.6.3)
usethis 1.5.1 2019-07-04 [1] CRAN (R 3.6.3)
vcfR 1.10.0 2020-02-06 [1] CRAN (R 3.6.3)
vctrs 0.2.4 2020-03-10 [1] CRAN (R 3.6.3)
vegan 2.5-6 2019-09-01 [1] CRAN (R 3.6.3)
viridisLite 0.3.0 2018-02-01 [1] CRAN (R 3.6.3)
webshot 0.5.2 2019-11-22 [1] CRAN (R 3.6.3)
withr 2.1.2 2018-03-15 [1] CRAN (R 3.6.3)
xfun 0.12 2020-01-13 [1] CRAN (R 3.6.3)
xtable 1.8-4 2019-04-21 [1] CRAN (R 3.6.3)
XVector 0.26.0 2019-10-29 [1] Bioconductor
zlibbioc 1.32.0 2019-10-29 [1] Bioconductor
[1] C:/Users/justi/Documents/R/win-library/3.6 [2] C:/Program Files/R/R-3.6.3/library
[StLaw_popmapFINAL.txt]
(https://github.com/thierrygosselin/assigner/files/4440233/StLaw_popmapFINAL.txt)