search

bodkan / admixr

An R package for reproducible and automated ADMIXTOOLS analyses
https://bodkan.net/admixr
Other
28 stars 9 forks source link

Fix a (potential) issue with log paths on Windows? #57

Open bodkan opened 4 years ago

bodkan commented 4 years ago

See this post: https://github.com/DReichLab/AdmixTools/issues/63#issuecomment-634424676

bodkan commented 4 years ago

This is really strange. I asked a colleague with a Windows machine to do a couple of quick checks for me and we don't seem to be able to reproduce the issue. What I thought might be a problem (fixed unix-like path separator in admixr forced on Windows) does not actually seem to be an issue. What's more, I have heard from other people who actually do use Windows with admixr?

Of course, hard to tell without having more information. I will leave this open for now.

ningyusino commented 4 years ago

Hi, @bodkan , glad to hear some more about this issue. Great thanks for your time and effort. Here is some information I think might help:

The related scripts are:

library(admixr)
library(tidyverse)

#load data by path 
prefix <- c("C:/writedata/bol2017/admixtools/bol2017")
snps_bol17 <- eigenstrat(prefix)

#chose some focal pops
pops <- c ("pop1","pop2","pop3")

# run a test
trialresult <- d(snps_bol17,W=pops,X="pop10",Y="pop6",Z="pop4")

All went well until the "run a test" step, the ERROR message:

#Error in file(con, "r") : cannot open connection
#additionally: Warning message:
#In file(con, "r") :
#cannot open file : 'C:\Users\NY_CAF\AppData\Local\Temp\Rtmpi6ginF/qpDstat__2090046701.log':No such file or directory

I went to this path and only two files were found. The .log file was NOT found. 1

I think "admixr" is a new and very cool R package. It will definitely become popular in population researchers, especially for those with little experiences for Unix environment, me included 😄. Hope to hear more and thanks again.

bodkan commented 4 years ago

Hi,

Thanks for reporting back.

Can you explain to me what you mean by

I use Ubuntu 18.04.1 as a subsystem

Are you running some dual-boot setup or are you running Ubuntu in some virtual machine/emulator within Windows? What is a "subsystem" in this context?

It's hard for me to understand what's going on without having access to this particular setup of a combination of Windows & Ubuntu & conda (none of the three I use).

I worry the only efficient solution might be to clone the admixr repository with this exact OS/conda setup, add a breakpoint (in RStudio) to the d() function and step through every line of code before one hits a suspicious piece of code.

Also, what happens when you run Sys.getenv("PATH") in RStudio?

bodkan commented 3 years ago

Closing for insufficient information. Anyone should feel free to reopen in case similar problems appear again.

XiaXiaTianTian commented 2 years ago

Hi, @bodkan

I run this in Windows using Rstudio. Loading admixr package. My working directory is setwd("E:/tmp/04.final/admixtools") , and all needed files (.ind .snp .geno) are in this directory too. But I got the same error: `#Error in file(con, "r") : cannot open connection

additionally: Warning message:

In file(con, "r") :

cannot open file : 'C:\Users\THINKC~1\AppData\Local\Temp\Rtmp6fOtxD/qpDstat__1072709677.log': No such file or directory`

In C:\Users\THINKC~1\AppData\Local\Temp\Rtmp6fOtxD/, there are files as follows, files

I follow your suggestion, run Sys.getenv("PATH"), got like this, [1] "D:\\rstudio\\R-4.0.5\\bin\\x64;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\ProgramData\\Oracle\\Java\\javapath;C:\\Windows\\system32;C:\\Windows;C:\\Windows\\System32\\Wbem;C:\\Windows\\System32\\WindowsPowerShell\\v1.0\\;C:\\Windows\\System32\\OpenSSH\\;C:\\Program Files (x86)\\NVIDIA Corporation\\PhysX\\Common;D:\\Xshell\\;D:\\mega\\MEGA11;D:\\xmanager\\;D:\\winscp\\WinSCP\\;C:\\Users\\ThinkCentre\\AppData\\Local\\Microsoft\\WindowsApps;D:\\Bandzip\\"

Great thanks for any possible suggestions.

Best, Xia

bodkan commented 2 years ago

Hello Xia,

Thanks for adding more information to this issue. I have never progressed with Windows support, probably because ADMIXTOOLS itself is officially supported on macOS and Linux (not Windows, AFAIK) and I guess many users who are running Windows are doing analyses on computing clusters (which are almost exclusively Linux, I'd say). It's nice to have more information.

I have a couple of follow-up questions before we can proceed:

  1. Could you please paste an example script which fails? I.e. a minimal possible script that gives you error (i.e. starting a line with library(admixr) all the way to the command that fails (I suspect it's d(...) in your case). I need to see the entire script, but as small as possible so that the error is immediately apparent. Just the error itself is unfortunately not sufficient.

  2. What do you get when you run library(admixr) in your R console? Do you see any message from R?

  3. Related question: what do you get when you run Sys.which("qpDstat")?

  4. Another related question: how did you install ADMIXTOOLS? I don't think the Reich lab provides Windows binaries?

Cheers, Martin

XiaXiaTianTian commented 2 years ago

Dear Martin,

I found ADMIXTOOLS version 7.0.2 6/22/21 (for Linux and Mac). But admixr is R package like other packages that can run on Rstudio in Windows, right? Are ADMIXTOOLS and admixr independent?

I followed your questions to check again. It showed warning message when run library(admixr) and it surely went error when run d(...). Additionaly, I did not install ADMIXTOOLS in windows or Rstuidio in Windows.

Here is some information. `> library(admixr) ADMIXTOOLS programs required by admixr are not in your $PATH variable. At the very least, the following programs should be available: qp3Pop, qpDstat, qpF4ratio, qpAdm, qpWave Make sure to modify the $PATH variable in your .Renviron file so that it points to the directory containing the ADMIXTOOLS programs.

Sys.which("qpDstat") qpDstat "" .libPaths() [1] "D:/rstudio/R-4.0.5/library" installed.packages()[,c('Package','Version','LibPath')] Package Version LibPath
admixr "admixr" "0.9.1" "D:/rstudio/R-4.0.5/library" which qpDstat error: unexpected symbol in "which qpDstat" `

When I continue to run command, it shows like following. `> setwd("E:/tmp/04.final/admixtools/test")

prefix <- "./test" snps <- eigenstrat(prefix) list.files(path = dirname(prefix), pattern = basename(prefix), full.names = TRUE) [1] "./test.geno" "./test.ind" "./test.map" "./test.ped" "./test.R" "./test.snp" pops <- c("o1", "e", "m", "a1", "o1", "a2" , "o2" , "a") result <- d(W = pops , X = "a1" , Y = "m", Z = "o1", data = snps)

Error in file(con, "r") : cannot open connection

additionally: Warning message: #In file(con, "r") : #cannot open file : 'C:\Users\THINKC~1\AppData\Local\Temp\Rtmp6fOtxD/qpDstat__1072709677.log': No such file or directory

`

Best, Xia

bodkan commented 2 years ago

Hello again Xia,

I'm afraid I have some bad news. Also, thanks to the information you've provided, I finally understand the issue that the user above had back then.

I found ADMIXTOOLS version 7.0.2 6/22/21 (for Linux and Mac). But admixr is R package like other packages that can run on Rstudio in Windows, right? Are ADMIXTOOLS and admixr independent?

Yes, admixr and ADMIXTOOLS are two different pieces of software. Briefly, ADMIXTOOLS is a collection of command-line programs maintained by the Reich group. These are officially distributed for macOS and for Linux (AFAIK there's no official Windows support). The admixr R package that I'm developing serves as an R interface connecting to the underlying ADMIXTOOLS program.

Without ADMIXTOOLS present on the system, admixr can never work. This is why you see this message when you first load it:

> library(admixr)
ADMIXTOOLS programs required by admixr are not in your $PATH variable.
At the very least, the following programs should be available:
qp3Pop, qpDstat, qpF4ratio, qpAdm, qpWave
Make sure to modify the $PATH variable in your .Renviron file so
that it points to the directory containing the ADMIXTOOLS programs.

I know there is one way to run Linux subsystem on Windows called WSL. I suggest you give this a shot. Because this starts essentially a full-featured Linux environment under Windows, you should be able to install ADMIXTOOLS based on instructions from its authors. If you run into issues, I'm sure you can ask them for help.

Once you get ADMIXTOOLS itself running, please feel free to let me know here and I'd be more than happy to help you connect admixr from R to ADMIXTOOLS programs running on your system.

jahnringge commented 9 months ago

Hello bodkan! has Xia asked you more about connecting admixr and ADMIXTOOLS?

I am trying to run admixr through windows R. I have installed the Linux subsystem (runs ubuntu) WSL on windows and have used conda / bioconda to install ADMIXTOOLS. I confirm that admixtools is accessible through WSL.

I am running into issues connecting R to admixtools. The previous error that other users also encountered in this thread : cannot open file 'C:\Users\jahn\AppData\Local\Temp\RtmpolV1nj/qpDstat__458398317.log': No such file or directory

What I have tried already: Sys.setenv("PATH"="//wsl.localhost/Ubuntu/home/jahnringger/miniconda3/pkgs/admixtools-7.0.2-h2469040_2/bin/")

and variations of this. Additionally, when I search 'which qpDstat' on the R terminal, the output is 'no qpDstat in (/c/Users/jahn {list of environments}).'

Do you know of any way to access admixtools through R? Thanks in advance for the help

bodkan commented 9 months ago

Hi,

I'm afraid I don't remember the exact background to this issue. But I can reopen it to see if we can figure it out. To my knowledge, there aren't many ADMIXTOOLS users on Windows so the feedback (or possibility to debug issues) has been a bit tricky.

I don't have access to a Windows machine and don't really know WSL but the first thing to make sure that works (before we even bring admixr to the picture) is to make sure R itself can find the qpDstat et al. binaries.

For instance, on my machine, I get this:

> Sys.getenv("PATH")
[1] "[... stuff ...]:/Users/mp/.my_local/AdmixTools/bin: [... more stuff ...]"

And then I can verify this:

> Sys.which("qpDstat")
                                     qpDstat 
"/Users/mp/.my_local/AdmixTools/bin/qpDstat"

So if R can see qpDstat et al. in the $PATH, then admixr can also find it.

Can you test this? Before Sys.which("qpDstat") gives a positive result, admixr can't really do its job.

bodkan commented 9 months ago

I have found a (very suboptimal and slow) way to run Windows in a virtual machine on my Mac but setting up WSL, conda, ADMIXTOOLS from conda, etc. would take me a while as I don't know a thing about Windows.

I'm currently swamped with other bugs and issues in my other software, but if you don't find a way to do the Sys.which() thing above, I could poke around a bit later.

jahnringge commented 9 months ago

Hi bodkan, thank you for your fast reply! Running 

> Sys.which("qpDstat")
""

returns nothing, so I'm guessing that windows R has an issue with the $PATH.

I have since installed R on WSL and am running it through the linux terminal. This works well and I am able to work with admixr on my data.

However this means that I have to continuously move my data files from the windows environment (where I do most of my data analysis) to the linux environment. This is not very effective in the long run...

The problem is not urgent, but if you find the time to work on the Sys.which() issue I would appreciate the help, as my solution to the issue of admixr/ADMIXTOOLS in windows was.... not to use windows :)

bodkan commented 9 months ago

OK, yes. This:

> Sys.which("qpDstat")
""

means that it's a problem with finding the WSL ADMIXTOOLS installation in your Windows system. I think it goes even beyond R, to be honest.

My suggestion would be to google around the topic of "access path to a wsl binary on windows". Something like this might be useful? Again, I don't know Windows or WSL, so can't vouch for anything at this point.

There must be a way to access files installed via WSL from Windows. Like, even in the Windows command prompt. If you find how to refer to them, then you have your answer.

Also, perhaps I'm revealing a bit too much about my ignorance but I would normally write this:

Sys.setenv("PATH"="<path>")

as this:

Sys.setenv(PATH="<path>")

No idea if this makes any difference though.