Open ajrgodfrey opened 2 years ago
I just made a change that reads HKCU before HKLM when attempting to detect R (this is the same order that RStudio uses when detecting R): https://github.com/quarto-dev/quarto-cli/commit/966fd980313795e7f995b0b1098657821e417bd5
This change is available in v0.9.624: https://github.com/quarto-dev/quarto-cli/releases/tag/v0.9.624
@ajrgodfrey It would be helpful if we could see the entire output of quarto check
(including the registry error)
@cderv Have you ever seen anything like this?
Output from quarto check install and quarto check knitr:
@cderv Have you ever seen anything like this?
@jjallaire I believe this could be linked to Windows User Access Control (UAC) mechanism. Reading registry key can require some specific privilege. Restriction seems to be per key or Usually on some computer (like mine), using regedit
program will asked for elevated privileges. It seems there is specific behavior on application (e.g this https://docs.microsoft.com/en-us/cpp/security/how-user-account-control-uac-affects-your-application), and regedit
must be aware and ask.
I don't remember having encounter such issue in my test, so it may not be something that applies everywhere. Like this works fine for me in Terminal with Deno without elevated privileged:
❯ Deno
Deno 1.23.1
exit using ctrl+d or close()
> const cmd = ['reg', 'query', 'HKLM\\Software\\R-core\\R', '/v', 'Current Version']
undefined
> const p = Deno.run({cmd})
undefined
>
HKEY_LOCAL_MACHINE\Software\R-core\R
Current Version REG_SZ 4.2.0
So I don't have trouble reading. Same with R.
> utils::readRegistry("Software\\R-core\\R", "HLM")
$`Current Version`
[1] "4.2.0"
$InstallPath
[1] "C:\\Program Files\\R\\R-4.2.0"
$`4.1.2`
[1] "<subkey>"
$`4.2.0`
[1] "<subkey>"
So no change with 0.9.624
It seems the fix you've made did not change anything, so it is probably something related to access control, or Just R is not in the registry.
@ajrgodfrey
utils::readRegistry("Software\\R-core\\R", "HLM")
? REG QUERY HKLM\Software\R-core\R
? Get-ItemProperty -Path "HKLM:\Software\R-core\R"
This is to see if the reading of registry is blocked also for other tools ?
If you have a UAC windows popping asking to give higher privileges, this will be interested to us.
Also if you are willing to gets your hand into this, it would be interesting to really check the permissions of this keys we are trying to read. Step would be:
regedit
and click Enter. Either you can't at all if this is blocked on your computer by whoever is the admin, or a UAC window will open asking you to give higher privileges. \HKEY_LOCAL_MACHINE\SOFTWARE\R-core\R
to see if you can read this key. (You can also open folder in the tree to get to this value.Thanks for your help.
BTW @jjallaire, this will probably also affect chrome
research that uses also a registry mechanism (a fix should be apply there too as it is not using the same function)
https://github.com/quarto-dev/quarto-cli/blob/966fd980313795e7f995b0b1098657821e417bd5/src/core/puppeteer.ts#L211-L239
@cderv provided tests/questions.
Questions:
Are you using a windows environment with any type of restriction, like controlled by your company or else ? A: It is a university controlled machine but I generally have admin privileges; freedom to install/remove software at admin level.
Is UAC present?
A: yes, I must elevate to admin to view regedit. I can traverse the folder tree and see the full path, and the Current version
and InstallPath
variables.
Test results:
In R: utils::readRegistry("Software\R-core\R", "HLM") Result: total success
In windows cmd prompt: REG QUERY HKLM\Software\R-core\R Result: total success
Where I count finding the path to the R installation to be success. My results are the same as yours, but only one version of R is present.
BTW: I have used regdmp successfully before now in an unrelated R workflow. I create shortcuts to a batch file that looked across installed versions to find the latest one on any given system so that I could make sure all teaching resources remained unproblematic as R versions came and went. That tool worked with R v4.1.3 but is now broken for v4.2.0 in that it cannot see it. I have removed all versions of R < 4.2.0 though but would reinstall an older one if that will help testing.
Jonathan
Thanks a lot for the test. This is helpful. It seems that you can read the registry key, except that Quarto does not.
To confirm this, can we try reading the registry within Deno runtime ?
Go into quarto
install directory where the binaries are to find a deno executable
bin/tools
"$((Get-Command quarto).Source)/../tools"
but it depends if you have quarto in PATH. <quarto_install_path>/bin/tools
deno
const cmd = ['reg', 'query', 'HKLM\\Software\\R-core\\R', '/v', 'Current Version']
const p = Deno.run({cmd})
Use CTRL+D
to quit and close
This is what I get
❯ Deno
Deno 1.23.1
exit using ctrl+d or close()
> const cmd = ['reg', 'query', 'HKLM\\Software\\R-core\\R', '/v', 'Current Version']
undefined
> const p = Deno.run({cmd})
undefined
>
HKEY_LOCAL_MACHINE\Software\R-core\R
Current Version REG_SZ 4.2.0
Aim is to see if Deno have issue to read the registry, or if this is something really inside Quarto code base and our usage.
Thanks for your help.
@cderv One interesting thing is that our Deno code as currently structured shouldn't be throwing an error when it can't read a key (it should just return undefined and move on). It makes me think there is some sort of external security/supervisory software running that is observing the read and terminating the program.
Note that another way to test out various scenarios in an isolated fashion here is for us to write a script that is executed by our quarto run
command: https://quarto.org/docs/projects/quarto-projects.html#project-scripts
Thanks! I forgot about this way to run TS script.
@ajrgodfrey easier way to see what Deno reading registry tells us
Put this into a test.ts
file
const cmd = ['reg', 'query', 'HKLM\\Software\\R-core\\R', '/v', 'Current Version']
const p = Deno.run({cmd, stdout: "piped", stderr: "piped"})
const { code } = await p.status();
// Reading the outputs closes their pipes
const rawOutput = await p.output();
const rawError = await p.stderrOutput();
if (code === 0) {
await Deno.stdout.write(rawOutput);
} else {
const errorString = new TextDecoder().decode(rawError);
console.log(errorString);
}
Deno.exit(code);
and then call
quarto run test.ts
This would help us understand when running reg.exe
in Deno returns.
Instructions complete and helpful 😊
The test.ts script (as provided) returns the current version number of R as required. I ran this in the cmd prompt.
The outcome has four lines between prompts. First line has plenty of white space; second line has the hkey, followed by some white space; third line has white space, “Current Version”, white space, “REG_SZ”, white space, “4.2.0”, and more white space. Fourth line is lots of white space.
Notes:
Naïve question: where/how is the outcome from the test.ts script supposed to be captured?
My attempt to help:
Looks like I need to try the next part of the job to see why the installation path isn’t picked up. Looking at my registry using regedit, I see two places. InstallPath is in parallel to the Current Version, and a subfolder for v4.2.0 has the same information under it.
If I replace the first line of your script with const cmd = ['reg', 'query', 'HKLM\\Software\\R-core\\R', '/v', 'InstallPath']
I get
The same four lines and two more. Current version is replaced by InstallPath and delivers correct outcome, but I then get a line starting with “Check file:” and the path to my script file.
If I change the first line of the script to const cmd = ['reg', 'query', 'HKLM\\Software\\R-core\\R\\4.2.0', '/v', 'InstallPath']
I get the exact same outcome.
HTH Jonathan
The test.ts script (as provided) returns the current version number of R as required. I ran this in the cmd prompt.
The outcome has four lines between prompts. First line has plenty of white space; second line has the hkey, followed by some white space; third line has white space, “Current Version”, white space, “REG_SZ”, white space, “4.2.0”, and more white space. Fourth line is lots of white space.
This means that registry can be read from Deno run within Quarto. This is the same command as within Quarto so I don't know what is going one here.
using regedit, I see two places. InstallPath is in parallel to the Current Version,
The workflow is to read "Current Version" first, then query "InstallPath" to get the actual path.
I still don't manage to reproduce this behavior. I'll see if I can come up with another test to run to get more information.
Looking at my registry using regedit, I see two places.
In regedit
, at the keys, can you check if there are specific permissions ?
Should be possible to see by using right-click the directory or keys, and then click Permissions.... You should be able to see if somehow this looks like default or have specific restricition.
Also, how did you install R by the way ?
My other theory is that there is some anti-malware or anti-virus software on the system that sees quarto attempting to read from HKLM and terminates it (this would be consistent w/ the fact that the code we have that reads the registry isn't supposed to throw or even print an error). Is that a possibility?
I proposed a change in https://github.com/quarto-dev/quarto-cli/pull/1234, this won't solved but I believed your change in https://github.com/quarto-dev/quarto-cli/commit/966fd980313795e7f995b0b1098657821e417bd5 made this issue surfaced.
Regarding the anti malware, or anti virus, I don't know for sure. I understand that @ajrgodfrey environment is controlled but quite flexible. I'll have a look at Windows defender to see if it can block quarto binary.
@ajrgodfrey as another test regarding priviledges, you could try running quarto check knitr
in an elevated terminal (usually right click on the program like your terminal, or CMD and choose exectute as administrator. As you said you have admin access, we could see if this solve the "access denied"
Maybe looking at permissions in registry would help see a non default setting.
Regarding quarto
being being block, maybe Security Event Logging on windows can help see that (https://docs.microsoft.com/en-us/windows/security/threat-protection/auditing/view-the-security-event-log) but this is hard to find one specific to quarto. Maybe run quarto check knitr
and then look at the log ?
Do you have a malware software on your computer by the way ?
Last thing @ajrgodfrey, I wrote this script to isolate what Quarto is doing. If this throws the same issue this could help us debug further. But the questions above regarding potential anti-malware software are the next trail to follow.
Note: update to latest quarto in your next tries BTW
quarto run full-search.ts
Elevated privileges on cmd: test results are unchanged
I have just re-installed R as administrator (paranoia) and all tests undertaken thus far are returning the exact same outcomes.
I would normally install R using all defaults, and not normally as administrator. This is so that my experience matches those of my students.
Checking permissions via regedit: full control for administrators and read access for all users. There is a "deny access" check box that is unticked. I assume don't deny = allow.
Malware protection: none in play aside from Windows Defender.
Theory: we have not tested the full workflow outside quarto to find the path; we know that the first step works to find the current version number. We do not know which step of the quarto process is giving me the Access Denied error.
Full search script: I put everything from import to the final } in a script file but I get an error when I ran it. It is of the bad syntax kind, not anything to do with what we're trying to resolve. I am not at all familiar with the ts language so I can't sort that out. It says my file is missing a ; } or
Here is the script in a zip file, which is working for me using quarto run
and latest quarto
full-search.zip
We do not know which step of the quarto process is giving me the Access Denied error.
The idea is to find where in the code the error happens. However, from your error
.REGDMP: Unable to open key 'HKEY_LOCAL_MACHINE\SOFTWARE\R-core\R' (2)
The key searched is the one for the R version number. So it seems for some reason it can't be read.
Malware protection: none in play aside from Windows Defender.
I'll see if Windows Defender has a mechanism to block some program.
Script ran successfully.
Check in HKCU was false (as expected)
Check in HKLM was true (as should be) and returned 4.2.0
Q: Given the script found the version number, surely it could find the InstallPath that is right beside it? From what I'm reading, the quarto process is to use that current version number to find the path for that version. That seems to use two steps when only one is necessary, or am I missing something?
Btw: I keep checking for changes to quarto, so now I have tried everything using 0.9.629 and I still have the same test results.
Q: Given the script found the version number, surely it could find the InstallPath that is right beside it? From what I'm reading, the quarto process is to use that current version number to find the path for that version. That seems to use two steps when only one is necessary, or am I missing something?
Here is the full script equivalent to what we are running inside Quarto full-search.zip
I added the path search for the version found.
If this is working, it really seems to be a permission issue, where somehow quarto
binary is blocked on your system regarding registry.
Results are the same. HKCU is false and HKLM is true, returning the correct R version number.
Results are the same. HKCU is false and HKLM is true, returning the correct R version number.
So the correct R path is found ? You should have some logging in the console with R version, R path used and R binary used.
This is all correct on your side ? You can share stdout here.
Yes. It shows me the full path to Rscript (without the .exe)
I checked the quarto check knitr again and it still throws the access denied error.
Unfortunately, I really don't know what is going one here.
Yes. It shows me the full path to Rscript (without the .exe)
If the script I sent you is working, it means the TS code executed with the deno runtime inside Quarto can access the registry keys.
Checking permissions via regedit: full control for administrators and read access for all users. There is a "deny access" check box that is unticked. I assume don't deny = allow.
Malware protection: none in play aside from Windows Defender.
It does not seem you have some specific restrictions - I can't find any documentation regarding Windows Defender being able to block registry reading for one app. You can try looked into the setting of Windows defender to see if you find something.
Apart from that, I see only the Users Policies that could be in place on your environment. If you have IT support, maybe you can ask them ? Maybe there is an anti malware on your system provided by your controlled environment and that you are not aware of ?
I am running out of ideas from what could be wrong on our side. It seems to be external from quarto and something is preventing it to run.
I checked the quarto check knitr again and it still throws the access denied error.
Can you share again the whole results of quarto check knitr
?
If the R version can't be found in registry, the last resort is to look for R in ProgramFiles where it should be installed. But it seems it does not even go to this step, and quarto exit with error. Can you confirm that ?
FWIW, workaround to make Quarto works for you is to provide R version ahead of registry look up. This means 👍
QUARTO_R
environment variable, R_HOME
(which should be used if you are using quarto within RStudio IDE)On windows, there is 2 more steps:
Can you share again the quarto check knitr
console output ?
And perhaps I have found the problem.
I believe when quarto was calling regdmp, that it used a version I had in another folder on the search path.
I removed the regdmp utility from my personal folder that is on the system path, rebooted, and hey presto, quarto can find R!
I don’t know how to check the version of regdmp I had but if it was doing something different to the quarto version, then maybe it is the fault.
I did get asked if I was running a 32 bit version of regdmp, but I can’t tell that either, except by age of the file.
I suggest making quarto use a specific installation of regdmp within quarto via a fully specified path to it, or removing the need for it at all.
Thanks for the help. I think this issue can now be closed.
Jonathan
@cderv I noticed that we have two functions which call the registry -- one calls "reg" and one calls "reg.exe". Nowhere do I see "regdmp" called. Is it possible that there is a difference between "reg" and "reg.exe" and one of them calls "regdump" ?
I removed the regdmp utility
@ajrgodfrey you mean regdmp.exe
?
Is it possible that there is a difference between "reg" and "reg.exe" and one of them calls "regdump"
Until last answer from @ajrgodfrey, I assume REGDMP in the error message was just the way reg.exe
was erroring when not working. Now that I understand this is another executable, it is possible that reg
without extension could be partially matched to regdump.exe
instead of reg.exe
by Deno.run
?
It seems a bit odd - regdmp.exe
does not have a query
command. I would assume the error would be different as we are calling reg query
. I can't test though as I don't have regdmp.exe
on my Windows.
Maybe reg query
is calling regdmp.exe
in the process when found and this is causing the issue.
But all our test above with quarto run
and a .ts
file containing Deno.run
on "reg"
(not reg.exe
) and are working fine. Only when quarto render
it was causing issues.
@ajrgodfrey is this easy to test again to reproduce the issue if we do change "reg"
to full filename "reg.exe"
?
I noticed that we have two functions which call the registry
Yes we have some possible refactoring that you wanted to postpone to v1.1 when I mentioned it (https://github.com/quarto-dev/quarto-cli/pull/1051#issuecomment-1145831117). I assume calling reg
and reg.exe
was the same, it would not make a difference. but maybe it is.
Sure, I'd like to help where I can. I will try, but you will need to suggest tests to run.
FWIW: I moved regdmp.exe to a different location so I could move it back.
Q: Once quarto has found an R installation, does it store its settings anywhere or does it search again each time R is required? If the settings are stored, how often will it look to update the settings?
BTW: I have now moved to quarto 0.9.644 and I keep updating every day or two. I haven't kept all the different installers though.
Jonathan
Hello again.
I discovered that regdmp wasn’t working in other circumstances and found a way to avoid having to use it. Perhaps my solution could be looked at for providing a simpler way to find an R installation in quarto as well.
My batch file that opens the current version of R using a terminal is now as follows:
I've got R v4.2.0 and Quarto downloaded this week. When I run quarto check in the Windows command prompt, I get an error that regdmp cannot fetch the hkey_local_machine... relevant to the R installation. I can open regedit and see that the desired path regdmp is looking for is there, and that it contains the right install path for R, but the "access denied" message is stopping me in my tracks. What shoudl I try next?