Closed BrutalSimplicity closed 6 years ago
Interesting. That's a good intuition. If you hadn't already dug into the PATH issue that's what I would have suggested.
The code for running aws from this program is a barebones as it gets. It doesn't even use a subshell, it execs aws directly as a subprocess. It relies on the OS to have aws on the PATH. It should just work out of the box.
This works for us on Amazon WorkSpaces, which runs Windows. I'll check with the folks who manage that next week to see if they needed any special workarounds to make this work.
Have you rebooted or logged out/in since installing the AWS CLI with Python? That might help.
Just tried restarting this morning, and am still receiving the same error 😞
Below, I've included my system information to possibly aid in diagnosing this issue. Please let me know if you find any additional information.
[System Summary]
Item Value
OS Name Microsoft Windows 10 Pro
Version 10.0.16299 Build 16299
Other OS Description Not Available
OS Manufacturer Microsoft Corporation
System Name DESKTOP-BRU8NOT
System Manufacturer Apple Inc.
System Model MacBookPro11,4
System Type x64-based PC
System SKU Unsupported
Processor Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz, 2501 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date Apple Inc. MBP114.88Z.0172.B09.1602151732, 2/15/2016
SMBIOS Version 2.7
BIOS Mode UEFI
BaseBoard Manufacturer Apple Inc.
BaseBoard Model Not Available
BaseBoard Name Base Board
Platform Role Mobile
Secure Boot State Unsupported
PCR7 Configuration Binding Not Possible
Windows Directory C:\WINDOWS
System Directory C:\WINDOWS\system32
Boot Device \Device\HarddiskVolume1
Locale United States
Hardware Abstraction Layer Version = "10.0.16299.248"
User Name DESKTOP-BRU8NOT\Kory
Time Zone Central Standard Time
Installed Physical Memory (RAM) 16.0 GB
Total Physical Memory 15.9 GB
Available Physical Memory 8.44 GB
Total Virtual Memory 19.9 GB
Available Virtual Memory 11.9 GB
Page File Space 4.04 GB
Page File C:\pagefile.sys
Virtualization-based security Not enabled
Device Encryption Support Reasons for failed automatic device encryption: TPM is not usable, PCR7 binding is not supported, Hardware Security Test Interface failed and device is not InstantGo, Un-allowed DMA capable bus/device(s) detected, TPM is not usable
A hypervisor has been detected. Features required for Hyper-V will not be displayed.
[Environment Variables]
Variable Value User Name
AppsRoot D <SYSTEM>
ChocolateyInstall C:\ProgramData\chocolatey <SYSTEM>
ComSpec %SystemRoot%\system32\cmd.exe <SYSTEM>
CONFIGURATION Production <SYSTEM>
FSHARPINSTALLDIR C:\Program Files (x86)\Microsoft SDKs\F#\4.1\Framework\v4.0\ <SYSTEM>
GOROOT C:\Go\ <SYSTEM>
JAVA_HOME C:\Program Files\Java\jdk1.8.0_121 <SYSTEM>
LDMS_LOCAL_DIR C:\Program Files (x86)\LANDesk\LDClient\Data <SYSTEM>
LDMS_PREFERRED_SERVER ogmgmt1.toptenreviews.local <SYSTEM>
NUMBER_OF_PROCESSORS 8 <SYSTEM>
OS Windows_NT <SYSTEM>
Path C:\Program Files\Docker\Docker\Resources\bin;C:\ProgramData\Oracle\Java\javapath;C:\Program Files (x86)\Intel\iCLS Client\;C:\Program Files\Intel\iCLS Client\;%SystemRoot%\system32;%SystemRoot%;%SystemRoot%\System32\Wbem;%SYSTEMROOT%\System32\WindowsPowerShell\v1.0\;C:\Program Files\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\DAL;C:\Program Files\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files (x86)\Intel\Intel(R) Management Engine Components\IPT;C:\Program Files\Git\cmd;%USERPROFILE%\.dnx\bin;C:\Program Files\Microsoft DNX\Dnvm\;C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\ManagementStudio\;C:\Tools\bin;C:\Program Files\Microsoft\Web Platform Installer\;c:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn\;c:\Program Files\Microsoft SQL Server\100\Tools\Binn\;c:\Program Files\Microsoft SQL Server\100\DTS\Binn\;C:\ProgramData\chocolatey\bin;C:\Program Files\dotnet\;C:\Program Files\PuTTY\;C:\Go\bin;C:\Program Files (x86)\sbt\bin;C:\Program Files\erl9.0\bin;C:\Program Files (x86)\Elixir\bin;%USERPROFILE%\.mix\escripts;C:\Program Files\nodejs\;C:\Program Files\Python36;C:\Program Files\Python36\Scripts; <SYSTEM>
PATHEXT .COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH;.MSC <SYSTEM>
PGDATA C:\Program Files\PostgreSQL\9.6\data <SYSTEM>
PROCESSOR_ARCHITECTURE AMD64 <SYSTEM>
PROCESSOR_IDENTIFIER Intel64 Family 6 Model 70 Stepping 1, GenuineIntel <SYSTEM>
PROCESSOR_LEVEL 6 <SYSTEM>
PROCESSOR_REVISION 4601 <SYSTEM>
PSModulePath %ProgramFiles%\WindowsPowerShell\Modules;%SystemRoot%\system32\WindowsPowerShell\v1.0\Modules;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\PowerShell\Modules\ <SYSTEM>
SBT_HOME C:\Program Files (x86)\sbt\ <SYSTEM>
TEMP %SystemRoot%\TEMP <SYSTEM>
TMP %SystemRoot%\TEMP <SYSTEM>
USERNAME SYSTEM <SYSTEM>
VBOX_MSI_INSTALL_PATH C:\Program Files\Oracle\VirtualBox\ <SYSTEM>
VS140COMNTOOLS C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\Tools\ <SYSTEM>
windir %SystemRoot% <SYSTEM>
Path %USERPROFILE%\AppData\Local\Microsoft\WindowsApps; NT AUTHORITY\SYSTEM
TEMP %USERPROFILE%\AppData\Local\Temp NT AUTHORITY\SYSTEM
TMP %USERPROFILE%\AppData\Local\Temp NT AUTHORITY\SYSTEM
ChocolateyLastPathUpdate Tue Jan 23 23:17:38 2018 DESKTOP-BRU8NOT\Kory
GOPATH C:\Users\Kory\Dropbox\go DESKTOP-BRU8NOT\Kory
JAVA_HOME C:\Program Files\Java\jdk1.8.0_121 DESKTOP-BRU8NOT\Kory
OneDrive C:\Users\Kory\OneDrive DESKTOP-BRU8NOT\Kory
Path C:\Users\Kory\.cargo\bin;C:\Ruby23-x64\bin;C:\Users\Kory\AppData\Local\Microsoft\WindowsApps;C:\Program Files (x86)\Microsoft VS Code\bin;C:\Program Files (x86)\GnuWin32\bin;C:\Users\Kory\AppData\Local\Google\Cloud SDK\google-cloud-sdk\bin;C:\Program Files\Python36;C:\Program Files\Python36\Scripts;C:\Program Files (x86)\Nmap;C:\Users\Kory\AppData\Local\Programs\Fiddler;C:\Users\Kory\AppData\Roaming\npm;%USERPROFILE%\AppData\Local\Microsoft\WindowsApps;C:\Users\Kory\AppData\Local\.meteor; DESKTOP-BRU8NOT\Kory
PATHEXT %PATHEXT%;.RB;.RBW DESKTOP-BRU8NOT\Kory
PSModulePath ;C:\Users\Kory\AppData\Local\Google\Cloud SDK\google-cloud-sdk\platform\PowerShell DESKTOP-BRU8NOT\Kory
TEMP %USERPROFILE%\AppData\Local\Temp DESKTOP-BRU8NOT\Kory
TMP %USERPROFILE%\AppData\Local\Temp DESKTOP-BRU8NOT\Kory
This works on Windows for all my users, so I’m not sure why it won’t work for you. It’s likely specific configuration on your machine causing the issue. I can’t debug that or practically support it, but I’ll do my best to give you the tools to fix this.
On Windows, PATH is one of multiple ways in which programs get found by name. They can also be found in the Registry.
While it should work anyway, I’m not certain that ProcessBuilder (what I’m using to invoke the aws subprocess) will refer to anything other than PATH.
CAVEAT: if the aws command doesn’t work from Command Prompt or PowerShell, then it is unrelated to this project and you’ll need to fix that independently.
If the aws command works from your Command Prompt or PowerShell, then you can use withokta and named profiles to work around the PATH issue.
Here’s the PowerShell version:
$env:OKTA_PROFILE = "myprofile"
./withokta cmd /c "aws —profile myprofile s3 ls"
This works by getting withokta to run cmd /c which should cause it to run the subcommand within a full Command Prompt subshell.
Here’s another possible workaround using withokta to establish a named profile and then separately running commands that specifically use that named profile:
$env:OKTA_PROFILE = "myprofile"
./withokta rundll32
aws —profile myprofile s3 ls
I understand that rundll32 is a Windows program that with no arguments does nothing. It’s a stand in for true, a command that does nothing and is common on UNIX-like OSes.
I haven’t tested these, but they should work in theory. They should give you some ideas of things to try here.
:-1: me too.
c:\Users\dlgoodr\.okta>java -classpath .\okta-aws-cli-1.0.1.jar com.okta.tools.awscli
Username: dlgoodr
Password:
Please choose the role you would like to assume:
[excised]
Account: [x]
[excised]
[ 4 ]: TEST
[excised]
Selection: 4
Exception in thread "main" java.io.IOException: Cannot run program "aws": CreateProcess error=2, The system cannot find the file specified
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at com.okta.tools.awscli.main(awscli.java:39)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.base/java.lang.ProcessImpl.create(Native Method)
at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:420)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:151)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
... 2 more
c:\Users\dlgoodr\.okta>aws
usage: aws [options] <command> <subcommand> [<subcommand> ...] [parameters]
To see help text, you can run:
aws help
aws <command> help
aws <command> <subcommand> help
aws: error: too few arguments
c:\Users\dlgoodr\.okta>which aws
/c/Python27/Scripts/aws
c:\Users\dlgoodr\.okta>
The withokta
workarounds do not work for me if I have to switch roles into another account. AccessDenied when calling AssumeRole.
@dlgoodr how does role-switching not work using withokta? You switch roles within your flow the same way you pick one initially in my example above. This tool doesn’t grant every profile the ability to AssumeRole between profiles. You need to invoke it explitly when switching roles.
Also, why are you invoking this with java explicitly and not using the provided bat files?
Make sure you pull the 1.0.2 release. It fixes a bunch of bugs that have been around for a while.
Switching Role with withokta
It appears to be rewriting my profile, changing the account number in the role_arn
and the source_profile
value.
PS C:\Users\dlgoodr\.okta> $env:OKTA_PROFILE = "target-account-profile"
PS C:\Users\dlgoodr\.okta> .\withokta.bat cmd /c "aws --profile target-account-profile iam list-roles"
C:\Users\dlgoodr\.okta>java -classpath "C:\Users\dlgoodr\.okta\*" com.okta.tools.WithOkta cmd /c "aws --profile target-account-profile iam list-roles"
An error occurred (AccessDenied) when calling the AssumeRole operation: User: arn:aws:sts::<landing_account>:assumed-role/myrole/dlgoodr@ad.corp.com is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::<landing_account>:role/myrole
PS C:\Users\dlgoodr\.okta>
my .aws\config
is:
[profile default]
output = json
region = us-east-1
[profile target-account-profile]
region = us-east-1
role_arn = arn:aws:iam::<landing_account>:role/myrole
source_profile = target-account-profile_source
before I ran withokta, source_profile
was something entirely different so i'm very confused about how this should work. Never prompted for creds, either, so ... it's either very broken or I have no idea how to use it. I'm ok with it being the second, but guidance would be helpful.
Using the .bat file instead of calling java explicitly doesn't change the behavior.
C:\Users\dlgoodr\.okta>awscli.bat
C:\Users\dlgoodr\.okta>rem
C:\Users\dlgoodr\.okta>rem Copyright 2017 Okta
C:\Users\dlgoodr\.okta>rem
C:\Users\dlgoodr\.okta>rem Licensed under the Apache License, Version 2.0 (the "License");
C:\Users\dlgoodr\.okta>rem you may not use this file except in compliance with the License.
C:\Users\dlgoodr\.okta>rem You may obtain a copy of the License at
C:\Users\dlgoodr\.okta>rem
C:\Users\dlgoodr\.okta>rem http://www.apache.org/licenses/LICENSE-2.0
C:\Users\dlgoodr\.okta>rem
C:\Users\dlgoodr\.okta>rem Unless required by applicable law or agreed to in writing, software
C:\Users\dlgoodr\.okta>rem distributed under the License is distributed on an "AS IS" BASIS,
C:\Users\dlgoodr\.okta>rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
C:\Users\dlgoodr\.okta>rem See the License for the specific language governing permissions and
C:\Users\dlgoodr\.okta>rem limitations under the License.
C:\Users\dlgoodr\.okta>rem
C:\Users\dlgoodr\.okta>java -classpath "C:\Users\dlgoodr\.okta\*" com.okta.tools.awscli
Username: dlgoodr
Password:
Exception in thread "main" java.io.IOException: Cannot run program "aws": CreateProcess error=2, The system cannot find the file specified
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1128)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1071)
at com.okta.tools.awscli.main(awscli.java:39)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
at java.base/java.lang.ProcessImpl.create(Native Method)
at java.base/java.lang.ProcessImpl.<init>(ProcessImpl.java:420)
at java.base/java.lang.ProcessImpl.start(ProcessImpl.java:151)
at java.base/java.lang.ProcessBuilder.start(ProcessBuilder.java:1107)
... 2 more
C:\Users\dlgoodr\.okta>
1.0.2 doesn't appear to be available yet, but I'll happily try it when it's available.
My mistake.1.0.1 should work.
I’ve been working on a fresh Amazon WorkSpace running Windows to try and figure this out.
Did you install awscli with PIP? I don’t think that will work properly. I haven’t tested it at least.
Try running this to install v1.0.1 with AWS CLI:
New-Item -Path $HOME\.okta -ItemType "folder"
(New-Object System.Net.WebClient).DownloadFile(
"https://github.com/oktadeveloper/okta-aws-cli-assume-role/releases/download/v1.0.1/okta-aws-cli-1.0.1.jar",
"$HOME\.okta\okta-aws-cli-1.0.1.jar")
if (!([System.IO.Directory]::Exists('$HOME\Documents\WindowsPowerShell\'))) {
New-Item -Path $profile -ItemType File -Force
}
Add-Content -Path $profile -Value '#OktaAWSCLI
function With-Okta {
Param([string]$Profile)
Write-Host $args
$OriginalOKTA_PROFILE = $env:OKTA_PROFILE
try {
$env:OKTA_PROFILE = $Profile
$InternetOptions = Get-ItemProperty -Path "HKCU:\Software\Microsoft\Windows\CurrentVersion\Internet Settings"
($ProxyHost, $ProxyPort) = $InternetOptions.ProxyServer.Split(":")
$NonProxyHosts = [System.String]::Join("|", ($InternetOptions.ProxyOverride.Replace("<local>", "").Split(";") | Where-Object {$_}))
java "-Dhttp.proxyHost=$ProxyHost" "-Dhttp.proxyPort=$ProxyPort" "-Dhttps.proxyHost=$ProxyHost" "-Dhttps.proxyPort=$ProxyPort" "-Dhttp.nonProxyHosts=$NonProxyHosts" -classpath $HOME\.okta\* com.okta.tools.WithOkta @args
} finally {
$env:OKTA_PROFILE = $OriginalOKTA_PROFILE
}
}
function aws {
Param([string]$Profile)
With-Okta -Profile $Profile aws --profile $Profile @args
}
function sls {
Param([string]$Profile)
With-Okta -Profile $Profile sls --stage $Profile @args
}
'
aws prod sts get-caller-identity
aws research sts get-caller-identity
You should be prompted for the first role selection and not for the second. The second role selection should come up without requiring re-authentication as it reuses your existing Okta session.
Try this and see if it works for you.
It seems that Windows users need to install the AWS CLI using the MSI Installer. The Okta AWS CLI Assume Role tool does not work with the AWS CLI installed using pip
.
Thank you, @junkangli. That’s my experience as well.
I suspect that the way pip installs AWS CLI on Windows makes it unable to run with a CreateProcess call. Someone would probably have to alter it to inject cmd
and /c
as the ProcessBuilder’s first parameters on Windows only to make it work. That’s going to require a lot of testing since it could break all other platforms and change command quoting behavior that downstream scripts depend on.
@mraible this can be closed. It's an unrelated issue due to installing AWSCLI using PIP which doesn't really work on Windows.
Totally disagree. awscli
from pip on Windows works marvelously in all cases except for when used by okta-aws-cli-assume-role. The fact that the tool can't support this use case is still a defect.
@dlgoodr I don't appreciate this. I spent almost 10 hours trying to find fixes for you here. You paid nothing for that. You had almost 4 months to try my suggestions, particularly https://github.com/oktadeveloper/okta-aws-cli-assume-role/issues/85#issuecomment-375064040.
Supporting creating sub-processes for scripts (like PIP installs) on Windows is tricky. They are not full-fledged programs so ProcessBuilder can't invoke them. They work from the command prompt and PowerShell because the script interpreter kicks in.
The MSI install full-fledged programs that ProcessBuider can invoke and thus it works. Colleagues of mine experienced the same problem and switched to the MSI install without issues.
Is there a reason why the MSI can't work for you?
Encountering this same issue as well, and it has been a time vampire...
The aws-cli userguide implies the pip install route as the preferred method (https://docs.aws.amazon.com/cli/latest/userguide/installing.html).
Installing with the MSI also did not work for me, getting the same java error.
Looking into this a bit further, it looks like there is a bug which causes okta-aws-cli-assume-role to fail when using python3 and aws-cli. @AlainODea @dlgoodr
How to Reproduce
Notes
Thank you for the analysis, @Ray-B.
It looks like other Java folks have encountered a similar problem, it appears that ProcessBuilder doesn’t respect PATHEXT: https://stackoverflow.com/a/40670841
I think the solution to that is a bit tricky. It could fail from trying to run aws to trying to run aws.cmd.
I’m sure there is a more efficient way to handle this automatically.
For the time being, you could use my PowerShell installer and change aws to aws.cmd on this line: https://github.com/oktadeveloper/okta-aws-cli-assume-role/blob/master/bin/Install-OktaAwsCli.ps1#L70
That should fix the issue for you.
Once 1.0.3 is released, the release number in the installer can be changed. I haven’t figured out a way to that automatically yet.
@Ray-B it's worth trying the solution I posted here https://github.com/oktadeveloper/okta-aws-cli-assume-role/issues/151#issuecomment-408704347. Let me know how that works out.
See https://github.com/oktadeveloper/okta-aws-cli-assume-role/issues/151#issuecomment-408704347 for a working, tested workaround. It's a pretty light workaround only requiring tweaks to PowerShell profile.
@AlainODea Right'O. Thanks for that, aliasing the AWS command should work for now. Should we note that in the docs perhaps?
Could also talk to the Java folks, maybe file an issue and see what they have to say on this? Dunno.
@Ray-B updating the readme sounds like a good plan. Do you want to write up the readme change and submit a PR?
Sure. I'll have some time this week. 👍
@AlainODea FYI, the documentation hasn't been updated yet.
I had this issue today (this is why I am here!); I had installed the AWS CLI tools via the MSI installer, which creates an aws.cmd
batch file which itself calls into the Python 3.6 interpreter the MSI installer installs in C:\Program Files\Amazon\AWSCLI\runtime\
.
I then ran the command to install okta-aws-cli-assume-role using Install-OktaAwsCli.ps1
, and received the IOException upon verifying my setup.
The fix, as was suggested here, was to update the powershell $profile to change aws
to aws.cmd
.
Can this issue be fixed in Install-OktaAwsCli.ps1
by checking the output of Get-Command aws.cmd
, and - if a result is returned - updating the powershell profile template accordingly?
@testworksau you're right
Changing this: https://github.com/oktadeveloper/okta-aws-cli-assume-role/blob/eb6904319026f09f0c2710f5d6c819653cc32cd5/bin/Install-OktaAwsCli.ps1#L85 To this:
With-Okta -Profile $Profile ((Get-Command aws).Name) --profile $Profile @args
Fixes the issue solidly regardless of whether the command file is aws or aws.cmd.
After installing the aws cli through python, I attempt to run
awscli.bat
, and get the following error:My initial guess was an issue with the way my
PATH
variable was setup, but runningaws
from the command line resolves correctly, and it also looks like the correct directoryC:\Program Files\Python36\Scripts
has been added to thePATH
.Can you all identify anything that I may have missed in order to execute this command correctly?
Below is execution of
aws
from Powershell with Admin access. The same result also occurs from my user-level account.This is a snapshot of my system-level environment variables. I have included the Python paths in both User and System.