davidmreed / amaxa

A multi-object ETL tool for Salesforce.
https://amaxa.readthedocs.io
BSD 3-Clause "New" or "Revised" License
62 stars 12 forks source link

Running on Windows errors when using sfdx credentials #45

Open bowdi opened 4 years ago

bowdi commented 4 years ago

When running on Windows the following error is received when using sfdx credentials:

[DBG]: PS C:\Users\Will> amaxa .\data\data-move.yml --credentials .\data\cred.yml
amaxa : Traceback (most recent call last):
At line:1 char:1
+ amaxa .\data\data-move.yml --credentials .\data\cred.yml
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (Traceback (most recent call last)::String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\runpy.py", line 193, in _run_module_as_main
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\runpy.py", line 85, in _run_code
  File "C:\portapps\Python\Scripts\amaxa.exe\__main__.py", line 9, in <module>
  File "c:\portapps\python\lib\site-packages\amaxa\__main__.py", line 55, in main
    credential_loader.load()
  File "c:\portapps\python\lib\site-packages\amaxa\loader\core.py", line 67, in load
    step()
  File "c:\portapps\python\lib\site-packages\amaxa\loader\credentials.py", line 20, in _load
    self._load_v2()
  File "c:\portapps\python\lib\site-packages\amaxa\loader\credentials.py", line 146, in _load_v2
    capture_output=True,
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\subprocess.py", line 472, in run
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\subprocess.py", line 775, in __init__
  File "D:\obj\Windows-Release\37amd64_Release\msi_python\zip_amd64\subprocess.py", line 1178, in _execute_child
FileNotFoundError: [WinError 2] The system cannot find the file specified

adding "shell=True" to the subprocess call in _load_v2 in credentials.py seems to resolve:

                ["sfdx", "force:org:display", "--json", "-u", sfdx_username],
                encoding="utf-8",
                capture_output=True,
                shell=True,
davidmreed commented 4 years ago

Thanks for reporting this issue, @willbowdi! Amaxa's integration tests run on Linux, so this issue slipped through. I'll get a patch out and look at closing that testing hole.

davidmreed commented 3 years ago

@willbowdi Following up a long time later... I wasn't immediately able to reproduce this problem. It looks like you've probably installed Amaxa via pip and are using an existing install of Python. If you use the Amaxa executable for Windows, do you see the same behavior?

chfosli commented 3 years ago

I got the same error. Installed amaxa via pip. Running on Windows 10. Worked with the fix above.

ccrs commented 3 years ago

I confirm the same error on either:

davidmreed commented 3 years ago

After more research and attempts to reproduce - I still cannot reproduce this bug, and I don't think setting shell=True is the right fix (it would allow shell injection attacks).

I believe that users who are experiencing this issue probably have the PATH set up in an unusual way such that sfdx is only in the PATH in specific environments. I'd be happy to work interactively with anyone experiencing the issue to try to document the cause and resolution steps, but my current belief is that it's not a bug in Amaxa.

peternoges commented 2 years ago

hey @davidmreed cool library! i'm seeing The system cannot find the file specified when running amaxa --credentials cred.yml op.yml within windows 10 visualstudio code terminal.

My windows installation steps: BTW i had to deviate slightly from the (linux) installation instructions for my Windows 10. Specifically, your doc https://amaxa.readthedocs.io/en/latest/install.html mentions this:

$ cd amaxa $ python3 -m venv venv $ source venv/bin/activate $ pip install poetry $ poetry install

But since i'm on Windows I had to run this instead: $ cd amaxa $ python -m venv venv $ C:\Users\pnoges\myCompany\Documents\amaxa\venv\Scripts>activate $ pip install poetry $ poetry install

Notice lines 2 and 3 are different for windows, in my experience. Specifically i dropped the 3 from python3, to python to avoid some Python not found error; because i had installed python 3.10 but my machine wasnt recognizing python3. Specifically my shell would output 3.10 when executing py but it was throw an error when i typed python3. Furthemore, the line beginning with "$ source" was linux syntax and didnt work for me on windows so instead i CD'd into the venv (windows-style) and just ran the activate file as some blogs suggest.

Unfortunately i'm seeing The system cannot find the file specified when running amaxa --credentials cred.yml op.yml within windows 10 visualstudio code terminal.

My current goal is to use sfdx for amaxa authentication, and perhaps my cred.yml is incorrect (more on that below). The docs confused me because im not sure how flexible my options are for sfdx/amaxa to work together. Can i use a cred.yml like the below?

//cred.yml ... sfdx: myusername@company.com.sandboxname

Questions: Must i name the file credentials-sfdx.yml as suggested in one of the about.md files? or will cred.yml work?

The docs says two (contradicting?) things: 1) give the org the alias amaxa 2) You can supply an org’s alias or username

^1 The first point makes me think that i must create a sfdx project within amaxa folder and then sfdx:authenticate and give it the "amaxa" alias. ^2 The second point gives me hope that i can be more descriptive in my connection/auth/alias naming by providing the actual username. Is that true? In other words In the spirit of avoiding ETL mistakes i would much rather my cred file list the exact destination username as opposed to "amaxa".

Edit: still no luck, and I've even created the sfdx project naming it "amaxa" in the same place that I cloned this amaxa repo. Essentially adding sfdx project files to the same root. Then I sfdx authenticated to my sandbox, I gave the connection the alias "amaxa" and both of these commands failed, citing The system cannot find the file specified. : amaxa --credentials credentials-sfdx.yml op.yml amaxa --credentials cred.yml op.yml where credentials-sfdx.yml has the same contents as the repo's example credentials-sfdx.yml, and cred.yml contains my sandbox org username.

The trace for amaxa --credentials credentials-sfdx.yml op.yml -v verbose is: Traceback (most recent call last): File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\Scripts\amaxa.exe__main.py", line 7, in File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\site-packages\amaxa\main.py", line 80, in main credential_loader.load() File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\site-packages\amaxa\loader\core.py", line 67, in load
step() File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\site-packages\amaxa\loader\credentials.py", line 21, in _load self._load_v2() File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\site-packages\amaxa\loader\credentials.py", line 150, in _load_v2 result = subprocess.run( File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 501, in run with Popen(*popenargs, **kwargs) as process: File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 966, in
init__ self._execute_child(args, executable, preexec_fn, close_fds, File "C:\Users\pnoges\AppData\Local\Programs\Python\Python310\lib\subprocess.py", line 1435, in _execute_child hp, ht, pid, tid = _winapi.CreateProcess(executable, args, FileNotFoundError: [WinError 2] The system cannot find the file specified

Edit: before this writeup I did add shell=True, as suggested by others in this Issue forum.

I must be making some mistake, perhaps?

davidmreed commented 2 years ago

@peternoges Thank you for sharing these details! I want to address three distinct threads in your query.

Installation

I generally recommend that users install the binary available on each release in GitHub wherever possible, rather than directly installing Python and setting up a virtual environment. I'll review the installation and setup guide. I do not develop on Windows (other than in WSL), so I may not be able to provide comprehensive setup instructions for development-style installs on that platform.

Using SFDX credentials

Must i name the file credentials-sfdx.yml as suggested in one of the about.md files? or will cred.yml work?

No. You can name the file whatever you like (although you should use a .yml extension, unless you prefer JSON format, in which case .json - Amaxa supports both). Just provide the file name as the argument to --credentials.

The docs says two (contradicting?) things: give the org the alias amaxa

This is for running test scripts. When you're working with Amaxa on your own orgs and data, you can use whatever alias you prefer.

You can supply an org’s alias or username

This should work, with any org in the SFDX keychain.

Error with SFDX

Neither of the questions/issues above appears to be related to this particular error. Thank you for your report; I would like very much to identify the underlying issue once and for all. A few questions to start with:

  1. Are you using the command prompt or PowerShell?
  2. How did you install the SFDX CLI?
  3. Are you aware of any modifications to your path environment variable?
  4. If, in the same terminal session where you ran Amaxa and received this error, you run any sfdx command, does it work?
peternoges commented 2 years ago

hey @davidmreed - here's more information in response to your recent message.

  1. both cmd and pwsh produce the same error message.
  2. i forget my exact sfdx cli installation steps but sfdx works well (use it all the time) and i believe i followed something like https://developer.salesforce.com/docs/atlas.en-us.sfdx_setup.meta/sfdx_setup/sfdx_setup_intro.htm
  3. i'll paste an image of my system and user environment path variables: paths
  4. from amaxa root folder i run sfdx and the output is:
    
    VERSION
    sfdx-cli/7.130.1 win32-x64 node-v16.13.1

USAGE $ sfdx [COMMAND]

TOPICS alias manage username aliases auth authorize an org for use with the Salesforce CLI config configure the Salesforce CLI force tools for the Salesforce developer info access cli info from the command line plugins add/remove/create CLI plug-ins

COMMANDS autocomplete display autocomplete installation instructions commands list all the commands help display help for sfdx plugins list installed plugins update update the sfdx CLI which show which plugin a command is in


I'd be happy to screenshare.