getmoto / moto

A library that allows you to easily mock out tests based on AWS infrastructure.
http://docs.getmoto.org/en/latest/
Apache License 2.0
7.6k stars 2.04k forks source link

Proxy mode doesn't work on Windows #7806

Closed dschultz0 closed 2 months ago

dschultz0 commented 3 months ago

I'm in the minority that uses Windows for development but I think I've found an issue with how proxy mode handles certificate names. When running it using the standard configuration, I received the following error trace. From a cursory look at the code, it appears that the CertificateCreator is trying to generate a cert file with a wildcard which isn't allowed in Windows.

I've been able to reproduce the problem by running the various environment variable examples described here: https://docs.getmoto.org/en/latest/docs/proxy_mode.html#environment-variables-configuration

Exception occurred during processing of request from ('127.0.0.1', 65381)
Traceback (most recent call last):
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\socketserver.py", line 692, in process_request_thread
    self.finish_request(request, client_address)
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\socketserver.py", line 362, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\site-packages\moto\moto_proxy\proxy3.py", line 94, in __init__
    BaseHTTPRequestHandler.__init__(self, *args, **kwargs)
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\socketserver.py", line 761, in __init__
    self.handle()
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\http\server.py", line 436, in handle
    self.handle_one_request()
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\http\server.py", line 424, in handle_one_request
    method()
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\site-packages\moto\moto_proxy\proxy3.py", line 116, in do_CONNECT
    certpath = self.cert_creator.create(self.path)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\djsch\anaconda3\envs\python311\Lib\site-packages\moto\moto_proxy\certificate_creator.py", line 82, in create
    with open(config_template_name, "w") as f:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 22] Invalid argument: 'C:\\Users\\djsch\\anaconda3\\envs\\python311\\Lib\\site-packages\\moto\\moto_proxy\\certs/*.us-east-1.amazonaws.com.conf'
bblommers commented 3 months ago

Hi @dschultz0, thanks for raising this. Fixing this should be relatively easy, but I am having trouble to even get to the stage where the proxy creates certificates.

Did you have any problems with the openssl command? We call openssl version using a regular Python subprocess, and later on some more involved openssl commands to actually create the certificates.

That part is currently failing in our CI, because Windows seems to require a full path to invoke this, i.e. C:/../openssl.exe version. At least when using regular bash - calling openssl works in Powershell, but I'm not seeing an easy to way to get our Python-code to explicitly use Powershell to invoke these commands.

dschultz0 commented 2 months ago

The existing code wasn't getting to the openssl step, but I grabbed the change from your pull request, and it worked great. Thanks for the fix!

I think the reason it worked for me and not in CI, is that I'm invoking it from within Powershell instead of the windows command shell (cmd). From within powershell any subprocesses are going to continue to use powershell, but for cmd there's no way to break out into a powershell subprocess. It looks like you're installing openssl in pwsh but then running motoproxy using cmd. If you change the execution to use pwsh it will probably work. If you really want to be comprehensive you can install and run it in cmd as well but that's probably overkill.

bblommers commented 2 months ago

The proxy actually calls openssl version at the startup, just to verify that the binary exists, that's why I assumed it could reach openssl. You can actually see that when running moto_proxy -v, that should print the SSL version in use.

Thanks for the hint about Powershell! That was indeed what was missing, now I managed to get it working in our CI.

This is now part of moto >= 5.0.12.dev2.