Document how to debug locally w/ popular IDEs

[di]

Create well-structured documentation about debugging functions locally with popular IDEs:

• [ ] PyCharm
• [ ] VSCode
• others?

Originally posted by @yuri-sergiichuk in https://github.com/GoogleCloudPlatform/functions-framework-python/issues/29#issuecomment-611113052

[ChristianSauer]

I think this is a very important topic - it confused me massivley at the beginning. As a problem: Even now I am unable to disable the debugger --debugger False does not work, and it seems to be active per default.

[di]

@ChristianSauer, can you file a separate issue with more details? I think I know what's happening but want to confirm.

[jduplessis]

I think this would be very helpful. I'm burning quite some time trying to figure out how to debug function-framework running in a container. It eventually led me to look into Python remote debugging on VSCode in general.

I found some great material on this by @marcel-dempers. Trying to convert it to something I can use with functions-framework has however not proven successful (...yet), mainly due to this:

I would really appreciate any pointers to existing material that could help.

[di]

@jduplessis Could you link to the guide you've tried to use so far?

[jduplessis]

I started off with this which enabled me to debug a Flask app running in a container. I then attempted to replace the flask "overhead" with functions-framework, but I haven't been able to hit a break point yet.

I made small changes to the code above, mainly trying to get something like this working:

This is what I have thus-far: https://github.com/jduplessis/getting_there

*Disclaimer: My "Python" isn't that great

[di]

It's possible that we might need to make some changes to the framework to support using ptvsd.

Would you be able to try installing the framework from this branch and see if you're able to hit a breakpoint?

You can install it from there by making this change to your Dockerfile:

- RUN pip install gunicorn functions-framework
+ RUN pip install git+https://github.com/GoogleCloudPlatform/functions-framework-python@cloud-debugger

EDIT: Actually, you won't have git installed in that base image, you'll need to do:

FROM python3.7.3-slim

RUN apt-get update
RUN apt-get install -y git

as well.

[jduplessis]

When I attach the debugger, I get this (which looks promising):

But then I run into this:

[di]

Hmm, any idea where it's getting the 45397 port from? That doesn't seem to be described at https://code.visualstudio.com/docs/containers/debug-python, just the ptvsd port.

[jduplessis]

I have no idea. The port in the error message changes every time I attach the debugger. I'm looking into a similar issue reported here: https://github.com/microsoft/ptvsd/issues/1560.

[di]

Ah, that seems to make sense. Can you try:

- CMD python -m ptvsd --host 0.0.0.0 --port 5678 --wait --multiprocess -m functions_framework --target hello --debug --port 8080
+ CMD python -m ptvsd --host 0.0.0.0 --port 5678 --wait --multiprocess -m functions_framework --target hello --port 8080

[jduplessis]

When I remove --debug I get a lot of errors in the container, but this focused my attention on --multiprocess. Removing the --multiprocess resolved the random port issue.

Still no luck with the break point:

EDIT: Removing only --debug resulted in this, but I'm not able to get a request through on 8080 anymore.

[di]

Thanks. It's very likely this is due to how the framework loads user code and additional work on your part won't resolve it. I will plan to take a closer look at this soon.

[jduplessis]

Thanks @di. I'll stick to functions-framework on local for now.

[jduplessis]

Apologies for hijacking this documentation issue. I thought I'd at least give some feedback for anyone who comes across this conversation.

We ended up using the VSCode extension for remote debugging in containers (Remote - Containers Extension).

This allows running the functions_framework Python module with debugging enabled (as you would locally), but still provides the isolated development environment we were looking for (in the container), and importantly... the breakpoints are hit!

[di]

Hi @jduplessis, glad to hear you were able to get it working!

If you're able to write up the steps you took in a little more detail, I think that would be a good starting point for the "VSCode" portion of this issue.

[jduplessis]

Sure, I'll provide details on our dev setup as soon as I have a quite moment. Tomorrow I'm going back in to the office for the first time since March, so things might be a bit crazy for a day or two.

[jduplessis]

We finally have a working dev setup. I asked @pengelbrecht2627 to provide some details. He put in most of the time on this. Our focus was on creating a dev environment that would:

1. Require very little setup on the development machine (VSCode, Extentions and Docker)
2. Work for a large number of functions in the same repo

@di, this might be overkill for what you had in mind for the documentation, but it can be simplified a lot. Also, this is only one way of setting up VSCode for debugging using functions-framework.

@pengelbrecht2627 it would be awesome if you could share your example repo as well.

[pengelbrecht2627]

While @jduplessis went at the solution from a container solution, I simultaneously tried a WSL and Poetry route. In the end we decided to streamline the WSL route to be closer to containers. This resulted in our use of the Remote - Containers Extension extension for VS Code while stripping Poetry.

In short, the extension allows VS Code to connect to a VS Code server in a Docker container. This container can be a pre-build Hub image, come from a Dockerfile or a docker-compose file(s). We just went the compose route because we will have multiple GCF in the same folder and want to share the configuration between all of them as much as possible.

Our compose setup builds two images. A base that just install the function's requirements.txt. Then a debug image extends this image with installs for linters, formatters, test frameworks, etc. (We have this separation because we plan on using the same setup for Cloud Run where the base will be deployed). Thus, the compose targets this debug image.

In the end, each GCF is separated from the others during development and debugging works as if the coding happens on the host directly. We actually used a test repo to figure this out - more details are there.

@di you will probably be most interested in the launch.json file for the debugging directly from VS Code.

[di]

Awesome, thank you to you both! I'd be interested to see how much we could simplify/generalize this example repo and integrate it as an example in this repo (if you are willing).

[pengelbrecht2627]

@di you are welcome too. We will likely integrate any simplifications back too 😄

[di]

@pengelbrecht2627 Unfortunately I am unable to without a license for your repo 🙂

[jduplessis]

@pengelbrecht2627 I'd be happy to donate some of your work time to generalize the solution for @di... Something that allows for easy setup of a dev environment to debug a single cloud function using functions-framework (on VSCode). The setup we have now is perfect for our purposes, but it makes it difficult to "teach" someone how to get the debug working quickly on a completely new project.

[pengelbrecht2627]

@di I created a minimal demo repo (with a MIT license). It is at pengelbrecht2627/functions-framework-python-vscode

[di]

Thanks! Looking at that, it seems like the only thing that's required is the correct launch.json file, is that correct?

[pengelbrecht2627]

Yes, and the plugin to make sense of the Python type option.

[joelgerard]

Possibly a horrible kluge, but this worked for me with PyCharm CE. Set a breakpoint in your own main.py, hit the Debug button, then curl or use your browser to hit the endpoint. @di, WDYT?

[di]

@joelgerard I'm curious why the "Script path" looks like that. Seems like that would be useful if you're working on the framework itself, but if you're just working on your own function, should that be something like functions-framework, no?

[joelgerard]

That's the kluge part. I couldn't get pycharm to run functions-framework and hook the debugger. I'll take another look later, but that's as far as I've got so far.

[joelgerard]

Better? https://github.com/joelgerard/functions-framework-python/blob/pycharm/PYCHARM.md

:)

[yuri-sergiichuk]

@joelgerard you can try to change the script path to module name:

Then PyCharm will pick up automatically your virtual/system Python and just do its job. That's the original approach I've mentioned in #29, but it was not working before the 1.3.0 release.

[faridghar]

For me with VS Code it was just a matter of adding the following debug configuration to my launch.json file:

{
   "name": "Functions Framework",
   "type": "python",
   "request": "launch",
   "program": "${env:VIRTUAL_ENV}/bin/functions-framework",
   "args": ["--target=main",]
}

One thing I did have to do though is add the following to my .zshrc file:

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

Apparently this has to do with a change that was introduced in Mac 10.13 as discussed here

[dkocich]

basically, I use this config on Win PC (see picture) now for debug and there is a button to save it in the repository as a file .run/myConf.xml which enables users to easily document and set up debugger in Pycharm for multiple users/IDEs

<component name="ProjectRunConfigurationManager">
  <configuration default="false" name="functions-framework opt_dev debug" type="PythonConfigurationType" factoryName="Python">
    <module name="berider-opt-be" />
    <option name="INTERPRETER_OPTIONS" value="" />
    <option name="PARENT_ENVS" value="true" />
    <envs>
      <env name="PYTHONUNBUFFERED" value="1" />
    </envs>
    <option name="SDK_HOME" value="" />
    <option name="WORKING_DIRECTORY" value="$PROJECT_DIR$" />
    <option name="IS_MODULE_SDK" value="true" />
    <option name="ADD_CONTENT_ROOTS" value="true" />
    <option name="ADD_SOURCE_ROOTS" value="true" />
    <EXTENSION ID="PythonCoverageRunConfigurationExtension" runner="coverage.py" />
    <EXTENSION ID="net.ashald.envfile">
      <option name="IS_ENABLED" value="false" />
      <option name="IS_SUBST" value="false" />
      <option name="IS_PATH_MACRO_SUPPORTED" value="false" />
      <option name="IS_IGNORE_MISSING_FILES" value="false" />
      <option name="IS_ENABLE_EXPERIMENTAL_INTEGRATIONS" value="false" />
      <ENTRIES>
        <ENTRY IS_ENABLED="true" PARSER="runconfig" />
      </ENTRIES>
    </EXTENSION>
    <option name="SCRIPT_NAME" value="venv/Lib/site-packages/functions_framework/" />
    <option name="PARAMETERS" value="--port 8085 --signature-type http --target opt_dev --debug" />
    <option name="SHOW_COMMAND_LINE" value="false" />
    <option name="EMULATE_TERMINAL" value="false" />
    <option name="MODULE_MODE" value="false" />
    <option name="REDIRECT_INPUT" value="false" />
    <option name="INPUT_FILE" value="" />
    <method v="2" />
  </configuration>
</component>

[haidar-h]

Worked by Adding this to launch.json in vscode

{
      "name": "Python: Firebase Cloud Function",
      "type": "python",
      "request": "launch",
      "module": "functions_framework",
      "cwd": "${workspaceFolder}",
      "args": ["--target=yourCloudFunctionName", "--source=path/to/main.py", "--debug"],
      "console": "integratedTerminal"
    }