cvat-ai / cvat

Annotate better with CVAT, the industry-leading data engine for machine learning. Used and trusted by teams at any scale, for data of any scale.
https://cvat.ai
MIT License
12.26k stars 2.95k forks source link

Webhook error 502 after SSRF improvement #6760

Closed Sieltek closed 1 year ago

Sieltek commented 1 year ago

My actions before raising this issue

I have a Flask server running on the same server as CVAT. When i create a webhook request and i click on ping for exemple, i have an error 502 with dns name and 407 with IP. I read the patch with SSRF sercurity improvement but even if i set the SMOKESCREEN_OPTS I'm getting the same error, the only difference is that I does'nt have error description when I ping with the API and i only have 502 and no 407.

Steps to Reproduce (for bugs)

  1. Set .env SMOKESCREEN_OPTS with your server IP then docker compose up
  2. Create flask server endpoint
  3. Create webhook pointing on same IP as server but different port

Expected Behaviour

Ping request should success with status_code 200

Current Behaviour

Ping request fails with status_code 502

Possible Solution

Maybe my SMOKESCREEN_OPTS in my .env is not properly set. Or SMOKESCREEN_OPTS does'nt work correctly with webhooks.

Context

Here is my webhook image with {{IP}} = my server IP, for exemple 123.45.67.89

@flask.route("/test", methods=['POST']) def cvat(): content = request.get_json(silent=True) print(content) return {}, 200

My Flask server was working great before the update 2.5.0 so i don't think that it comes from this one.

Here is my .env: CVAT_VERSION=v2.6.0 SMOKESCREEN_OPTS=123.45.67.89

Your Environment

zhiltsov-max commented 1 year ago

Hi, for a local deployment with docker you can try the following:

  1. Create a user-defined bridge network: docker network create -o "com.docker.network.bridge.enable_icc=true" -o "com.docker.network.driver.mtu=1500" -o "com.docker.network.bridge.enable_ip_masquerade=true" cvat-local-bridge

  2. Update the CVAT docker-compose.dev.yml file with these lines:

    
    [...]

networks: cvat: name: cvat-local-bridge external: true


And restart the containers with `docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d`.

3. You can deploy the other service you're developing on the localhost or in a container. If deploying in a container, make sure to join the container to this new network (either manually with `docker network connect cvat-local-bridge <yourcontainer>` or automatically in docker-compose the same way as for CVAT). Note that the localhost is also visible in this network, so you can reach it if needed. The gateway ip (for localhost) can be found in the `docker network inspect cvat-local-bridge` "IPAM" section (another way is to use `ip addr show dev br-<network id from 'docker network ls'>`).

The commands above are for linux, so windows deployments can have some differences. Please do not use this approach in production deployments.
SpecLad commented 1 year ago

SMOKESCREEN_OPTS needs to be set to a Smokescreen option (EDIT: or list of options), not just an IP address. For example, SMOKESCREEN_OPTS=--allow-address=123.45.67.89.

You can find the list of available options in the Smokescreen README: https://github.com/stripe/smokescreen/blob/master/README.md.

Sieltek commented 1 year ago

SMOKESCREEN_OPTS needs to be set to a Smokescreen option (EDIT: or list of options), not just an IP address. For example, SMOKESCREEN_OPTS=--allow-address=123.45.67.89.

You can find the list of available options in the Smokescreen README: https://github.com/stripe/smokescreen/blob/master/README.md.

Thanks for your help everyone, @SpecLad solution works perfectly <3

DenisN03 commented 10 months ago

Hello! I'm facing the same problem. The only difference is that I am using fastapi in a docker container. Can you tell me what other settings I need to look at? My docker-compose.yml:

version: '2.3'
services:
  cvat_wh:
    image: cvat_wh
    container_name: cvat_wh
    build:
      context: .
      dockerfile: Dockerfile
      network: "host"
    ports:
      - 8001:8001
    restart: always

When running CVAT, I also do an export, but with the port specified: export SMOKESCREEN_OPTS=--allow-address={IP}:8001 Additionally, I checked that my server on fasapi is running. When accessed from the system, the server returns code 200.

zhiltsov-max commented 10 months ago

@DenisN03, have you tried the advice about using a user-defined network in docker compose? It's possible that CVAT server can't reach your application from the docker container.

DenisN03 commented 10 months ago

Merged everything into user-defined network. Tried with fastapi ip 0.0.0.0.0 and 127.0.0.1, but the problem remains. What ip should I specify for the fastapi server?

zhiltsov-max commented 10 months ago

@DenisN03, you can find the container ip by calling docker container inspect <your_container> | jq '.[0]["NetworkSettings"]["Networks"]'. You'll also need to add this ip into the smokescreen params. The fastapi server itself can serve at 0.0.0.0.

DenisN03 commented 10 months ago

I got it! Thank you very much for your help!