tus / tusd

Reference server implementation in Go of tus: the open protocol for resumable file uploads
https://tus.github.io/tusd
MIT License
2.95k stars 467 forks source link

Hooks not trigger #1002

Closed lgcshy closed 9 months ago

lgcshy commented 9 months ago

Question I set up http hook endpoint. There is no way to distinguish the type of hook when receiving a hook, because the type parsed through json is always empty.

Setup details my docker-compose.yml:

version: "3"
services:
  minio:
    image: minio/minio:RELEASE.2019-08-14T20-37-41Z
    networks: 
      - tusd
    ports:
      - "9000:9000"
    volumes:
      - "./data/:/data/"
    environment: 
      MINIO_ACCESS_KEY: AKIAIOSFODNN7EXAMPLE
      MINIO_SECRET_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
    command: server /data
  tusd:
    image: tusproject/tusd:latest
    networks: 
      - tusd
    ports:
      - "1080:1080"
    environment: 
      AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
      AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
      AWS_REGION: eu-west-1
    command: -s3-bucket mybucket -s3-endpoint http://minio:9000 -hooks-http http://x.x.x.x:8889/write -hooks-http-retry 3 -hooks-http-backoff 3  -hooks-enabled-events pre-create,post-finish -verbose
      - minio
  frontend:
    image: nginx:1.17.3-alpine
    ports:
      - "9506:80"
    volumes:
      - "./public/:/usr/share/nginx/html:ro"
    depends_on: 
      - tusd
networks: 
  tusd: 

my hook server:

func main() {
    app := fiber.New()
    app.Post("/write", func(c *fiber.Ctx) error {
        fmt.Printf("REQUEST %+v\n", c.Body())
        data := c.Body()
        body := make([]byte, len(data))
        copy(body, data)
        req := &hooks.HookRequest{}
        var bb interface{}
        json.Unmarshal(body, &bb)
        fmt.Printf("BODY %+v\n", bb)
        if err := json.Unmarshal(body, req); err != nil {
            return c.Status(fiber.StatusOK).JSON(hooks.HookResponse{
                HTTPResponse: tusd.HTTPResponse{
                    StatusCode: fiber.StatusBadRequest,
                    Body:       "failed",
                },
            })
        }
        if req.Type == hooks.HookPreCreate {
            token := req.Event.HTTPRequest.Header.Get("Authorization")
            fmt.Printf("token: %s\n", token)
            metaData := req.Event.Upload.MetaData
            metaData["token"] = token
            metaData["creation_time"] = time.Now().Format("2006-01-02 15:04:05")
            id := uuid.New().String()
            metaData["id"] = id
            return c.Status(fiber.StatusOK).JSON(hooks.HookResponse{
                HTTPResponse: tusd.HTTPResponse{
                    StatusCode: fiber.StatusOK,
                    Body:       "OK",
                },
                ChangeFileInfo: tusd.FileInfoChanges{
                    ID: id,
                    MetaData: metaData,
                },
            })

        }
        if req.Type == hooks.HookPostFinish {
            fmt.Println("HOOK POST FINISH")
            return c.Status(fiber.StatusOK).JSON(hooks.HookResponse{
                HTTPResponse: tusd.HTTPResponse{
                    StatusCode: fiber.StatusOK,
                    Body:       "OK",
                },
            })
        }
        return c.Status(fiber.StatusOK).JSON(hooks.HookResponse{
            HTTPResponse: tusd.HTTPResponse{
                StatusCode: fiber.StatusOK,
                Body:       "OK",
            },
        })
    })

    app.Listen(":8889")
}

----------------------logs -------------------- tusd logs:

docker-tusd-1      | [tusd] 2023/09/20 04:07:02.699462 event="ResponseOutgoing" status="201" method="POST" path="" requestId="" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.718478 event="RequestIncoming" method="OPTIONS" path="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" requestId="" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.718504 event="ResponseOutgoing" status="200" method="OPTIONS" path="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" requestId="" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.726321 event="RequestIncoming" method="PATCH" path="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" requestId="" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.729428 event="ChunkWriteStart" id="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" maxSize="4515" offset="0" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.733479 event="ChunkWriteComplete" id="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" bytesWritten="4515" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.737551 event="ResponseOutgoing" status="204" method="PATCH" path="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" requestId="" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.737656 event="UploadFinished" id="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" size="4515" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.737662 event="HookInvocationStart" type="post-finish" id="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" 
docker-tusd-1      | [tusd] 2023/09/20 04:07:02.738332 event="HookInvocationFinish" type="post-finish" id="2f4521384942cd6efdc9ff7f0a7bd74c+fc3602c9-58cf-4a8d-8baf-d4f6a58408a9" 
hook server log:
BODY map[HTTPRequest:map[Header:map[Accept:[*/*] Accept-Encoding:[gzip, deflate] Accept-Language:[zh-CN,zh;q=0.9,en;q=0.8,en-GB;q=0.7,en-US;q=0.6] Connection:[keep-alive] Content-Length:[0] Host:[10.10.102.106:1080] Origin:[http://10.10.102.106:5221] Referer:[http://10.10.102.106:5221/] Tus-Resumable:[1.0.0] Upload-Length:[185890609] Upload-Metadata:[relativePath bnVsbA==,name YWxpbms4MC0yMDIyLTEyLTEzLnNxbA==,type YXBwbGljYXRpb24vb2N0ZXQtc3RyZWFt,filetype YXBwbGljYXRpb24vb2N0ZXQtc3RyZWFt,filename YWxpbms4MC0yMDIyLTEyLTEzLnNxbA==] User-Agent:[Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Edg/109.0.1518.140]] Method:POST RemoteAddr:10.6.10.55:51265 URI:/files/] Upload:map[ID: IsFinal:false IsPartial:false MetaData:map[filename:alink80-2022-12-13.sql filetype:application/octet-stream name:alink80-2022-12-13.sql relativePath:null type:application/octet-stream] Offset:0 PartialUploads:<nil> Size:1.85890609e+08 SizeIsDeferred:false Storage:<nil>]]

Unable to get exact hook type.......

Acconut commented 9 months ago

The latest tag for the Docker image still points to tusd v1, where the hook type is not present in the hook request: https://github.com/tus/tusd/blob/a1e1fdfbd78d50df108e98c2ad5a6b6477ece3ff/cmd/tusd/cli/hooks/http.go#L46 The main branch of this repo already contains the code for v2, which we hope to release today and you are already using in your hook implementation.

It should be enough for you to use the latest v2 release candidate image: tusproject/tusd:2.0.0-rc21

lgcshy commented 9 months ago

Thank you Aconut, thank you for your answer, it helped me a lot.