Unable to Create/Update/Delete secrets via API w/ Machine Identity (E2EE disabled)

jacksch commented 8 months ago

Describe the bug

Apologies if this is a duplicate issue, or known issue. I've not been able to find any other mentioned of this issue.

Using Machine Identities, I am unable to POST, PATCH, or DELETE secrets using the REST API w/ E2EE disabled on v0.43.14 (latest)

POST and PATCH return 500 - See error message below
DELETE returns 401 - See error message below
GET works great.

I have followed the documentation outlined here: https://infisical.com/docs/api-reference/overview/examples/e2ee-disabled The provided curl commands do not work for POST, PATCH or DELETE. I've taken a look at the API spec and I appear to have it right.

The Machine Identity has been elevated to full admin permissions to try and alleviate any permission errors.

It's very possible I'm missing something here. I also appreciate that this is a very new feature.

Deploying Infisical self-hosted with the provided Docker Compose template on Debian. No further changes have been made.

To Reproduce

Steps to reproduce the behavior: Deploy Infisical using provided Docker Compose template. Connect directly. Create admin account, create project, and create Machine Identity. Create client credentials.

Obtain bearer token from /api/v1/auth/universal-auth/login Make a POST, PATCH or DELTE request with the bearer to /api/v3/secrets/raw/<secretName> using API reference curl command

Expected behavior

The secret should be created, updated, or deleted

Platform you are having the issue on:

Self-hosted. Docker, using Compose template on Debian remote server. HTTP requests attempted from Windows (pwsh - Invoke-RestMethod) and Linux (curl)

Additional context

POST/PATCH error

{
    "type": "internal_server_error",
    "message": "The server encountered an error while processing the request",
    "context": {},
    "level": 50,
    "level_name": "ERROR",
    "status_code": 500,
    "datetime_iso": "2024-01-19T04:22:52.213Z",
    "application": "unknown",
    "extra": []
}

DELETE error

{
    "type": "unauthorized",
    "message": "Failed workspace authorization due to blind indices not being enabled",
    "context": {},
    "level": 30,
    "level_name": "INFO",
    "status_code": 401,
    "datetime_iso": "2024-01-19T04:31:48.817Z",
    "application": "unknown"
}

dangtony98 commented 8 months ago

Hi @jacksch,

Can you test this out on a new project on Infisical? — Based on the error, it seems like you might be trying to fetch secrets via API for an old project in Infisical.

jacksch commented 8 months ago

Hi @dangtony98

Thanks for your help. I've created a new project and updated the workspace ID in my requests to suit.

However, unfortunately I am still only able to GET a secret that I created in that project via the web. Attempting to POST a new secret, or PATCH the previously created secret is met with the same 500 error.

Let me know if there is any further troubleshooting I can do on my end. Are there any application logs I can retrieve from the Docker container?

maidul98 commented 8 months ago

Logs would be helpful, they can be found in your docker containers standard out. Also have you disabled end to end encryption in the project stetting?

jacksch commented 8 months ago

Hi @maidul98,

I've done some more troubleshooting on this today and I've been able to get it to work. Seems I didn't have my Content-Type headers set correctly, so Infisical was expecting JSON, but receiving x-www-form-urlencoded (or vice versa). I'm now able to POST, PATCH and DELETE. I'm working with Powershell primarily here so some things are a black box. Default behaviour/headers from Invoke-RestMethod appears to differer between methods. Sweet... However, curl is still troublesome for me. I may not be formulating my requests correctly, but I get a JSON parsing error according to the logs, even passing in raw, valid JSON using --data-raw.

Anyhow, I can keep on moving now. Looking at the container logs certainly did help with my own troubleshooting. If you believe the above curl issue is worth looking into, this is the error I get when executing the exact curl command documented here for a POST request.

{"level":50,"time":1706058318593,"pid":1,"hostname":"82da11ee5103","severity":"ERROR","msg":{"_logLevel":50,"_logName":"ERROR","statusCode":500,"type":"internal_server_error","context":{"exception":"Unexpected token T in JSON at position 108"},"extra":[],"stacktrace":["SyntaxError: Unexpected token T in JSON at position 108","    at JSON.parse (<anonymous>)","    at parse (/backend/node_modules/body-parser/lib/types/json.js:89:19)","    at /backend/node_modules/body-parser/lib/read.js:128:18","    at AsyncResource.runInAsyncScope (node:async_hooks:203:9)","    at invokeCallback (/backend/node_modules/raw-body/index.js:231:16)","    at done (/backend/node_modules/raw-body/index.js:220:7)","    at IncomingMessage.onEnd (/backend/node_modules/raw-body/index.js:280:7)","    at IncomingMessage.emit (node:events:525:35)","    at IncomingMessage.emit (node:domain:489:12)","    at endReadableNT (node:internal/streams/readable:1358:12)"]}}

Thank you to everyone for your time. I understand this is a noobie issue to run into haha. Can mark this as closed if you wish.

Infisical / infisical