Closed forthal closed 2 years ago
Hey @forthal,
I've created a test branch with the wanted funtionality. Sadly I can not test it myself, so could you try and test it? https://github.com/BenediktBertsch/proxmox-api-go/tree/add-httpheaders
There is an option to use a proxy server for requests, for example mitmproxy. You can then add your headers to mitmproxy https://docs.mitmproxy.org/stable/addons-examples/
Hi @BenediktBertsch,
Tried your changes - no success. Headers are present (I've checked promox-go-api
command with -debug
), but seems like Client/Session is not handling properly Cloudflare (used same token for Nomad and it works in same shell) - getting 401 Unauthorized:
./proxmox-api-go -debug start 100
2022/10/03 20:52:00 >>>>>>>>>> REQUEST:
GET /api2/json/cluster/resources?type=vm HTTP/1.1
Host: proxmox.***.com
User-Agent: Go-http-client/1.1
Accept: application/json
Cf-Access-Client-Id: ***
Cf-Access-Client-Secret: ***
Cookie: PVEAuthCookie=PVE:***@***:***::***
Csrfpreventiontoken: ***:***
Accept-Encoding: gzip
2022/10/03 20:52:03 <<<<<<<<<< RESULT:
HTTP/2.0 401 Unauthorized
Connection: close
Alt-Svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
Cache-Control: max-age=0
Cf-Cache-Status: DYNAMIC
Cf-Ray: ***
Date: Mon, 03 Oct 2022 18:52:04 GMT
Expires: Mon, 03 Oct 2022 18:52:01 GMT
Nel: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}
Pragma: no-cache
Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v3?s=***"}],"group":"cf-nel","max_age":604800}
Server: cloudflare
Set-Cookie: CF_Authorization=***; Expires=Tue, 04 Oct 2022 18:52:01 GMT; Path=/; Secure; SameSite=none
Maybe it's related to some redirections that occurs on Cloudflare side? Strange that using Nomad provider it's working like a charm. Using same Service Token I'm able to hit to the login page:
curl --request GET \
--url https://proxmox.***.com/ \
--header 'CF-Access-Client-Id: ***' \
--header 'CF-Access-Client-Secret: ***'
response:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<title>*** - Proxmox Virtual Environment</title>
<link rel="icon" sizes="128x128" href="/pve2/images/logo-128.png" />
<link rel="apple-touch-icon" sizes="128x128" href="/pve2/images/logo-128.png" />
<link rel="stylesheet" type="text/css" href="/pve2/ext6/theme-crisp/resources/theme-crisp-all.css?ver=7.0.0" />
<link rel="stylesheet" type="text/css" href="/pve2/ext6/crisp/resources/charts-all.css?ver=7.0.0" />
<link rel="stylesheet" type="text/css" href="/pve2/fa/css/font-awesome.css" />
<link rel="stylesheet" type="text/css" href="/pve2/css/ext6-pve.css?ver=7.2-7" />
<link rel="stylesheet" type="text/css" href="/pwt/css/ext6-pmx.css?ver=3.5.1" />
<script type='text/javascript'>function gettext(buf) { return buf; }</script>
<script type="text/javascript" src="/pve2/ext6/ext-all.js?ver=7.0.0"></script>
<script type="text/javascript" src="/pve2/ext6/charts.js?ver=7.0.0"></script>
<script type="text/javascript" src="/pve2/js/u2f-api.js"></script>
<script type="text/javascript" src="/qrcode.min.js"></script>
<script type="text/javascript">
Proxmox = {
Setup: { auth_cookie_name: 'PVEAuthCookie' },
defaultLang: 'en',
NodeName: '***',
UserName: '',
CSRFPreventionToken: 'null'
};
</script>
<script type="text/javascript" src="/proxmoxlib.js?ver=3.5.1"></script>
<script type="text/javascript" src="/pve2/js/pvemanagerlib.js?ver=7.2-7"></script>
<script type="text/javascript" src="/pve2/ext6/locale/locale-en.js?ver=7.0.0"></script>
<script type="text/javascript">
if (typeof(PVE) === 'undefined') PVE = {};
Ext.History.fieldid = 'x-history-field';
Ext.onReady(function() { Ext.create('PVE.StdWorkspace');});
</script>
</head>
<body>
<!-- Fields required for history management -->
<form id="history-form" class="x-hidden">
<input type="hidden" id="x-history-field"/>
</form>
</body>
</html>
EDIT: ok, it's related to Cloudflare - I'll try to tinker the setup.
Using:
curl --request POST \
--url https://proxmox.***.com/api2/json/access/ticket \
--header 'CF-Access-Client-Id: ***' \
--header 'CF-Access-Client-Secret: ***' \
-d 'username=***@pam' \
--data-urlencode 'password=***'
response:
{
"data": {
"ticket": "PVE:***@pam:***::***",
"CSRFPreventionToken": "***:***",
"cap": {
"storage": {
"Datastore.Allocate": 1,
"Datastore.AllocateSpace": 1,
"Datastore.Audit": 1,
"Datastore.AllocateTemplate": 1,
"Permissions.Modify": 1
},
"vms": {
"Permissions.Modify": 1,
"VM.Config.Memory": 1,
"VM.Snapshot": 1,
"VM.Config.CDROM": 1,
"VM.Audit": 1,
"VM.Snapshot.Rollback": 1,
"VM.Config.Network": 1,
"VM.Config.HWType": 1,
"VM.Config.Options": 1,
"VM.Config.CPU": 1,
"VM.Clone": 1,
"VM.Backup": 1,
"VM.Allocate": 1,
"VM.Config.Cloudinit": 1,
"VM.Console": 1,
"VM.PowerMgmt": 1,
"VM.Config.Disk": 1,
"VM.Migrate": 1,
"VM.Monitor": 1
},
"sdn": {
"SDN.Allocate": 1,
"Permissions.Modify": 1,
"SDN.Audit": 1
},
"access": {
"User.Modify": 1,
"Group.Allocate": 1,
"Permissions.Modify": 1
},
"nodes": {
"Sys.Audit": 1,
"Sys.Modify": 1,
"Sys.Console": 1,
"Sys.Syslog": 1,
"Sys.PowerMgmt": 1,
"Permissions.Modify": 1
},
"dc": {
"SDN.Audit": 1,
"SDN.Allocate": 1,
"Sys.Audit": 1
}
},
"username": "***@pam"
}
}
Hi @mleone87, mitmproxy adds much more complexity - you need to deploy it somewhere (you have no access to Terraform Cloud runner, you could use some "hax" with local provisioners, etc., but to be honest - adding and handling headers is much more simpler) .
Hey @forthal,
just read your good debug, whats about this:
Cf-Access-Client-Id: Cf-Access-Client-Secret:
Is it just a debugging thing that the f in CF is lowercase? I think its important to be uppercase.
Hi @BenediktBertsch,
I think "debug" mode is lowercasing this. I've double checked I'm using proper headers:
export HTTP_HEADERS="CF-Access-Client-Id,***,CF-Access-Client-Secret,***"
EDIT: I think I found the culprit: https://pkg.go.dev/net/http#Header.Add:
The key is case insensitive; it is canonicalized by CanonicalHeaderKey.
Hey @forthal,
pushed a fix for the case-sensitivity for headers. Pls check it out and test it once again.
Hi @BenediktBertsch, I've applied same fix locally, but having:
2022/10/03 23:18:15 invalid character '<' looking for beginning of value
after building and running. Setting up a debugger would help a lot :D
EDIT: I think I've solved it. Check out PR created to your fork.
I wanted to ask If there can be an option to add custom headers for request to Proxmox API? I have Proxmox behind Cloudflare Access and for accessing Proxmox I need to add 2 custom headers to request https://developers.cloudflare.com/cloudflare-one/identity/service-tokens/ so then in: https://registry.terraform.io/providers/Telmate/proxmox/latest I could specify those "secrets" in order to access Proxmox hosted in homelab which is behing Cloudflare Access using Terraform Cloud.
Something like this: https://github.com/hashicorp/terraform-provider-nomad/blob/main/nomad/provider.go#L103
Thanks,