Add HAProxy service

[simonjcarr]

First of all thanks for all the hard work on this package so far.

Is there any chance you will be able to add HAProxy support?

I know this is a tall ask due to the complexity of HAProxy, but this would open some amazing opportunities.

[jaredhendrickson13]

Hey!

Thanks for the request. This was requested quite a bit a while back (#10) but the project was not far enough along where it was a feasible request. In brief, the plan is to focus on pfSense core services until the majority of use cases have been met; then open it up to add-on packages. That being said, I believe we are very close to that point now. Service Watchdog and HAproxy are the two I've planned to do first. However, an addition to the API framework needs to be written first that can indicate which endpoints interact with an add-on package and also verify that the package is installed before accepting requests. As of right now there has been no progress towards that, but it should not be a difficult task. Afterward, development for package based endpoints like HAproxy can be done more reliably.

I'll keep this open as a feature request and update this issue as progress is made. It's unlikely this will make v1.4.0 but potentially v1.5.0.

Thanks again!

[sherif-fanous]

Hey @jaredhendrickson13

Thanks for a great package.

Just wanted to check on whether HAProxy is still on the roadmap.

Thanks

[jaredhendrickson13]

@sherif-fanous There's been a bit of a freeze for a few months on new endpoints and features due to the large refactor required for v1.6.0 in order to support pfSense 2.7.0 and pfSense Plus 23.01+; so there's no immediate plans for work on an HAProxy endpoint. After v1.6.0 it might be picked up again, but the priority will be attempting to get this package in a suitable state to be merged into pfSense's repos.

PRs are also welcome for anyone willing to take on the task.

[jaredhendrickson13]

HAProxy endpoints are available in the latest dev build of v2. I am still working on tests for these endpoints. If you're able, please test these endpoints and report any issues so they can be fixed and included in automated testing before v2 enters beta.

Thanks!

[KristijanL]

i am trying to use api/v2/services/haproxy/backend and have an issue with balance

❯ curl -X 'POST' \
  'https://domain.com/api/v2/services/haproxy/backend' \
  -H 'accept: application/json' \
  -H 'Authorization: KeyAuth 1234' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "test",
  "servers": [
    {
      "name": "test",
      "status": "active",
      "address": "192.168.1.1",
      "port": "80",
      "weight": 1,
      "ssl": false,
      "sslserververify": false
    }
  ],
  "balance": "",
  "connection_timeout": 30000,
  "server_timeout": 30000,
  "check_type": "none"
}'
{"code":400,"status":"bad request","response_id":"FIELD_EMPTY_NOT_ALLOWED","message":"Field `balance` cannot be empty.","data":[]}

❯ curl -X 'POST' \
  'https://domain.com/api/v2/services/haproxy/backend' \
  -H 'accept: application/json' \
  -H 'Authorization: KeyAuth 1234' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "test",
  "servers": [
    {
      "name": "test",
      "status": "active",
      "address": "192.168.1.1",
      "port": "80",
      "weight": 1,
      "ssl": false,
      "sslserververify": false
    }
  ],
  "balance": "none",
  "connection_timeout": 30000,
  "server_timeout": 30000,
  "check_type": "none"
}'
{"code":400,"status":"bad request","response_id":"FIELD_INVALID_CHOICE","message":"Field `balance` must be one of [, roundrobin, static-rr, leastconn, source, uri]","data":[]}

is it not possible to set "none" option.

[KristijanL]

shouldn't api/v2/services/haproxy/frontends return all available frontends?

current state:

❯ curl -X 'GET' \
  'https://domain.com/api/v2/services/haproxy/frontends?limit=0&offset=0' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic 12345' | jq .

{
  "code": 200,
  "status": "ok",
  "response_id": "SUCCESS",
  "message": "",
  "data": [
    {
      "id": "item",
      "name": null,
      "descr": null,
      "status": "active",
      "a_extaddr": null,
      "max_connections": null,
      "type": null,
      "ha_acls": null,
      "a_actionitems": null,
      "backend_serverpool": null,
      "socket_stats": false,
      "dontlognull": false,
      "dontlog_normal": false,
      "log_separate_errors": false,
      "log_detailed": false,
      "a_errorfiles": null,
      "client_timeout": 30000,
      "forwardfor": null,
      "httpclose": "http-keep-alive",
      "advanced_bind": null,
      "advanced": null
    }
  ]
}

[jaredhendrickson13]

@KristijanL

i am trying to use api/v2/services/haproxy/backend and have an issue with balance

❯ curl -X 'POST' \
  'https://domain.com/api/v2/services/haproxy/backend' \
  -H 'accept: application/json' \
  -H 'Authorization: KeyAuth 1234' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "test",
  "servers": [
    {
      "name": "test",
      "status": "active",
      "address": "192.168.1.1",
      "port": "80",
      "weight": 1,
      "ssl": false,
      "sslserververify": false
    }
  ],
  "balance": "",
  "connection_timeout": 30000,
  "server_timeout": 30000,
  "check_type": "none"
}'
{"code":400,"status":"bad request","response_id":"FIELD_EMPTY_NOT_ALLOWED","message":"Field `balance` cannot be empty.","data":[]}

❯ curl -X 'POST' \
  'https://domain.com/api/v2/services/haproxy/backend' \
  -H 'accept: application/json' \
  -H 'Authorization: KeyAuth 1234' \
  -H 'Content-Type: application/json' \
  -d '{
  "name": "test",
  "servers": [
    {
      "name": "test",
      "status": "active",
      "address": "192.168.1.1",
      "port": "80",
      "weight": 1,
      "ssl": false,
      "sslserververify": false
    }
  ],
  "balance": "none",
  "connection_timeout": 30000,
  "server_timeout": 30000,
  "check_type": "none"
}'
{"code":400,"status":"bad request","response_id":"FIELD_INVALID_CHOICE","message":"Field `balance` must be one of [, roundrobin, static-rr, leastconn, source, uri]","data":[]}

is it not possible to set "none" option.

The none option is set as an empty string in the config. The balance field was missing the allow_empty attribute that allows the field to be empty I added a fix for this in the latest dev build.

shouldn't api/v2/services/haproxy/frontends return all available frontends?

current state:

❯ curl -X 'GET' \
  'https://domain.com/api/v2/services/haproxy/frontends?limit=0&offset=0' \
  -H 'accept: application/json' \
  -H 'Authorization: Basic 12345' | jq .

{
  "code": 200,
  "status": "ok",
  "response_id": "SUCCESS",
  "message": "",
  "data": [
    {
      "id": "item",
      "name": null,
      "descr": null,
      "status": "active",
      "a_extaddr": null,
      "max_connections": null,
      "type": null,
      "ha_acls": null,
      "a_actionitems": null,
      "backend_serverpool": null,
      "socket_stats": false,
      "dontlognull": false,
      "dontlog_normal": false,
      "log_separate_errors": false,
      "log_detailed": false,
      "a_errorfiles": null,
      "client_timeout": 30000,
      "forwardfor": null,
      "httpclose": "http-keep-alive",
      "advanced_bind": null,
      "advanced": null
    }
  ]
}

The config path for the frontends just needed to be adjusted slightly, should also be fixed in the latest dev build.

Thanks!

[KristijanL]

works! will continue to test with my use case.

whats the correct way to update to latest dev version?

the pfsense-restapi update throws error

Fatal error: Uncaught Error: Call to undefined function RESTAPI\Core\Tools\get_pfsense_version() in /usr/local/pkg/RESTAPI/.resources/scripts/manage.php:303
Stack trace:
#0 /usr/local/pkg/RESTAPI/.resources/scripts/manage.php(453): update()
#1 {main}
  thrown in /usr/local/pkg/RESTAPI/.resources/scripts/manage.php on line 303
PHP ERROR: Type: 1, File: /usr/local/pkg/RESTAPI/.resources/scripts/manage.php, Line: 303, Message: Uncaught Error: Call to undefined function RESTAPI\Core\Tools\get_pfsense_version() in /usr/local/pkg/RESTAPI/.resources/scripts/manage.php:303
Stack trace:
#0 /usr/local/pkg/RESTAPI/.resources/scripts/manage.php(453): update()
#1 {main}

[KristijanL]

there is an issue with the api/v2/services/haproxy/frontend/acl the name doesn't have to be unique, as sometimes you want to match multiple stuff to the same backend

curl -X 'POST' \
  'https://domain.com/api/v2/services/haproxy/frontend/acl' \
  -H 'accept: application/json' \
  -H 'x-api-key: 12345' \
  -H 'Content-Type: application/json' \
  -d '{
  "parent_id": 1,
  "name": "app_acl",
  "expression": "host_matches",
  "value": "some-domain.com",
  "casesensitive": false,
  "not": false
}'

{
  "code": 400,
  "status": "bad request",
  "response_id": "FIELD_MUST_BE_UNIQUE",
  "message": "Field `name` must be unique. Value is in use by object with ID `6`.",
  "data": []
}

[jaredhendrickson13]

@KristijanL Some of the package's update components were broken until the latest dev build. You should be able to run the update command going forward, but you'll need to enable the 'Allow pre-releases' option in the REST API settings for the package to consider pre-release versions as an acceptable update candidate.

As for the unique constraint on names, it was added that way because it must be unique for automatic field relations on the frontend to work. I can rework some things to remove the constraint and get that working correctly. Thanks for pointing that out!