postgres datatype too small for larger `custom_runner_definition` payloads

jeffreymlewis commented 4 years ago

Describe the bug When using postgres, one cannot POST custom_runner_definition to /v1/config over a certain size. The error message is,

$ cat error.txt | jq .stack | awk '{gsub(/\\n/,"\n")}1'
"SequelizeDatabaseError: value too long for type character varying(255)
    at Query.formatError (/usr/node_modules/sequelize/lib/dialects/postgres/query.js:366:16)
    at /usr/node_modules/sequelize/lib/dialects/postgres/query.js:72:18
    at tryCatcher (/usr/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/usr/node_modules/bluebird/js/release/promise.js:547:31)
    at Promise._settlePromise (/usr/node_modules/bluebird/js/release/promise.js:604:18)
    at Promise._settlePromise0 (/usr/node_modules/bluebird/js/release/promise.js:649:10)
    at Promise._settlePromises (/usr/node_modules/bluebird/js/release/promise.js:725:18)
    at _drainQueueStep (/usr/node_modules/bluebird/js/release/async.js:93:12)
    at _drainQueue (/usr/node_modules/bluebird/js/release/async.js:86:9)
    at Async._drainQueues (/usr/node_modules/bluebird/js/release/async.js:102:5)
    at Immediate.Async.drainQueues [as _onImmediate] (/usr/node_modules/bluebird/js/release/async.js:15:14)
    at processImmediate (internal/timers.js:456:21)"

The above error does not occur with SQLite.

The above error does not occur if the overall size of the custom_runner_definition data structure is smaller. (See example below.)

To Reproduce Steps to reproduce the behavior:

Launch predator with a PostgreSQL 11.7 backend

POST a "large" JSON payload to the /v1/config endpoint. The following is what failed for me.

{
"custom_runner_definition": {
"spec": {
  "template": {
    "spec": {
      "containers": [{
        "resources": {
          "limits": {
            "memory": "512Mi",
            "cpu": "1"
          },
          "requests": {
            "memory": "192Mi",
            "cpu": "1"
          }
        }
      }],
      "nodeSelector": {
        "lifecycle": "C5nSpot"
      },
      "tolerations": [
        {
          "key": "instances",
          "operator": "Equal",
          "value": "c5n",
          "effect": "NoSchedule"
        }
      ]
    }
  }
}
}
}

Note error in predator logs

Expected behavior I expect to be able to post custom_runner_definition of arbitrary size.

Version:

Predator: 1.4
helm chart: 1.4.0
Predator-runner: latest
Database: postgres: 11.7

Additional context Something like issue https://github.com/Zooz/predator/issues/385 might solve this problem on kubernetes.

enudler commented 4 years ago

Thanks for submitting it. I created pull request #394 and it will be merged to master today It's going be part of 1.5 released which is scheduled for next week

Regarding #385, we want to do it, will do our best get to it

enudler commented 4 years ago

Fixed and merged to master #394

Zooz / predator

postgres datatype too small for larger `custom_runner_definition` payloads #393