API not work correctly if only use new admin settings

[y-oksaku]

What happened:

I cannot access GET /pipelines/{id} with API access token when the pipeline is private and the access token has only Screwdriver admin permission (NOT have repository permission).

This problem was caused by the following. I use only new admin settings, but this is not support new admin settings.

https://github.com/screwdriver-cd/screwdriver/blob/5c040c803659893d46c2f882536617d58ac846b6/plugins/banners/index.js#L26-L43

What you expected to happen:

Authenticate correctly when using Screwdriver admin's API tokens.

How to reproduce it:

[sagar1312]

In launcher, creator is declared to be map whose key and values must be of type String map[string]string https://github.com/screwdriver-cd/launcher/blob/master/screwdriver/screwdriver.go#L200

But in domain model, new attribute id has been added which could be either String or Integer https://github.com/screwdriver-cd/data-schema/blob/master/core/scm.js#L8

We should normalize the type to String in our schema instead of bleeding SCM specific data types.

Eventually, we also need to start storing SCM user ID in user table under a new column (of Type Sequelize.String) and identify the user based on SCM user ID instead of SCM username in all the workflows (especially Screwdriver user settings and pipeline tokens created by a user)

We need to update the data-schema, to just limit the data type to String

id: Joi.string().optional()

[adong]

Want to attach the log:

2023/06/22 21:27:22 Error running launcher: Fetching Event ID 18: Parsing JSON response "{\"id\":18,\"parentEventId\":17,\"groupEventId\":17,\"causeMessage\":\"Manually started by adong\",\"commit\":{\"author\":{\"id\":15989893,\"avatar\":\"https://avatars.githubusercontent.com/u/15989893?v=4\",\"name\":\"Alan\",\"username\":\"adong\",\"url\":\"https://github.com/adong\"},\"committer\":{\"id\":15989893,\"avatar\":\"https://avatars.githubusercontent.com/u/15989893?v=4\",\"name\":\"Alan\",\"username\":\"adong\",\"url\":\"https://github.com/adong\"},\"message\":\"init\",\"url\":\"https://github.com/adong/sd-storybook-sample/commit/a073c080fdb116171361d017b85cdfd3bf44b07f\"},\"createTime\":\"2023-06-22T21:27:18.822Z\",\"creator\":{\"id\":15989893,\"avatar\":\"https://avatars.githubusercontent.com/u/15989893?v=4\",\"name\":\"Alan\",\"username\":\"adong\",\"url\":\"https://github.com/adong\"},\"meta\":{},\"pipelineId\":1,\"sha\":\"a073c080fdb116171361d017b85cdfd3bf44b07f\",\"startFrom\":\"main\",\"type\":\"pipeline\",\"workflowGraph\":{\"nodes\":[{\"name\":\"~pr\"},{\"name\":\"~commit\"},{\"name\":\"main\",\"id\":1},{\"name\":\"something\",\"id\":2}],\"edges\":[{\"src\":\"~commit\",\"dest\":\"main\"},{\"src\":\"~pr\",\"dest\":\"something\"}]},\"pr\":{}}": json: cannot unmarshal number into Go struct field Event.creator of type string

See the screenshot I got from docker-desktop app:

[sagar1312]

We also need to make SCM user id type fix for gitlab https://github.com/screwdriver-cd/scm-gitlab/pull/57/files#r1239773753

[adong]

Got the new error: "id" must be a string

API Server error:

230626/180346.007, (1687802623956:PPQF2LJ7NX:88284:lj8v06ak:10048) [request,server,error] data: ValidationError: "id" must be a string

UI:

[y-oksaku]

@adong

Got the new error: "id" must be a string

I could not reproduce it. Could you give me more details please?

[adong]

@y-oksaku Setup local development with screwdriver backend SHA commit version:

68549e68eeb2ab0715350381126fd4ca88ebbf97

rm -rf node_modules
rm package-lock.json
rm mw-data && mkdir mw-data

npm install
npm start

Then create a pipeline with the following git repo

git@github.com:adong/sd-storybook-sample.git

Run into above issue, ValidationError: "id" must be a string

[adong]

False alert, I got it working by removing npm directory ~/.npm

My docker app version is 4.18.0 (104112)

[adong]

I'm unblocked and I have a new error which requires me to clear the builds everytime I boot up my backend server

> screwdriver-api@6.0.0 start
> ./bin/server

(sequelize) Warning: SQLite does not support 'INTEGER' with UNSIGNED or ZEROFILL. Plain 'INTEGER' will be used instead.
>> Check: https://www.sqlite.org/datatype3.html
{"level":"warn","message":"Coverage plugin undefined is not supported","timestamp":"2023-06-29T17:33:31.019Z"}
{"level":"info","message":"Datastore ddl sync enabled: true","timestamp":"2023-06-29T17:33:31.145Z"}
(sequelize) Warning: SQLite does not support TEXT with options. Plain `TEXT` will be used instead.
>> Check: https://www.sqlite.org/datatype3.html
Unhandled Rejection at: Promise Promise {
  <rejected> Error
      at Database.<anonymous> (/Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query.js:185:27)
      at /Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query.js:183:50
      at new Promise (<anonymous>)
      at Query.run (/Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query.js:183:12)
      at /Users/adong/github/sd/mw/node_modules/sequelize/lib/sequelize.js:315:28
      at async SQLiteQueryInterface.changeColumn (/Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query-interface.js:43:7)
      at async events.sync (/Users/adong/github/sd/mw/node_modules/sequelize/lib/model.js:984:11)
      at async Sequelize.sync (/Users/adong/github/sd/mw/node_modules/sequelize/lib/sequelize.js:377:9) {
    name: 'SequelizeUniqueConstraintError',
    errors: [ [ValidationErrorItem] ],
    parent: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: events_backup.sha] {
      errno: 19,
      code: 'SQLITE_CONSTRAINT',
      sql: 'INSERT INTO `events_backup` SELECT `id`, `parentEventId`, `groupEventId`, `causeMessage`, `commit`, `createTime`, `creator`, `meta`, `pipelineId`, `sha`, `configPipelineSha`, `startFrom`, `type`, `workflowGraph`, `pr`, `prNum`, `baseBranch` FROM `events`;'
    },
    original: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: events_backup.sha] {
      errno: 19,
      code: 'SQLITE_CONSTRAINT',
      sql: 'INSERT INTO `events_backup` SELECT `id`, `parentEventId`, `groupEventId`, `causeMessage`, `commit`, `createTime`, `creator`, `meta`, `pipelineId`, `sha`, `configPipelineSha`, `startFrom`, `type`, `workflowGraph`, `pr`, `prNum`, `baseBranch` FROM `events`;'
    },
    fields: [ 'sha' ],
    sql: 'INSERT INTO `events_backup` SELECT `id`, `parentEventId`, `groupEventId`, `causeMessage`, `commit`, `createTime`, `creator`, `meta`, `pipelineId`, `sha`, `configPipelineSha`, `startFrom`, `type`, `workflowGraph`, `pr`, `prNum`, `baseBranch` FROM `events`;'
  }
} reason: Error
    at Database.<anonymous> (/Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query.js:185:27)
    at /Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query.js:183:50
    at new Promise (<anonymous>)
    at Query.run (/Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query.js:183:12)
    at /Users/adong/github/sd/mw/node_modules/sequelize/lib/sequelize.js:315:28
    at async SQLiteQueryInterface.changeColumn (/Users/adong/github/sd/mw/node_modules/sequelize/lib/dialects/sqlite/query-interface.js:43:7)
    at async events.sync (/Users/adong/github/sd/mw/node_modules/sequelize/lib/model.js:984:11)
    at async Sequelize.sync (/Users/adong/github/sd/mw/node_modules/sequelize/lib/sequelize.js:377:9) {
  name: 'SequelizeUniqueConstraintError',
  errors: [
    ValidationErrorItem {
      message: 'sha must be unique',
      type: 'unique violation',
      path: 'sha',
      value: null,
      origin: 'DB',
      instance: null,
      validatorKey: 'not_unique',
      validatorName: null,
      validatorArgs: []
    }
  ],
  parent: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: events_backup.sha] {
    errno: 19,
    code: 'SQLITE_CONSTRAINT',
    sql: 'INSERT INTO `events_backup` SELECT `id`, `parentEventId`, `groupEventId`, `causeMessage`, `commit`, `createTime`, `creator`, `meta`, `pipelineId`, `sha`, `configPipelineSha`, `startFrom`, `type`, `workflowGraph`, `pr`, `prNum`, `baseBranch` FROM `events`;'
  },
  original: [Error: SQLITE_CONSTRAINT: UNIQUE constraint failed: events_backup.sha] {
    errno: 19,
    code: 'SQLITE_CONSTRAINT',
    sql: 'INSERT INTO `events_backup` SELECT `id`, `parentEventId`, `groupEventId`, `causeMessage`, `commit`, `createTime`, `creator`, `meta`, `pipelineId`, `sha`, `configPipelineSha`, `startFrom`, `type`, `workflowGraph`, `pr`, `prNum`, `baseBranch` FROM `events`;'
  },
  fields: [ 'sha' ],
  sql: 'INSERT INTO `events_backup` SELECT `id`, `parentEventId`, `groupEventId`, `causeMessage`, `commit`, `createTime`, `creator`, `meta`, `pipelineId`, `sha`, `configPipelineSha`, `startFrom`, `type`, `workflowGraph`, `pr`, `prNum`, `baseBranch` FROM `events`;'
}

Would love to get some helps on this, thanks!

[adong]

The previous issue is no longer there after the latest Data schema PR got merged.