Import issues error if using -e flag

trae410 commented 3 years ago

Can export the issues from a repo into the CSV generated file as expected but when importing to another repo an error status 422 is thrown.

Operation: githubCsvTools -e then githubCsvTools [path to file]

In version 3.1.2

Error message in console:

RequestError [HttpError]: Validation Failed: {"value":"\"url\":\"https://api.github.com/repos/.../labels/UX\"","resource":"Label","field":"name","code":"invalid"}

at ...\npm\node_modules\github-csv-tools\node_modules\@octokit\request\dist-node\index.js:86:21
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async Job.doExecute (...\npm\node_modules\github-csv-tools\node_modules\bottleneck\light.js:405:18) {
  status: 422,
  ...

Node version: v12.18.3 Windows system32

trae410 commented 3 years ago

Related to #48 ?

gavinr commented 3 years ago

To import issues, the CSV must have specific column names. See https://github.com/gavinr/github-csv-tools#to-import-issues or https://github.com/gavinr/github-csv-tools/tree/master/test. Note that this in not necessarily the exact format the the "export" process exports.

Perhaps we could better document this. PRs to the README are welcome

trae410 commented 3 years ago

Okay, what I think I'm seeing though is that custom labels in the exported csv through the -e flag basically break the import command.

I checked out the differences in the issueInfo object that is generated by the the createIssue function and saw the following:

When githubCsvToolswith no flags is ran then run the import command into a new repo with no issues and nothing but default labels in the repo:

{
  owner: 'me',
  repo: 'test-issues-4',
  title: 'sample issue',
  body: 'some text',
  labels: [ 'customLabel']
}

When githubCsvToolswith the -e flag is ran then run the import command into a new repo with no issues and nothing but default labels in the repo:

{
  owner: 'me',
  repo: 'test-issues-4',
  title: 'sample issue',
  body: 'some text',
  labels: [
    '[{"id":xyz',
    '"node_id":"xyz"',
    '"url":"https://api.github.com/repos/.../labels/customLabel"',
    '"name":"customLabel"',
    '"color":"5E63E8"',
    '"default":false',
    '"description":""}']`
  ]
}

I'm guessing that the label arrays can only handle string values... In the octoKit docs it says "Note that Milestones and Labels need to exist in the repository before creating the issue. Refer to the Milestones and Labels sections for more details." https://octokitnet.readthedocs.io/en/latest/issues/

I am not farmilliar with octoKit but maybe we could either:

[ ] strip the labels properties and make the labels property into an array of strings or
[ ] add a function that maps through all the custom labels in the csv and creates new labels in the repo first then continue on with creating the issues

trae410 commented 3 years ago

@gavinr I made a fork of this that will allow importing of comments from a csv by breaking down the csv nested properties if the index is not found (see snip of import.js below) and added a transfer command that transfers a copy or updates the issues if they already exist in the destination repo. (and its comments if using the -c flag) I didn't test using the -e or -a flags. This is definitely harder that I originally thought since octokit doesn't seem to support just having one big issue POST request that includes labels, comments etc.

the fork

import.js importFile

     ...
        // get indexes of the fields we need
        var indexes = {
          title: cols.indexOf("title"),
          number: cols.indexOf("number"),
          body: cols.indexOf("body"),
          labels: cols.indexOf("labels"),
          milestone: cols.indexOf("milestone"),
          assignee: cols.indexOf("assignee"),
          state: cols.indexOf("state"),
        };
        // for every comment on an issue, a seperate issue was created with only the comment data different
        const commentUserIndex = cols.indexOf("comment.user");
        const commentBodyIndex = cols.indexOf("comment.body");

        if (indexes.title === -1) {
          var newTitleIndex = cols.indexOf("issue.title");
          // get the value in case the nested properties formatted as item.item eg: "issue.title",
          if (newTitleIndex) {
            // change all indexes to find with prefix of "issue."
            for (key in indexes) {
              indexes[key] = cols.indexOf(`issue.${key}`);
            }
          } else ...
        }

gavinr / github-csv-tools

Import issues error if using -e flag #55