search

europeana / assembla-to-jira

DEPRECATED - Tool for migrating data from Assembla to Jira.
European Union Public License 1.1
3 stars 4 forks source link

readme

This project is not maintained anymore

Assembla-to-Jira

A collection of advanced tooling for a complete data migration from Assembla to JIRA.

Introduction

Have you ever wanted to use JIRA instead of Assembla, but were afraid that the switch to Jira was too risky because you already have so much important data in Assembla?

JIRA does offer a number of standard add-ons to make migration easier, but unfortunately it does not offer any tools for migrating to Assembla.

However, you are now in luck! By using these Assembla-to-Jira migration tools, it should be very easy to export all of the relevant Assembla data and import most (if not all) of it into a Jira project.

Usage is made of the Assembla API and the JIRA API in order to hook up both environments and make the data transformations.

Most of the actions can be done automatically via a pipeline of scripts, after proper configuration of the required parameters in the .env file.

However, there are a few manual actions required since the Jira API does not support all possible actions, but these are minimal. It is important NOT to skip these manual changes, as the successful migration depends on them.

It is best to start with a fresh installation, e.g. one in which the desired project has not yet been created. Otherwise, unexpected problems may occur.

If you need help, please check out the support section.

Installation

The toolset has been written with the Ruby programming language. In order to be able to use it, you will have to have downloaded and installed the following on your computer:

Once this has been done, you can checkout and install the toolset from the github repository.

$ git clone https://github.com/kgish/assembla-to-jira.git
$ cd assembla-to-jira
$ gem install bundler
$ bundle install

At this point everything should be ready to use as explained in the following sections.

Pipeline Steps

The complete migration from start to finish consists of a series of scripts which are to be executed in order.

Like a pipeline, each script processes data and generates a dump file to store the intermediate results which in turn are used as input for the following script.

The reason for doing this that if something goes wrong you do not lose everything and can restart from the previous step.

Each step will generate a log of the results in the form of a csv file for reference purposes, e.g. detecting which requests failed and why. For example, importing tickets will create the data/jira/jira-tickets.csv file.

Assembla export

  1. Space (space_tools, users, user roles, tags, milestones, ticket statuses, ticket custom fields, documents, wiki pages and tickets)
  2. Tickets (comments, attachments, tags, associatations)
  3. Report users
  4. Report tickets

Jira import

  1. Create project (and board)
  2. Create issue link types
  3. Get issue types
  4. Get issue priorities
  5. Get issue resolutions
  6. Get user roles
  7. Get issue statuses
  8. Get projects
  9. Import users
  10. Download ticket attachments
  11. Import tickets
  12. Import ticket comments
  13. Import ticket attachments
  14. Update ticket status (resolutions)
  15. Update ticket associations
  16. Update ticket watchers

Scrum/Kanban board

  1. Create sprints
  2. Update board

Preparations

You will need to go to to the Jira website and login as admin.

Define the project .env file as ASSEMBLA_SPACE=space-name.

Create the following new issue type:

The issue type spike or bug will be defined for any tickets whose summary starts with Spike: or Bug:.

Additionally, any tickets whose summary starts with Epic : will be defined as issue type epic (already part of the default Jira ticket types on project creation).

You will also need to configure the issue type scheme for the project like this:

Custom fields

After the project is created (see below), you will need to define manually the following custom fields (text field read-only):

and assign them to the following screens:

Otherwise the ticket import will fail with the error message Field 'field-name' cannot be set. It is not on the appropriate screen, or unknown.

Additionally the following already existing custom fields need to be assigned the the same screens:

On the View Field Configuration Page ensure the same for:

The same applies to the Configure Screen Page for the following additional (default) fields:

Environment

An example configuration file .env.example is provided for you to define a number evironment parameters which affect the behavior.

# --- General settings --- #
TICKETS_CREATED_ON=YYYY-MM-DD
DEBUG=false

# --- Assembla settings --- #
ASSEMBLA_API_HOST=https://api.assembla.com/v1
ASSEMBLA_API_KEY=api-key
ASSEMBLA_API_SECRET=api-secret
ASSEMBLA_URL_TICKETS=https://app.assembla.com/spaces/space-name/tickets
ASSEMBLA_SPACE=space-name
ASSEMBLA_SKIP_ASSOCIATIONS=parent,child,story,subtask
# Ticket types extracted from ticket summary, e.g. starting with 'Spike: '
ASSEMBLA_TYPES_IN_SUMMARY=epic,spike,bug

# --- Jira API settings --- #
JIRA_API_HOST=https://jira.example.org/rest/api/2
JIRA_API_PROJECT_NAME=Project Name
# Project type must be scrum (default) or kanban
JIRA_API_PROJECT_TYPE=scrum
JIRA_API_ADMIN_USERNAME=john.doe
JIRA_API_ADMIN_PASSWORD=secret
JIRA_API_UNKNOWN_USER=unknown.user
JIRA_API_IMAGES_THUMBNAIL=description:false,comments:true
# Status mappings (from:to, if :to missing then same as from)
JIRA_API_STATUSES=New:To Do,In Progress,Ready for Testing,Done,Invalid:Done

# --- Jira Agile settings --- #
JIRA_AGILE_HOST=https://jira.example.org/rest/agile/1.0
JIRA_BOARD_NAME=Name of Scrum Board

By using the filter TICKETS_CREATED_ON you can limited the tickets to those that were created on or after the date indicated. So for example:

TICKETS_CREATED_ON=2017-06-01

would only include those tickets created on or after the first of June in the year 2017.

$ cp .env.example .env

Export data from Assembla

You can run the export in a number of stages, output files being generated at each point in the process.

The output files are located in the directory data/assembla/:space/:project as follows:

$ ruby 01-assembla_export_space.rb # => space_tools.csv, users.csv, user_roles.csv tags.csv \
    milestones.csv, tickets-statuses.csv, tickets-custom-fields.csv, documents.csv, \
    wiki_pages.csv, tickets.csv
$ ruby 02-assembla_export_tickets.rb [type] # => ticket-comments.csv, ticket-attachments.csv, \
    ticket-tags.csv, ticket-associations.csv
$ ruby 03-assembla_report_users.rb # => report-users.csv
$ ruby 04-assembla_report_tickets.rb # => report-tickets.csv

Notice that executing 02-assembla_export_tickets.rb can be very time consuming, so you might want to break it up into smaller chunks by passing the optional type (comments, attachments, tags or associations) as the first argument.

$ ruby 02-assembla_export_tickets.rb comments # => ticket-comments.csv
$ ruby 02-assembla_export_tickets.rb attachments # => ticket-attachments.csv
$ ruby 02-assembla_export_tickets.rb tags # => ticket-tags.csv
$ ruby 02-assembla_export_tickets.rb associations # => ticket-associations.csv

This allows you to recover better to the previous step in case of failure, for example near the end where you would lose all the data in the dump files.

Import data into Jira

You can run the import in a number of stages, output files being generated at each point in the process.

Create project (and board)

POST /rest/api/2/project
{
  key: project_key,
  name: project_name,
  projectTypeKey: 'software',
  description: project_description,
  projectTemplateKey: "com.pyxis.greenhopper.jira:gh-#{type}-template",
  lead: username
}

where '#{type}' must be either 'scrum' or 'kanban'.

$ ruby 05-jira_create_project.rb

Depending on the value of JIRA_API_PROJECT_TYPE in the .env file, a scrum or kanban board will be created as well with board name {projectKey} board.

The projectKey is usually just the abbreviation of the project name in all capitals. Here is an example of a project with key ECT:

Create issue link types

$ ruby 07-jira_create_issuelink_types.rb # => data/jira/jira-issuelink-types.csv

Get general information

$ ruby 07-jira_get_issue_types.rb # => data/jira/jira-issue-types.csv
$ ruby 08-jira_get_priorities.rb  # => data/jira/jira-priorities.csv
$ ruby 09-jira_get_resolutions.rb # => data/jira/jira-resolutions.csv
$ ruby 10-jira_get_roles.rb       # => data/jira/jira-roles.csv
$ ruby 11-jira_get_statuses.rb    # => data/jira/jira-statuses.csv
$ ruby 12-jira_get_projects.rb    # => data/jira/jira-projects.csv

Import users

POST /rest/api/2/user
{
  name: user['login'],
  password: user['login'],
  emailAddress: user['email'],
  displayName: user['name']
}

Read in the Assembla user file data/:space/:project/users.csv and create the Jira users if they do not already exist.

$ ruby 13-jira_import_users.rb # => data/jira/jira-users.csv

The following user:

as defined in the .env file as JIRA_API_UNKNOWN_USER.

Download attachments

Before the attachments can be imported, they must first be downloaded to a local directory after which they can be imported into Jira.

This is accomplished by executing the following command:

$ ruby 14-jira_download_attachments.rb # => data/jira/jira-attachments-download.csv

The downloaded attachments are placed in the data/jira/attachments directory with the same filename, and the meta information is logged to the file data/jira/jira-attachments-download.csv containing the following columns:

created_at|assembla_ticket_id|jira_ticket_id|filename|content_type

which is used to import the attachments into Jira in the following section. A check is made if the file already exists in order to avoid name collisions.

Note that in Jira images are treated as attachments and can be accessed that way via [[image:IMAGE|NAME]].

Important: this step needs to be done before importing tickets (next section) in order that the markdown for embedded attachment (images) will work correctly.

Import tickets

POST /rest/api/2/issue
{
  create: {},
  fields: {
    project: { id: project_id },
    summary: summary,
    issuetype: { id: issue_type[:id] },
    assignee: { name: assignee_name },
    reporter: { name: reporter_name },
    priority: { name: priority_name },
    labels: labels,
    description: description,
    ...
    customfield_assembla_id: ticket_number,
    customfield_assembla_theme: theme_name,
    customfield_assembla_status: status_name,
    customfield_assembla_milestone: milestone[:name],
    customfield_rank: story_rank,

    customfield_assembla_reporter: UNKNOWN_USER, # if reporter is missing
    customfield_assembla_assignee: '',           # if assignee cannot be assigned issues
    customfield_epic_name: EPIC_NAME,            # if issue type is epic
    parent: { id: parent_id },                   # if issue type is sub-task
    ...
  }
}

Now you are ready to import all of the tickets. Execute the following command:

$ ruby 15-jira_import_tickets.rb # => data/jira/jira-tickets.csv

Results are saved in the output file data/jira/jira-tickets-all.csv with the following columns:

jira_ticket_id|jira_ticket_key|project_id|summary|issue_type_id|issue_type_name|assignee_name| \
reporter_name|priority_name|status_name|labels|description|assembla_ticket_id|assembla_ticket_number| \
theme_name|milestone_name|story_rank

For the individual issue types data/jira/jira-tickets-{issue-type}.csv where issue-type is: bug, epic, spike, story, task or sub-task.

Import comments

POST /rest/api/2/issue/{issueIdOrKey}/comment
{
  body: "comments go here..."
}

Now you are ready to import all of the comments. Execute the following command:

$ ruby 16-jira_import_comments.rb # => data/jira/jira-comments.csv

Results are saved in the output file data/jira/jira-comments.csv with the following columns:

jira_comment_id|jira_ticket_id|assembla_comment_id|assembla_ticket_id|user_login|body

Import attachments

curl -D- -u admin:admin -X POST -H "X-Atlassian-Token: no-check" -F "file=@myfile.txt" api/2/issue/{issueIdOrKey}/attachments

Now you are ready to import all of the attachments that were downloaded earlier. Execute the following command:

$ ruby 17-jira_import_attachments.rb # => data/jira/jira-attachments-import.csv

Update ticket status

Now you are ready to update the Jira tickets in line with the original Assembla state. Execute the following command:

$ ruby 18-jira_update_status.rb # => data/jira/jira-update-status.csv

Update ticket associations

For the default Assembla associations the relationship names are:

# Name Ticket2 Ticket1
0 Parent is parent of is child of
1 Child is child of is parent of
2 Related related to
3 Duplicate is duplication of
4 Sibling is sibling of
5 Story is story is subtask of
6 Subtask is subtask of is story
7 Dependent depends on
8 Block blocks 
0 - Parent (ticket2 is parent of ticket1 and ticket1 is child of ticket2)
1 - Child  (ticket2 is child of ticket1 and ticket2 is parent of ticket1)
2 - Related (ticket2 is related to ticket1)
3 - Duplicate (ticket2 is duplication of ticket1)
4 - Sibling (ticket2 is sibling of ticket1)
5 - Story (ticket2 is story and ticket1 is subtask of the story)
6 - Subtask (ticket2 is subtask of a story and ticket1 is the story)
7 - Dependent (ticket2 depends on ticket1)
8 - Block (ticket2 blocks ticket1)

For the default Jira issue link types we have:

Name Inward Outward
Blocks is blocked by blocks
Cloners is cloned by clones
Duplicate is duplicated by duplicates
Relates relates to relates to 
POST /rest/api/2/issueLink
{
  type: {
    name: name
  },
  inwardIssue: {
    id: ticket1_id
  },
  outwardIssue: {
    id: ticket2_id
  }
}

However, since Jira already takes care of a number of issue links during issue creation (story, subtask, etc), we should disable them in the .env configuration file like this:

ASSEMBLA_SKIP_ASSOCIATIONS=parent,child,story,subtask

If for some reason you do not want to do this, simply comment out the line, or if you prefer to skip other Assembla association just edit the line.

Now you are ready to update the Jira tickets to reflect the original Assembla associations. Execute the following command:

$ ruby 19-jira_update_association.rb # => data/jira/jira-update-associations.csv

Update ticket watchers

POST /rest/api/2/issue/{issueIdOrKey}/watchers
'"username"'

Now you are ready to convert the Assembla followers list to the Jira issue watchers list. Execute the following command:

$ ruby 20-jira_update_watchers.rb # => data/jira/jira-update-watchers.csv

Scrum Board

You are now ready to setup the scrum board, create sprints, and assign issues to the correct sprints as well as the backlog. In the .env file, take notice of the following values:

JIRA_API_PROJECT_NAME=Project Name
JIRA_API_PROJECT_TYPE=scrum
JIRA_BOARD_NAME=name:Scrum Board Name

These will be used as placeholder values below.

Create sprints

When the scrum board was created with the project, all issues are assigned to the project are automatically put in the backlog.

Now you are ready to setup the sprints by executing the following command:

$ ruby 21-jira_create_sprints.rb # => data/jira/jira-create-sprints.csv

The issues are redistibuted to the sprints they belong to and the most recent sprint is set as the active sprint.

Update board

The final step after the board and sprints have been created is to copy the Assembla cardwall columns (ticket statuses) to the Jira board and to order the issues by rank as they were in Assembla.

In order to achieve this, execute the following command:

$ ruby 22-jira_update_board.rb # => data/jira/jira-update-board.csv

Create statuses

JIRA_API_STATUSES=New:To Do,In Progress,Blocked,Testable,Ready for Acceptance, \
    In Acceptance Testing,Ready for Deploy,Done,Invalid:Done

Create workflow

Ticket field conversions

Most of the ticket fields are converted from Assembla to Jira via a one-to-one mapping and are indicated as bold below.

Assembla ticket fields:

Jira issue fields:

Default

Custom

Associations

The Assembly associations are converted into Jira issue links.

0 - Parent (ticket2 is parent of ticket1 and ticket1 is child of ticket2)
1 - Child  (ticket2 is child of ticket1 and ticket2 is parent of ticket1)
2 - Related (ticket2 is related to ticket1)
3 - Duplicate (ticket2 is duplication of ticket1)
4 - Sibling (ticket2 is sibling of ticket1)
5 - Story (ticket2 is story and ticket1 is subtask of the story)
6 - Subtask (ticket2 is subtask of a story and ticket1 is the story)
7 - Dependent (ticket2 depends on ticket1)
8 - Block (ticket2 blocks ticket1)

See: http://api-docs.assembla.cc/content/ref/ticket_associations_fields.html

Statuses and states

The Assembla ticket statuses are: new, in progress, blocked, testable, ready for acceptance, in acceptance testing, ready for deploy, done and invalid.

An Assembla ticket can have two states: 0 - closed (done or invalid) and 1 - open (all others).

The Jira statuses are: todo and done. On creation, all Jira tickets are set initially to todo by default.

The possible transitions for this initial todo state are start progress => in progress and done => done.

During the migration, Assembla tickets that are marked as closed will result in Jira issues marked as done with resolution set to fixed for Assembla ticket status done and won't fix for Assembla ticket status invalid.

For Assembla tickets marked as in progress the imported Jira issue will be set to in progress.

IMPORTANT: all the other statuses will be ignored unless the administrator modifies the workflow for the given Jira project to include them explicitly.

The names of these newly defined transitions MUST be the same as the Assembla status names in order for the status migration to work properly.

Story points

The story_importance field for Assembla tickets is ONLY used for story type Jira issues.

Components

For the time being components have not yet been implemented.

According to the Assembla API Documentation: Ticket components API is deprecated. Please use custom fields.

Markdown

The Assembla markdown syntax is different from JIRA Markdown. Therefore, the certain markdown notations will need to be translated as follows.

Equivalent (no changes required)

h1. TITLE
h2. TITLE
*bold*
_italic_
Bullet list
Numbered list
Numbered - Bullet list

Ignore (will be ignored and passed through unchanged)

Wiki links
[[ticket:NUMBER]]
[[user:NAME]]
[[user:NAME|TEXT]]

Reformat (will be reformatted into Jira markdown)

[[image:IMAGE]] => !name(IMAGE)|thumbnail!
[[image:IMAGE|text]] => !name(IMAGE)|thumbnail!
@login => [~login]
@inline code@ => {{inline code}} (monospaced)
[[url:URL|TEXT]] => [TEXT|URL]
[[url:URL]] => [URL|URL]
#Assembla-ticket-number => Issue-Key

Code blocks

In Assembla a block of code looks like this:

<pre><code>
code-snippet
</code></pre>

which will be transformed into Jira format like this:

{code:java}
code-snippet
{code}

Note that the images will have original or thumbnail sizes depending on the value of JIRA_API_IMAGES_THUMBNAIL in the .env file.

So for example:

JIRA_API_IMAGES_THUMBNAIL=description:false,comments:true

would insert original size images in the Jira issue description and thumbnail images in the Jira issue comments (which happens to be the default).

For the content available in the ticket summaries, descriptions and comments we have:

[summary, description, comments].each do |content|
  content = reformat_markdown(content, list_of_logins)
end

where reformat_markdown will do the following global substitutions:

gsub(/\[\[url:(.*)\|(.*)\]\]/, '[\2|\1]')
gsub(/\[\[url:(.*)\]\]/, '[\1|\1]')
gsub(/@([^@]*)@/, '{\1}')
gsub(/@([a-z.-_]*)/i) { |name| markdown_name(name, list_of_logins) }.
gsub(/\[\[image:(.*)(\|(.*))?\]\]/i) { |image| markdown_image(image, list_of_images, content_type) }

Trouble-shooting

To do

With such a complicated tool, there will always be some loose ends and/or additional work to be done at a later time. Hopefully in the not so distant future, I'll have some time to tackle one or more of the following items:

References

License

Licensed under the EUPL V.1.1.

For full details, see LICENSE.md.

Support

If you require assistance with the migration or need some functionality that is not yet present in the latest version, then I can certainly help you. Feel free to contact me!

Author

Kiffin Gish

kiffin.gish@planet.nl

http://gishtech.com