[Bug] Current Sprint and Sprint not handling nulls well

[ghost]

Is there an existing issue for this?

• [X] I have searched the existing issues

Describe the issue

If an issue is removed from a sprint, it is still considering that last sprint (in history) as the current_sprint.

1) add issue to sprint 2) remove issue from sprint 3) run query:

select
  issue_key
  , sprint
from
  <db_name>.jira.issue_enhanced
where
  issue_key = <issue_key> 

Relevant error log or model output

No response

Expected behavior

1) current_sprint should be null if sprint is null 2) sprint should be null if ''

dbt Project configurations

  # jira
  jira_database: fivetran_db
  jira_schema: jira
  jira_using_sprints: true
  jira_using_components: false
  jira_using_versions: true
  jira_include_comments: true

Package versions

  - package: fivetran/jira
    version: 0.12.2

What database are you using dbt with?

snowflake

dbt Version

> palm shell
Executing command `bash` in compose...
root@eeb4fa054be1:/app# dbt --version
Core:
  - installed: 1.4.5
  - latest:    1.4.5 - Up to date!

Additional Context

No response

Are you willing to open a PR to help address this issue?

• [X] Yes.
• [ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
• [ ] No.

[fivetran-joemarkiewicz]

Hi @murphymoulds-palmetto thanks for raising this issue! The behavior you shared where the issue being removed from a sprint should not result in the current sprint still showing as the latest sprint.

Just to confirm, when you add from sprint and remove from sprint, are you re-running the package models? I just want to make sure that this isn't caused by the need to re run the models.

[ghost]

Hi @murphymoulds-palmetto thanks for raising this issue! The behavior you shared where the issue being removed from a sprint should not result in the current sprint still showing as the latest sprint.

Just to confirm, when you add from sprint and remove from sprint, are you re-running the package models? I just want to make sure that this isn't caused by the need to re run the models.

@fivetran-joemarkiewicz - For sure, the code runs regularly (e.g., hourly, merge)

[fivetran-joemarkiewicz]

Thanks for sharing the clarification @murphymoulds-palmetto.

My team is currently scoping this out and will respond once we have a better understanding as to what may be occurring in this scenario and if there is a possible fix we can implement.

[fivetran-jamie]

hey i think i identified the bug -- we need to use a left join when joining issues with their current sprints here (with the inner join, null sprints are getting filtered out)

it'd be great if you had time to test this out @murphymoulds-palmetto!

packages:
# replace any jira package lines with the following in your packages.yml
  - git: https://github.com/fivetran/dbt_jira.git
    revision: bug/null-latest-sprint

[ghost]

hey i think i identified the bug -- we need to use a left join when joining issues with their current sprints here (with the inner join, null sprints are getting filtered out)

it'd be great if you had time to test this out @murphymoulds-palmetto!

packages:
# replace any jira package lines with the following in your packages.yml
  - git: https://github.com/fivetran/dbt_jira.git
    revision: bug/null-latest-sprint

Happiness - testing now, thank you @fivetran-jamie

[ghost]

hey i think i identified the bug -- we need to use a left join when joining issues with their current sprints here (with the inner join, null sprints are getting filtered out) it'd be great if you had time to test this out @murphymoulds-palmetto!

packages:
# replace any jira package lines with the following in your packages.yml
  - git: https://github.com/fivetran/dbt_jira.git
    revision: bug/null-latest-sprint

Happiness - testing now, thank you @fivetran-jamie

@fivetran-jamie - tested and not resolved 😿 - I went upstream (stepwise through ctes) and found the raw table time column is incorrect (see below).

select
  issue_id
  , time
  , value
from
  fivetran_db.jira.issue_multiselect_history
where
  issue_id = 23016
  and field_id = 'customfield_10020'
order by time;

ISSUE_ID	TIME	VALUE
23016	2022-11-22 19:34:48.546
23016	2023-03-01 16:49:24.213	233
23016	2023-03-13 15:57:40.674	234
23016	2023-03-13 15:57:40.674	233

I am going to do a full resync to rule out a glitch in the matrix, if it still remains incorrect, I suppose my bug ticket should go somewhere else? 😅

[fivetran-jamie]

😮 very strange -- so the time fields here should align with the screenshot from Jira that you shared, correct? Also, just to confirm, does issue_id 23016 have issue_key BIZOPS-242 or BIZOPS-67?

it sounds like it might be a connector issue, but do you possibly have multiple fields called sprint in your fivetran_db.jira.field table?

[fivetran-jamie]

@murphymoulds-palmetto friendly bump 🤠

also, we just released a new version of the package, and I'm wondering if the updates potentially resolved your issue. could you test out v0.13.0 of the package?

[ghost]

My sincere apologies @fivetran-jamie - I am going to create a ticket for someone on my team to follow up on this. I am more than happy if this is closed so that other issues are prioritized. Again, my apologies for the outlandishly late response.

[fivetran-joemarkiewicz]

No worries at all @murphymoulds-palmetto! Let us know once you are able to validate this issue is no longer persisting on your end. Following, we can either keep this issue open or close it depending on your investigation. Otherwise, we can close this ticket in time.

Thanks again!

[fivetran-joemarkiewicz]

Hi all I will be closing this ticket for the time being. Please feel free to reopen if the issue persists. Thanks!