Current_date snowflake function cause to lag in data

[RivkiHofman]

Hi

I have a question about lag of data in our Jira dashboards. Until the hour 10 AM (Israel time), we can't see the data of current date, although that in Fivetran there is data of current date. I investigated this issue and find that the int table :int_jira__issue_calendar_spine use with filter: "where date_day <= current_date".

The function current_date bring me the real date just starting from 10 AM (Israel time), and this is the reason of lag in our data.

First I will happy to understand what is the meaning of current_date, which time is it?(I'm sorry, I know that it is snowflake function, but I don't understand according by this function brings the date) . And also I want to ask if there is an option to change it for UTC time (for example sysdate() function) or to give an option to users, to choose the desired time.

Thank a lot for your feedback, and have a nice week. Rivki

[fivetran-joemarkiewicz]

Hi @RivkiHofman thanks so much for raising this issue.

That is very interesting that the current_date() Snowflake function only brings you time starting at 10am. I believe there should be a way to convert the current_date() to UTC to then allow for this error to no longer persist (I would imagine).

Let me look a bit more into this today and I will respond here with a working branch you should be able to test out.

[fivetran-joemarkiewicz]

Hi @RivkiHofman 👋

I have been able to do some digging into this question. While I do not have a good explanation for how the current_date() function works within Snowflake, I did see that there is a useful macro within dbt_utils that allows for timestamp conversion to UTC that will work for all warehouses!

I have just pushed a hotfix to the below working branch and opened the above PR to integrate this into the next release of the Jira package. Would you be able to test the below package dep and let me know if this 10am issue is still persisting for you?

packages:
    - git: https://github.com/fivetran/dbt_jira.git
      revision: hotfix/current-date-utc
      warn-unpinned: false

Let me know how it goes, thanks!

[RivkiHofman]

Hi @fivetran-joemarkiewicz

Thank a lot for your quick support and response. We checked and it works great. I will happy to be updated when it will go live.

Thanks and have a nice day.

[fivetran-joemarkiewicz]

Thanks so much for the quick reply! I will plan to merge this into our next release/v0.6.0 branch along with a few other changes addressed by other users of the package.

[RivkiHofman]

Hi @fivetran-joemarkiewicz I jוst want to ask about the temporary package:

packages:

• git: https://github.com/fivetran/dbt_jira.git revision: hotfix/current-date-utc warn-unpinned: false

Can I use it in our production environment? Is it like as current version(0.5.1) + the change of current_date? I'm afraid to declare the new version: v0.6.0 without comprehensive tests on the new features. Is there a documentation for the new version?

Thanks and have a nice weekend.

[fivetran-joemarkiewicz]

Hey @RivkiHofman I will actually be cutting v0.6.0 later today. If you would like to test out the other features before I go live that would be great! Otherwise, I will post back here later today once the version is live.

packages:
    - git: https://github.com/fivetran/dbt_jira.git
      revision: release/v0.6.0
      warn-unpinned: false

[RivkiHofman]

Hi @fivetran-joemarkiewicz

I understand that the v0.6.0 will go live today, Good luck 👍 . My question is if I can stay in the meantime with the temporary branch(revision: hotfix/current-date-utc), or maybe this branch will be deleted or changed?

Thanks.

[fivetran-joemarkiewicz]

@RivkiHofman you can keep the temporary branch in the meantime! I will be sure not to delete it 😄

[fivetran-joemarkiewicz]

Hi @RivkiHofman I wanted to let you know that the latest version of the dbt_jira package v0.6.0 is now live on the dbt hub!

Be sure to read the Release Notes to be aware of the changes that are included in the latest release along with the change from this issue, and to run a dbt run --full-refresh after upgrading in order to capture changes in the incremental models within this package.

Thanks again for helping raise this issue and test out the working branch before release! We are extremely grateful for your contributions to this package and for helping make this package better for all it's users. 🎉 ❤️ Please feel free to open a new issue if you experience any other bugs or a feature request if you think of a new enhancement to that the package would benefit from integrating.