fivetran / dbt_netsuite

Data models for Fivetran's Netsuite connector, built using dbt.
https://fivetran.github.io/dbt_netsuite/
Apache License 2.0
36 stars 34 forks source link

[Bug] Converted amount coming through as Null on Transaction_Details #70

Closed mikerenderco closed 1 year ago

mikerenderco commented 1 year ago

Is there an existing issue for this?

Describe the issue

So when we run our Job the converted_amount on the transaction_details is not coming through. It looks like its because we don't have a generalrate field coming through fivetran from netsuite as its not an option.

We have multibookaccounting set to false. How do we get the converted amount to show up? We don't use exchange rates.

Relevant error log or model output

No response

Expected behavior

converted_amount populated with values

dbt Project configurations

vars: netsuite_data_model: netsuite2 #netsuite by default netsuite_database: confirm-biosciences-379522 netsuite_schema: fivetran_netsuite netsuite2multibook_accounting_enabled: false netsuite2using_vendor_categories: false netsuite2__using_jobs: false

Package versions

packages:

What database are you using dbt with?

bigquery

dbt Version

name: 'dbt_utils' version: '0.1.0'

require-dbt-version: [">=1.3.0", "<2.0.0"]

config-version: 2

target-path: "target" clean-targets: ["target", "dbt_modules", "dbt_packages"] macro-paths: ["macros"] log-path: "logs"

Additional Context

No response

Are you willing to open a PR to help address this issue?

fivetran-joemarkiewicz commented 1 year ago

Hi @mikerenderco thanks so much for creating this issue and sharing your experience of seeing null converted amounts.

The first item in your issue description that jumps out at me is the fact that you are using Netsuite and do not have transactions in other currencies. Our initial build of the package was framed under the assumption that multiple currencies would always be used. However, that is not the case for you (and likely others per the a similar FR #71 that was opened recently).

Before moving forward with applying a solution, I wanted to understand a bit more what your ideas are around how the package should operate for users that do not need to take into account exchange rates. Mainly I am curious if you are not using exchange rates, do you care about converted amounts? Would you rather just have the standard amount, or would you still want to take into account converted amounts as there may be a day where you do start operating in multiple currencies. A few thoughts I have include the following:

Personally I am between the first and second option. I would love your thoughts before we move forward. Similarly, please let me know if non of the above sound like good options and if you have something else in mind. Thanks!

mikerenderco commented 1 year ago

Thank you for your response Joe. I like option 1 the best. The ability to turn this off and on would be invaluable. We implement this dbt job in both exchange rate and non exchange rate netsuite applications so having the ability to turn it on and off would be awesome. What we had to do in this instance is we created our own fork and modified models/netsuite2/intermediate/int_netsuite2__tran_with_converted_amounts.sql this file line 48 and 49. we simply changed the multiplication to *1 that bypasses the exchange rate. [image: image.png]

But we don't like this path because then we are off the upgrade path on the fivetran job. So if we get get a true false variable that would be ideal.

On Wed, Jun 14, 2023 at 11:40 AM Joe Markiewicz @.***> wrote:

Hi @mikerenderco https://github.com/mikerenderco thanks so much for creating this issue and sharing your experience of seeing null converted amounts.

The first item in your issue description that jumps out at me is the fact that you are using Netsuite and do not have transactions in other currencies. Our initial build of the package was framed under the assumption that multiple currencies would always be used. However, that is not the case for you (and likely others per the a similar FR #71 https://github.com/fivetran/dbt_netsuite/issues/71 that was opened recently).

Before moving forward with applying a solution, I wanted to understand a bit more what your ideas are around how the package should operate for users that do not need to take into account exchange rates. Mainly I am curious if you are not using exchange rates, do you care about converted amounts? Would you rather just have the standard amount, or would you still want to take into account converted amounts as there may be a day where you do start operating in multiple currencies. A few thoughts I have include the following:

  • We add a variable that allows users to disable currency exchange rate information and converted amounts are ignored. If this variable is used, we will instead have a new field that only highlights the non converted amounts which should pass through no issue.
  • We always include both a converted and non converted amount. You would still see a null value for the converted amounts, but the non converted amount would include your total. You can then use the non converted for the time you do not use exchange rates, then switch to converted amount when you do end up using exchange rates.
  • A bit of an expansion of the above one. Instead of the converted amount being null, we just make the converted amount be the same as the non converted amount (until multiple currencies are in use). This way you could use the converted amount no matter if you need to use exchange rates or not.

Personally I am between the first and second option. I would love your thoughts before we move forward. Similarly, please let me know if non of the above sound like good options and if you have something else in mind. Thanks!

— Reply to this email directly, view it on GitHub https://github.com/fivetran/dbt_netsuite/issues/70#issuecomment-1591797941, or unsubscribe https://github.com/notifications/unsubscribe-auth/A23MTHYIUL2ADRYMEGCD2ITXLIAQTANCNFSM6AAAAAAY4YXN5Y . You are receiving this because you were mentioned.Message ID: @.***>

-- Mike Lambeth 775.813.0944 @.***

fivetran-joemarkiewicz commented 1 year ago

Thanks for the reply! With this I think we should be able to add a variable disable all exchange rate calculations and swap the exchange rate total for the standard non exchange rate total based on user preference.

Our current sprint just kicked off so we may not get to this until next sprint, but I will do some investigating to see if I can get a WIP branch in place that entertains this enhancement to the package. I'll share more here as our team continues to work on this.

mikerenderco commented 1 year ago

Thank you - We have a solution for the moment but would greatly appreciate this change.

On Thu, Jun 15, 2023 at 9:11 AM Joe Markiewicz @.***> wrote:

Thanks for the reply! With this I think we should be able to add a variable disable all exchange rate calculations and swap the exchange rate total for the standard non exchange rate total based on user preference.

Our current sprint just kicked off so we may not get to this until next sprint, but I will do some investigating to see if I can get a WIP branch in place that entertains this enhancement to the package. I'll share more here as our team continues to work on this.

— Reply to this email directly, view it on GitHub https://github.com/fivetran/dbt_netsuite/issues/70#issuecomment-1593357158, or unsubscribe https://github.com/notifications/unsubscribe-auth/A23MTH3QFJ5RVPCKEXBWZ63XLMX3NANCNFSM6AAAAAAY4YXN5Y . You are receiving this because you were mentioned.Message ID: @.***>

-- Mike Lambeth 775.813.0944 @.***