[FEATURE] No jobs table in NS2 schema but package trying import

fivetran / dbt_netsuite_source

Data models for Fivetran's Netsuite source package, built using dbt.

https://fivetran.github.io/dbt_netsuite_source/

Apache License 2.0

14 stars 20 forks source link

[FEATURE] No jobs table in NS2 schema but package trying import #29

Open ghost opened 1 year ago

ghost commented 1 year ago

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the issue

the package is trying to import a table that does not exist in NS2 schema [JOB]

Relevant error log or model output

Database Error in model stg_netsuite2__jobs_tmp
JOB does not exist or not authorized

Expected behavior

No error

dbt Project configurations

Now is "enabled = false" to avoid error

Package versions

0.7.0

What database are you using dbt with?

snowflake

dbt Version

1.3.2

Additional Context

No response

Are you willing to open a PR to help address this issue?

[ ] Yes.
[ ] Yes, but I will need assistance and will schedule time during our office hours for guidance
[ ] No.

fivetran-joemarkiewicz commented 1 year ago

Hi @ropalloni thanks for opening this issue. From my understanding it looks like you are not using the jobs component within Netsuite2. Is that correct?

If so, this may be an easy update to simply add a variable to disable these models if you do not have the jobs component (similar to how you are enabled: false configuring them for the time being). They also currently do not flow downstream to the transform package so it shouldn't impact any downstream transformations if they are not being leveraged.

Ideally we would include these models within the disable sources section of our README. Are there any other sources that you are currently not using which our package assumes you to have?

ghost commented 1 year ago

Ok thanks. I followed your suggestion thanks. No trouble with other sources.

However, I'm actually having trouble with the (old, NS1) transaction_tax_details table. Apparently it is not in the new NS2 schema. Anyway somehow they should figure out how to link lines and taxes properly since taxes are lines as well.

This is the mapping transaction_tax_details did in NS1

Any hint on this?

fivetran-jamie commented 1 year ago

hey @ropalloni , as for your second question, i think tax details and transaction lines are linked if you have the SuiteTax feature enabled https://docs.oracle.com/en/cloud/saas/netsuite/ns-online-help/section_4283867856.html (which we don't have internally)

ghost commented 1 year ago

hi @fivetran-jamie, thanks for this. Very helpful. One last question. Is there any macro in the package to handle the shift of STATUS values from NS1 and NS2. Apparently SuiteQL provides you an internal function to solve the status, see here https://timdietrich.me/blog/netsuite-suiteql-transaction-statuses/