Dremio cloner unable to deploy 1 VDS (vds ln_dly_calc_fact)

[StephenLevett]

I am opening this on behalf of Pawan Teja at Fannie Mae ("Nyshadham, Pawan Teja x (Contractor)" pawan_teja_x_nyshadham@fanniemae.com).

Description:

We had earlier deployment on OCT end and they are unable to push this vds (LN_dly_calc_fact) in

Prod & Acpt environment

Attempts: Dev team has tried various scenarios to push the code , removed comments & reduced lines of sqlScript and it was unsuccessful .

below are the new attributes to the vds which could not be deployed

Line 201 :  Loan_Final_Additional_Tier_1_Cost_Of_Capital_Basis_Point_Rate
Line : 225 :   Loan_Final_Tier_2_Cost_Of_Capital_Basis_Point_Rate

Please see attached files .:

fnm_config_write_dir (1).json import_10_31_2023_13_53_20 (2).log LN_DLY_CALC_FACT_VW (1).txt

Pawan originally opened this via a ticket, but max suggested a github request.

[StephenLevett]

This is the commentary from Max to Pawan:

Looking at what they have provided, I have the following follow-up questions:

On a general note, they may want to consider splitting this 700 line SQL definition into smaller views to make it easier to debug issues

Q: Does the SQL code that they have shared for F5G/crs/ERCF_Reporting_SF/LN_DLY_CALC_FACT_VW run successfully in the UI? Are they able to create a view from that? A: Yes.

The log they provided ends at the following step: DEBUG:2023-10-31 13:53:25,950:_order_vds: processing vds dataset:F5G/crs/ERCF_Reporting_SF/LN_DLY_CALC_FACT_VW

Q: What happens after this step? A: They validate the Vds deloyed using the dremio UI .

Q: Does the program simply hang? A: No.

Q: Can they look at the access.log to see if a PUT request for the dataset has been received by Dremio? A: Pawna requires further details/assistance on this, Max.

Q: Can they share the JSON definition of the view in question from their CI/CD folder tmp/dremcicd_f5g_prod? If they have a tool like Postman, we could try running PUT /api/v3/catalog/2ab12ea4-d7a0-47f1-91e7-983cd71a0473 with the JSON as payload to see what the response by Dremio is. A: See attached: GH-issue-43.zip

[pawantejan1]

Please let us know if you hve any updates on this ticket ?

[mxmarg]

The issue is not reproducible from my end and requires deeper investigation on the Dremio cluster in question into the following areas:

• Looking at access.log to see if the PUT/POST request is received by Dremio
• Try to manually run the PUT/POST request for the object using a tool like curl or Postman
• Run the Python code locally in debug mode to step through the code where the issue occurs

I would suggest you get in contact with your account team to find a resource that is best suited for helping you on this issue