search

oracle-samples / usage-reports-to-adw

Usage2adw is a tool which uses the Python SDK to extract the usage and cost reports from your tenant and load it to Oracle Autonomous Database. (DbaaS can be used as well) Authentication to OCI by User or instance principals.
Other
15 stars 9 forks source link

Error processing computation. #59

Closed VARUNADITYAV closed 1 month ago

VARUNADITYAV commented 1 month ago

When I click run application "OCI Usage and Cost Report," I get the below error. please help. Error processing computation. Technical Info (only available for developers) is_internal_error: true apex_error_code: APEX.COMPUTATION.UNHANDLED_ERROR ora_sqlcode: -942 ora_sqlerrm: ORA-00942: table or view "WKSP_CFBFCC2Z2ZFM"."OCI_RESOURCES" does not exist ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 317 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 238 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1618 ORA-06512: at "SYS.WWV_DBMS_SQL_APEX_240100", line 731 ORA-06512: at "SYS.DBMS_SYS_SQL", line 1579 ORA-06512: at "SYS.WWV_DBMS_SQL_APEX_240100", line 719 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 2014 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1511 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1634 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 109 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 306 component.type: APEX_APPLICATION_PAGES component.id: 100000009999 component.name: OCI Usage and Cost Report - Sign In error_backtrace: ORA-06512: at "SYS.DBMS_SYS_SQL", line 1579 ORA-06512: at "SYS.WWV_DBMS_SQL_APEX_240100", line 719 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 317 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 238 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1618 ORA-06512: at "SYS.WWV_DBMS_SQL_APEX_240100", line 731 ORA-06512: at "SYS.DBMS_SYS_SQL", line 1579 ORA-06512: at "SYS.WWV_DBMS_SQL_APEX_240100", line 719 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 2014 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1511 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1634 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 109 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 306 error_statement: select count(*) from OCI_RESOURCES Developer Toolbar

adizohar commented 1 month ago

H Varun,

Your setup on the VM probably failed, can you please login to the compute VM and post the boot.log error ?

VARUNADITYAV commented 1 month ago

Hello, i can't find boot.log in OL8, i have uploaded dmesg and /var/log/messages dmesg.log messages.log Thanks

adizohar commented 1 month ago

The boot.log was created by the application at /home/opc Did you install it using the terraform ? or manually ?

adizohar commented 1 month ago

if you install it manually, you have setup.log, please attach it Did you follow every step ? and no failure ? btw, the error you got initially happened when usage2adw.py failed to execute.

VARUNADITYAV commented 1 month ago
  1. Execute the python script - usage2adw.py or setup_usage2adw.sh --> SKIPPED as i have done 14 to 18 manually, attached setup.log setup.log

if you want to skip 14 to 18, execute the script /home/opc/usage_reports_to_adw/usage2adw_setup.sh -setup_app

Load the data:

/home/opc/usage_reports_to_adw/shell_scripts/run_multi_daily_usage2adw.sh

adizohar commented 1 month ago

setup.log didn't capture the errors because it was executed manually Can you run again the /home/opc/usage_reports_to_adw/usage2adw_setup.sh -setup_app And post the full screen messages or the last error

adizohar commented 1 month ago

You have issue with below policy, please add it and try again Allow dynamic-group UsageDownloadGroup to read secret-bundles in compartment {APPCOMP} The APPCOMP need to be the secret compartment

adizohar commented 1 month ago

Looks like you don’t have access to the usage files or you are running on a child tenant, please add the correct policies from the documentation

VARUNADITYAV commented 1 month ago

I don't have a child tenancy. I have followed the steps 2 and 3 for creating dynamic group and policy, what is the meaning of access to the usage files?

VARUNADITYAV commented 1 month ago

Can you share the architecture diagram, how this VM and ADB connect, and overall DFD as well? it is looking for object storage and says that BucketNotFound

adizohar commented 1 month ago

The VM download the cost and usage files from object storage biling namespace, you will need to have permission to do it Please check the documentation - https://docs.oracle.com/en-us/iaas/Content/Billing/Concepts/costusagereportsoverview.htm

Based on the error you got, the policy is and dynamic group not set properly, can you please share printscreen of the dynamic group and the policy ? if you are using different IAM Domains make sure you specify the domain name before group name

VARUNADITYAV commented 1 month ago

Can we have a quick call in MS teams? it would be easy for me to share the screen

adizohar commented 1 month ago

Please fix the policy to include the proper tenancy of the biling namespace (This should not be changed to your tenant id)

define tenancy usage-report as ocid1.tenancy.oc1..aaaaaaaaned4fkpkisbwjlr56u7cj63lf3wffbilvqknstgtvzub7vhqkggq

Happy to do Teams, but I have back to back meetings today, let try to solve it here, if not we can schedule meeting for tomorrow.

adizohar commented 1 month ago

I noticed you didn't add limit to the usage download date, therefore, it will download everything which can take very long If you want to limit to 2024-01 please update /home/opc/usage_reports_to_adw/config.user Once fixed please run: /home/opc/shell_scripts/run_multi_daily_usage2adw.sh

VARUNADITYAV commented 1 month ago

updated the policy and EXTRACT_DATE=2024-01, getting the below error now.

  1. Enable APEX Application. Internal LOG=/home/opc/usage_reports_to_adw/log/enable_apex_application_20241021_154708.log ORA-20001: Package variable g_security_group_id must be set. ORA-20001: Package variable g_security_group_id must be set. ORA-01403: no data found ORA-20001: Package variable g_security_group_id must be set. Error enabling APEX Application, please check log /home/opc/usage_reports_to_adw/log/enable_apex_application_20241021_154708.log, aborting. [opc@usagevm usage_reports_to_adw]$
adizohar commented 1 month ago

You setup the apex manually, therefore you won't be able to complete the shell apex setup, you can ignore this error, please run

/home/opc/shell_scripts/run_multi_daily_usage2adw.sh
VARUNADITYAV commented 1 month ago

./run_multi_daily_usage2adw.sh ran successfully, Press on the application "Usage and Cost Report", getting the same error

Error processing computation. Technical Info (only available for developers) is_internal_error: true apex_error_code: APEX.COMPUTATION.UNHANDLED_ERROR ora_sqlcode: -942

ora_sqlerrm: ORA-00942: table or view "WKSP_CFBFCC2Z2ZFM"."OCI_RESOURCES" does not exist ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 317 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 238 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1618 ORA-06512: at "SYS.WWV_DBMS_SQL_APEX_240100", line 731 ORA-06512: at "SYS.DBMS_SYS_SQL", line 1579 ORA-06512: at "SYS.WWV_DBMS_SQL_APEX_240100", line 719 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 2014 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1511 ORA-06512: at "APEX_240100.WWV_FLOW_DYNAMIC_EXEC", line 1634 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 109 ORA-06512: at "APEX_240100.WWV_FLOW_COMPUTATION", line 306

Merging statistics into OCI_COST_STATS... Merge Completed, 8980 rows merged, Process Time 00:00:01

Merging statistics into OCI_PRICE_LIST... Merge Completed, 45 rows merged, Process Time 00:00:01

Update OCI_PRICE_LIST for empty currency... Merge Completed, 6 rows merged, Process Time 00:00:00

Merging statistics into OCI_COST_REFERENCE ... Merging statistics from OCI_COST... Merge Completed, 98 rows merged, Process Time 00:00:04 Handling Tag Special Key 'Oracle-Tags.CreatedBy' Merge Completed, 1 rows merged, Process Time 00:00:00

Check OCI_TENANT for new TENANT_ID... Update Tenant Display Name Table, 1 rows inserted, Process Time 00:00:00 Update Current Tenant Name, 1 rows Updated, Process Time 00:00:00

Merging Public Rates into OCI_RATE_CARD... Update Completed, 37 rows updated., Process Time 00:00:21

Completed at 2024-10-21 16:07:12 Finish Mon Oct 21 16:07:12 GMT 2024 - local Completed at Mon Oct 21 16:07:12 GMT 2024..

adizohar commented 1 month ago

The only reason your APEX does not work, the APEX does not see the usage tables You probably created the workspace pointing to a different owner, you need to make sure it points to the user that has the tables, from the top, APEX is pointing to WKSP_CFBFCC2Z2ZFM Where are the tables located ? The default is USAGE user If you need help how to point APEX to the correct schema, you can check the APEX Documentation

VARUNADITYAV commented 1 month ago

I have dropped the workspace and re-ran the scripts. all working fine. thanks

adizohar commented 1 month ago

Perfect thank you