search

oracle / oci-python-sdk

Oracle Cloud Infrastructure SDK for Python
https://cloud.oracle.com/cloud-infrastructure
Other
383 stars 275 forks source link

usageapi not working for ap-mumbai region #679

Closed Gopichand-Heeddata closed 2 weeks ago

Gopichand-Heeddata commented 1 month ago

While calling the usage API call, we are getting data for ap-hyderabad region but we are encountering the below mentioned error for ap-mumbai region. ERROR:

Data import failed: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', 'opc-request-id': XXXXXXX, 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-17T09:24:13.337088+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST [https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage'](https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage%27), 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.', 'troubleshooting_tips': "See https://docs.oracle.com/iaas/Content/API/References/apierrors.htm#apierrors_500__500_internalservererror for more information about resolving this error. Also see https://docs.oracle.com/iaas/api/#/en/usage/20200107/UsageSummary/RequestSummarizedUsages for details on this operation's requirements. If you are unable to resolve this usageapi issue, please contact Oracle support and provide them this full error message."}
Traceback (most recent call last):
  File "/usr/src/app/diworker/diworker/main.py", line 230, in process_task
    self.report_import(body)
  File "/usr/src/app/diworker/diworker/main.py", line 151, in report_import
    importer.import_report()
  File "/usr/src/app/diworker/diworker/importers/base.py", line 502, in import_report
    self.data_import()
  File "/usr/src/app/diworker/diworker/importers/base.py", line 492, in data_import
    self.load_raw_data()
  File "/usr/src/app/diworker/diworker/importers/oci.py", line 83, in load_raw_data
    self._load_usage_data(currency_rate)
  File "/usr/src/app/diworker/diworker/importers/oci.py", line 105, in _load_usage_data
    compartments_details=self.cloud_adapter.discover_compartments(self.period_start)
  File "/usr/src/app/tools/cloud_adapter_handler/clouds/oci.py", line 1435, in discover_compartments
    request_summarized_usages = self.usage.request_summarized_usages(
  File "/usr/local/lib/python3.9/site-packages/oci/usage_api/usageapi_client.py", line 2448, in request_summarized_usages
    return retry_strategy.make_retrying_call(
  File "/usr/local/lib/python3.9/site-packages/oci/retry/retry.py", line 308, in make_retrying_call
    response = func_ref(*func_args, **func_kwargs)
  File "/usr/local/lib/python3.9/site-packages/oci/base_client.py", line 535, in call_api
    response = self.request(request, allow_control_chars, operation_name, api_reference_link)
  File "/usr/local/lib/python3.9/site-packages/circuitbreaker.py", line 159, in wrapper
    return call(function, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/circuitbreaker.py", line 170, in call
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/oci/base_client.py", line 724, in request
    self.raise_transient_service_error(request, response, service_code, message, operation_name, api_reference_link, target_service, request_endpoint, client_version, timestamp, deserialized_data)
  File "/usr/local/lib/python3.9/site-packages/oci/base_client.py", line 906, in raise_transient_service_error
    raise exceptions.TransientServiceError(
oci.exceptions.TransientServiceError: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', opc-request-id': XXXXXXX, 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-17T09:24:13.337088+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST [https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage'](https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage%27), 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.', 'troubleshooting_tips': "See https://docs.oracle.com/iaas/Content/API/References/apierrors.htm#apierrors_500__500_internalservererror for more information about resolving this error. Also see https://docs.oracle.com/iaas/api/#/en/usage/20200107/UsageSummary/RequestSummarizedUsages for details on this operation's requirements. If you are unable to resolve this usageapi issue, please contact Oracle support and provide them this full error message."

CODE WE HAVE WRITTEN:

import oci
from oci.retry import RetryStrategyBuilder, retry_checkers

CUSTOM_RETRY_STRATEGY = RetryStrategyBuilder().add_max_attempts(max_attempts=8) \
                    .add_total_elapsed_time(total_elapsed_time_seconds=120) \
                    .add_service_error_check(service_error_retry_config=retry_checkers.RETRYABLE_STATUSES_AND_CODES,
                                            service_error_retry_on_any_5xx=True) \

def config_creds(self):
        if self._config_creds:
            return self._config_creds
        self._config_creds = {
            'log_requests': False,
            'region': self.config['Region'],
            'tenancy': self.config['Tenancy_OCID'],
            'user': self.config['User_OCID'],
            'key_content': self.config['Private_Key'],
            'fingerprint': self.config['Finger_Print'],
            'pass_phrase': self.config['Pass_Phrase']
        }
        return self._config_creds
def get_usage(self, start_date, range_end=None, limit=None):
        """

        https://docs.oracle.com/en-us/iaas/tools/python-sdk-examples/2.126.2/usageapi/request_summarized_usages.py.html

        https://docs.oracle.com/en-us/iaas/tools/python/2.126.2/sdk_behaviors/retries.html

        """
        lis=[]
        config=self.config_creds
        end_date_later = datetime.now(timezone.utc) + timedelta(days=1)
        end_date = end_date_later.replace(second=0, microsecond=0, minute=0, hour=0, tzinfo=None)
        interval = timedelta(days=3*30)
        if not start_date:
            return lis
        current_date = start_date
        LOG.info(f"The billing import is happening for oci tenancy_id:{self.Tenancy_OCID} from {current_date} and {end_date}")
        while current_date < end_date:
            request_summary_details = RequestSummarizedUsagesDetails(
                tenant_id=self.Tenancy_OCID,
                granularity='DAILY',
                query_type='COST',
                group_by=['region', 'resourceId', 'skuName', 'service'],
                time_usage_started=self.str_from_datetime(current_date),
                time_usage_ended=self.str_from_datetime(min(end_date,current_date+interval)),
                compartment_depth=2
            )

                    .get_retry_strategy()
            request_summarized_usages = self.usage.request_summarized_usages(
                request_summary_details,
                retry_strategy=self.CUSTOM_RETRY_STRATEGY
            )
            lis.extend(request_summarized_usages.data.items)
            current_date += interval
        return lis

@adizohar, can you please provide your insights

adizohar commented 1 month ago

Usage-API can run only on home-region, if you run on different region you will get error Home Region will produce summary for all regions Please check my code at https://github.com/oracle/oci-python-sdk/tree/master/examples/showusage

Gopichand-Heeddata commented 1 month ago

No, we are running the Usage-API on home-region, Mumbai is home-region but still facing the same issue.

adizohar commented 1 month ago

Gopi, if you are running on home region and get error, please try the cost analysis on the console, if still error exist, I would open support ticket to Oracle Support from the console

Gopichand-Heeddata commented 1 month ago

@adizohar, I have tried several times, and am attaching logs. In the OCI console cost analysis working fine.

07/16/2024 12:39 AM UTC
Billing data import for cloud account OCI (748f62e4-88b2-4225-a918-c7326d00bbb5) failed: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', 'opc-request-id': 'E16EC7B29C12433789384239CA70AF7A/0AB1794750B83A8B3D4DBED00C42EE03/56BD42F7470B58A9604ADA26BAF783B3', 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-16T00:39:40.191053+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage', 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.', '"}
07/16/2024 06:39 AM UTC
Billing data import for cloud account OCI (748f62e4-88b2-4225-a918-c7326d00bbb5) failed: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', 'opc-request-id': '669EA4629FF84A939CBC5CBEC8742CE8/D10CE770F98FFEDC24459B0157516669/6040B32532E3E59FED14FB110517714A', 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-16T06:39:19.864215+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage', 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.',"}
07/16/2024 06:53 PM UTC
Billing data import for cloud account OCI (748f62e4-88b2-4225-a918-c7326d00bbb5) failed: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', 'opc-request-id': '8A6BD226282B486D983EB0C6483465B9/6A66D87C65462E60C6ACC7D4D88A5CD9/9C27EF6EFAAF12D40A1E895F94C81E52', 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-16T18:53:21.926699+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage', 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.',''}
07/17/2024 12:58 AM UTC
Billing data import for cloud account OCI (748f62e4-88b2-4225-a918-c7326d00bbb5) failed: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', 'opc-request-id': '0EB3AC14F8A2454186716B37821ADE3E/A27BEF1C202615B0558A4D286D961B3F/A0837973BFFCFAFA4408F97E62C05847', 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-17T00:58:58.839713+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage', 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.',"}
07/17/2024 06:38 AM UTC
Billing data import for cloud account OCI (748f62e4-88b2-4225-a918-c7326d00bbb5) failed: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', 'opc-request-id': '6D26B5B6EB2745A4A6DD7D47B821F4A2/89A2592209F7E2164F3417EEB7272753/17F753B4B5898D504FCDDB52328C7CEE', 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-17T06:38:27.786053+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage', 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.',}
07/17/2024 12:33 PM UTC
Billing data import for cloud account OCI (748f62e4-88b2-4225-a918-c7326d00bbb5) failed: {'target_service': 'usageapi', 'status': 500, 'code': 'InternalServerError', 'opc-request-id': 'A1CBC982246D48BDB39174590E2B3755/BF9B78C8DBBD9233E8C166EA9AF0A442/19A5776B7C4725470CDA0A325E9F4D74', 'message': 'Internal Service Error, please try again', 'operation_name': 'request_summarized_usages', 'timestamp': '2024-07-17T12:33:27.566882+00:00', 'client_version': 'Oracle-PythonSDK/2.125.0', 'request_endpoint': 'POST https://usageapi.ap-mumbai-1.oci.oraclecloud.com/20200107/usage', 'logging_tips': 'To get more info on the failing request, refer to https://docs.oracle.com/en-us/iaas/tools/python/latest/logging.html for ways to log the request/response details.'}

can you please look.

adizohar commented 1 month ago

I will leave it to the SDK team to open ticket accordingly, internal error is service issue.

Gopichand-Heeddata commented 1 month ago

Thankyou @adizohar

Gopichand-Heeddata commented 1 month ago

@adizohar any update on this issue ?

adizohar commented 1 month ago

Gopi, it is been handle by the SDK team, if this is urgent and you still get internal error, please open support ticket at Oracle Support portal.