SAP-samples / data-warehouse-cloud-connectors

Connecting third party clients to SAP Data Warehouse Cloud.
Apache License 2.0
11 stars 4 forks source link

Adding Views directly via Advanced Query Editor - Expensive Result Sets #22

Closed ghost closed 1 month ago

ghost commented 8 months ago

Adding a datasource via the Wizzard approach will always show the data preview. This data preview request is a full request (all columns, no filter) - just a Top 1000 is added. There are situations, where this request times out or where the result set is too large.

In this situation, you could add the data source directly in the Advanced Query Editor of PowerBI: Find below a sample where we reduced the Top condition to 100 (good for relational sources, typically not that effective on analytical sources) and selected two specific columns (in the example the columns "HOST" and "VALUE"):

let
     Source = SapDataWarehouseCloudConnector.Contents(
         "<tenant name>",
         "relational", 
         "<Space Name>",
         "<View Name>",
         "<Parameters> or null"),

     M_HOST_INFORMATION_S_table =
        Source{[Name="M_HOST_INFORMATION_S",Signature="table"]}[Data],

     #"TOP" = Table.FirstN(M_HOST_INFORMATION_S_table,100),

     #"COLUMNS" = Table.SelectColumns(#"TOP",{"HOST", "VALUE"})
in
     #"COLUMNS"

This snipped has been created using the following steps:

Note that PowerBI offers additional relevant options like setting filters on columns.

sknoepfler commented 2 months ago

@OlafFischer : I followed your exact steps (Top 10 and 2 columns) with a problematic analytic model with an input parameter and got the following error:

DataSource.Error: Microsoft.Mashup.Engine1.Library.Resources.HttpResource: Request failed: 
OData Version: 3 and 4, Error: The remote server returned an error: (500) Internal Server Error. (Unable to process query: Error: Debug [IOLayer]: (#99999) e4e2117f-9e7a-4714-770b-9b0a52d2534d
Error [Server]: (#42014) Caught exception : exception 70000004: cannot allocate enough memory: SQL Error

)
OData Version: 4, Error: The remote server returned an error: (500) Internal Server Error. (Unable to process query: Error: Debug [IOLayer]: (#99999) 251596f8-4744-4598-6903-6f0550f464f0
Error [Server]: (#42014) Caught exception : exception 70000004: cannot allocate enough memory: Allocation failed; Reason: configured statement memory limit from global configuration reached (SAP Note 3191224)
)
OData Version: 3, Error: The remote server returned an error: (500) Internal Server Error. (Unable to process query: Error: Debug [IOLayer]: (#99999) 2752614e-b8b3-4375-68d3-5311aaf54a64
Error [Server]: (#42014) Caught exception : exception 70000004: cannot allocate enough memory: SQL Error

Is there any specific setting in PowerBI that I am missing?

Thanks in advance Sven

ghost commented 1 month ago

Hi Sven, it seems that even the limited result-set consumes more memory than available for your user. Best regards, Olaf