microsoft / semantic-link-labs

Early access to new features for Microsoft Fabric's Semantic Link.
MIT License
178 stars 37 forks source link

In run_report_bpa not working export #198

Closed d-cegielka closed 1 month ago

d-cegielka commented 1 month ago

Describe the bug Calling the sempy_labs.report.run_report_bpa function in export mode results in an error.

To Reproduce

rep.run_report_bpa(report=report, export=True)

Expected behavior Result dataframe was saved in lakehouse

Logs

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[43], line 1
----> 1 rep.run_report_bpa(report=report, export=True)

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy/_utils/_log.py:273, in mds_log.<locals>.get_wrapper.<locals>.log_decorator_wrapper(*args, **kwargs)
    270     raise
    272 try:
--> 273     result = func(*args, **kwargs)
    275     # The invocation for get_message_dict moves after the function
    276     # so it can access the state after the method call
    277     message.update(extractor.get_completion_message_dict(result, arg_dict))

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy_labs/report/_report_bpa.py:252, in run_report_bpa(report, rules, workspace, export, return_dataframe)
    233     export_df["RunId"] = export_df["RunId"].astype(int)
    235     export_df = [
    236         [
    237             "Capacity Name",
   (...)
    250         ]
    251     ]
--> 252     save_as_delta_table(
    253         dataframe=export_df,
    254         delta_table_name=delta_table_name,
    255         write_mode="append",
    256         merge_schema=True,
    257     )
    259     return
    261 finalDF.replace(
    262     {"Warning": icons.warning, "Error": icons.error, "Info": icons.info},
    263     inplace=True,
    264 )

File ~/cluster-env/clonedenv/lib/python3.11/site-packages/sempy_labs/_helper_functions.py:464, in save_as_delta_table(dataframe, delta_table_name, write_mode, merge_schema, schema, lakehouse, workspace)
    459 if " " in delta_table_name:
    460     raise ValueError(
    461         f"{icons.red_dot} Invalid 'delta_table_name'. Delta tables in the lakehouse cannot have spaces in their names."
    462     )
--> 464 dataframe.columns = dataframe.columns.str.replace(" ", "_")
    465 spark = SparkSession.builder.getOrCreate()
    467 type_mapping = {
    468     "string": StringType(),
    469     "str": StringType(),
   (...)
    478     "timestamp": TimestampType(),
    479 }

AttributeError: 'list' object has no attribute 'columns'

Environment Runtime: 1.3 (Spark 3.5, Delta 3.2)

m-kovalsky commented 1 month ago

Thanks for reporting. Made a fix for this for the next release.

m-kovalsky commented 1 month ago

Fixed in 0.8.3.