search

Snowflake-Labs / sfquickstarts

Follow along with our tutorials to get you up and running with Snowflake.
Apache License 2.0
350 stars 656 forks source link

Pivot Table error when running a cell in sfguide-data-engineering-pipelines-with-pandas-on-snowflake tutorial #1691

Open igindragunawan-paycor opened 3 weeks ago

igindragunawan-paycor commented 3 weeks ago

Describe the bug Pivot Table error when running a cell: SnowparkSQLException: (1304): 01b80516-0000-8ba2-002a-f703000a343a: 002003 (42S02): SQL compilation error: Object 'SNOWPARK_TEMP_TABLE_AIUDGY0C73READONLY' does not exist or not authorized.

To Reproduce Steps to reproduce the behavior:

  1. Go to '(https://github.com/Snowflake-Labs/sfguide-data-engineering-pipelines-with-pandas-on-snowflake/blob/main/notebooks/0_start_here.ipynb)'
  2. Run the Notebook from beginning until you reach the code block below
  3. the code in the notebook to reproduce this bug : 
    spd_lineitem_agg_pivot_df = spd_lineitem_agg.pivot_table(
                            values=['NBR_OF_ITEMS', 'TOT_DISCOUNT_AMOUNT'], 
                            index=['L_ORDERKEY'],
                            columns=['L_RETURNFLAG'], 
                            aggfunc="sum")

here is the Full Error message:

SnowparkSQLException: (1304): 01b80516-0000-8ba2-002a-f703000a343a: 002003 (42S02): SQL compilation error: Object 'SNOWPARK_TEMP_TABLE_AIUDGY0C73READONLY' does not exist or not authorized.
Traceback:
File "Cell [cell19]", line 2, in <module>
    spd_lineitem_agg_pivot_df = spd_lineitem_agg.pivot_table(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/telemetry.py", line 414, in wrap
    return _telemetry_helper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/telemetry.py", line 341, in _telemetry_helper
    raise e
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/telemetry.py", line 327, in _telemetry_helper
    result = func(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/extensions/dataframe_overrides.py", line 1770, in pivot_table
    query_compiler=self._query_compiler.pivot_table(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/modin/logging/logger_decorator.py", line 125, in run_and_log
    return obj(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/compiler/snowflake_query_compiler.py", line 9018, in pivot_table
    raise e
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/compiler/snowflake_query_compiler.py", line 8972, in pivot_table
    pivotted_frame = pivot_helper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/pivot_utils.py", line 390, in pivot_helper
    ) = perform_pivot_and_concatenate(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/pivot_utils.py", line 114, in perform_pivot_and_concatenate
    ) = single_pivot_helper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/pivot_utils.py", line 633, in single_pivot_helper
    pivot_ordered_dataframe = pivot_ordered_dataframe.pivot(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 834, in pivot
    return OrderedDataFrame(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 244, in __init__
    self._dataframe_ref.snowflake_quoted_identifiers
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 166, in snowflake_quoted_identifiers
    f.column_identifier.quoted_name for f in self.schema.fields
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/modin/plugin/_internal/ordered_dataframe.py", line 154, in schema
    return self.snowpark_dataframe.schema
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/functools.py", line 993, in __get__
    val = self.func(instance)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/dataframe.py", line 4214, in schema
    return StructType._from_attributes(self._plan.attributes)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/functools.py", line 993, in __get__
    val = self.func(instance)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 401, in attributes
    output = analyze_attributes(self.schema_query, self.session)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/schema_utils.py", line 93, in analyze_attributes
    return session._get_result_attributes(sql)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/session.py", line 2340, in _get_result_attributes
    return self._conn.get_result_attributes(query)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 208, in wrap
    raise ne.with_traceback(tb) from None
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/snowflake_plan.py", line 139, in wrap
    return func(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/server_connection.py", line 248, in get_result_attributes
    self._run_new_describe(self._cursor, query), self.max_string_size
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/server_connection.py", line 254, in _run_new_describe
    result_metadata = run_new_describe(cursor, query)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/snowpark/_internal/analyzer/schema_utils.py", line 147, in run_new_describe
    return cursor._describe_internal(query)  # pyright: ignore
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 1063, in _describe_internal
    self.execute(*args, **kwargs)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/cursor.py", line 1018, in execute
    Error.errorhandler_wrapper(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/errors.py", line 232, in errorhandler_wrapper
    handed_over = Error.hand_to_other_handler(
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/errors.py", line 287, in hand_to_other_handler
    cursor.errorhandler(connection, cursor, error_class, error_value)
File "/usr/lib/python_udf/ea4de7322d96cfdb8d71af67018277d34aa9e8d6e32b5548305eadb861c13650/lib/python3.9/site-packages/snowflake/connector/errors.py", line 165, in default_errorhandler
    raise error_class(

Screenshots image

Desktop (please complete the following information):

Additional context Run this in Snowsight Notebooks

jabjunk commented 1 week ago

I have the same problem