[BUG] flytedecks example in documentation does not work

nuhaj commented 1 month ago

Describe the bug

I found the online examples here https://docs.flyte.org/en/latest/user_guide/development_lifecycle/decks.html#decks do not work . There is a scikit-learn and pandas dependency not mentioned that need an image_spec.

This page needs to also be updated since it uses the deprecated disable_deck=False

code and error running test flytedeck

Traceback (most recent call last):

      File "/usr/local/lib/python3.11/site-packages/flytekit/exceptions/scopes.py", line 183, in system_entry_point
        return wrapped(*args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/flytekit/core/base_task.py", line 747, in dispatch_execute
        self._write_decks(native_inputs, native_outputs_as_map, ctx, new_user_params)
      File "/usr/local/lib/python3.11/site-packages/flytekit/core/python_function_task.py", line 369, in _write_decks
        python_dependencies_deck.append(renderer.to_html())
                                        ^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/site-packages/flytekit/deck/renderer.py", line 107, in to_html
        installed_packages = json.loads(
                             ^^^^^^^^^^^
      File "/usr/local/lib/python3.11/json/__init__.py", line 346, in loads
        return _default_decoder.decode(s)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/usr/local/lib/python3.11/json/decoder.py", line 340, in decode
        raise JSONDecodeError("Extra data", s, end)

Message:

    JSONDecodeError: Extra data: line 2 column 1 (char 6887)

SYSTEM ERROR! Contact platform administrators.

and code

import pandas as pd
from flytekitplugins.deck.renderer import FrameProfilingRenderer
from flytekit import task, ImageSpec
import flytekit

custom_image = ImageSpec(

    packages=[ "pandas"],
    apt_packages=["git"],
    env={"Debug": "True"},
    registry="****",
)

@task(enable_deck=True,  container_image=custom_image)
def frame_renderer() -> None:
    df = pd.DataFrame(data={"col1": [1, 2], "col2": [3, 4]})
    flytekit.Deck("Frame Renderer", FrameProfilingRenderer().to_html(df=df))

Expected behavior

Render a simple flytedeck example similar to https://docs.flyte.org/en/latest/user_guide/development_lifecycle/decks.html#decks

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

[X] Yes

Have you read the Code of Conduct?

[X] Yes

runllm[bot] commented 1 month ago

The issue you're encountering is due to missing dependencies and the use of deprecated parameters in the Flyte documentation examples. To resolve this, you need to ensure that the required dependencies are included in the ImageSpec and update the code to use the enable_deck parameter instead of the deprecated disable_deck.

Here's an updated version of your code:

import pandas as pd
from flytekitplugins.deck.renderer import FrameProfilingRenderer
from flytekit import task, ImageSpec, Deck

custom_image = ImageSpec(
    packages=["pandas", "scikit-learn"],
    apt_packages=["git"],
    env={"Debug": "True"},
    registry="****",
)

@task(enable_deck=True, container_image=custom_image)
def frame_renderer() -> None:
    df = pd.DataFrame(data={"col1": [1, 2], "col2": [3, 4]})
    Deck("Frame Renderer", FrameProfilingRenderer().to_html(df=df))

This code includes the necessary dependencies (pandas and scikit-learn) in the ImageSpec and uses the enable_deck parameter.

Would you like more details on how to set up and run this example?

Sources:

This is an AI-generated response and your feedback is appreciated! Please leave a 👍 if this is helpful and 👎 if it is not.

Mention @runllm to ask follow-up questions. See runllm.com for more!

thomasjpfan commented 1 month ago

I opened a few PRs to fix the docs, but I am not able to reproduce the raise JSONDecodeError("Extra data", s, end) error.

nuhaj commented 1 month ago

I was able to get decks working only after the following steps

update to flytekit and flytekitplugins-deck-standard and flytekit-envd 1.13.3
add flytekitplugins-deck-standard to imagespec. This is not in the documentation
after the above push a new image

import pandas as pd
from flytekitplugins.deck.renderer import FrameProfilingRenderer
from flytekit import task, ImageSpec
import flytekit

custom_image = ImageSpec(

    packages=[ "pandas", "flytekitplugins-deck-standard"],
    registry="****",
).force_push()

@task(enable_deck=True 
, container_image=custom_image)
def frame_renderer() -> None:
    df = pd.DataFrame(data={"col1": [1, 2], "col2": [3, 4]})
    flytekit.Deck("Frame Renderer", FrameProfilingRenderer().to_html(df=df))

# pyflyte --verbose run --remote -p "workspace" basic.py frame_renderer

flyteorg / flyte