[Docs] pima_diabetes flytekit version is pinned to an old version

[hududed]

Description

Tried the diabetes tutorial and ran this:

flytectl demo start
cd examples/pima_diabetes
pyflyte run --remote pima_diabetes/diabetes.py diabetes_xgboost_model # as usually practiced in basics

But somehow getting this flytekit.exceptions.user.FlyteValidationException: Input pima_diabetes/diabetes.py must be in format '<file.py>:<worfklow>'

@cosmicBboy referenced that its a version issue https://github.com/flyteorg/flytesnacks/blob/master/examples/pima_diabetes/requirements.txt#L45 requirements.txt

Potentially related: pyflyte run --remote pima_diabetes/diabetes.py:diabetes_xgboost_model shows up in the console but for some reason unable to find the module No module named 'marshmallow_enum' even though it's installed in a poetry environment

Are you sure this issue hasn't been raised already?

• [X] Yes

Have you read the Code of Conduct?

• [X] Yes

[welcome[bot]]

Thank you for opening your first issue here! 🛠

[hududed]

So I updated flytekit and its dependencies in pyproject.toml - running pyflyte run pima_diabetes/diabetes.py diabetes_xgboost_model within a poetry environment works. These are the dependencies:

[tool.poetry.dependencies]
python = ">=3.9,<3.11"
arrow = "1.2.2"
binaryornot = "0.4.4"
certifi = "2021.10.8"
chardet = "4.0.0"
charset-normalizer = "2.0.12"
checksumdir = "1.2.0"
click = "8.1.2"
cloudpickle = "2.0.0"
cookiecutter = "1.7.3"
croniter = "1.3.4"
cycler = "0.11.0"
dataclasses-json = "0.5.7"
decorator = "5.1.1"
deprecated = "1.2.13"
diskcache = "5.4.0"
docker-image-py = "0.1.12"
docstring-parser = "0.13"
fonttools = "4.32.0"
idna = "3.3"
importlib-metadata = "4.11.3"
jinja2 = "3.1.1"
jinja2-time = "0.2.0"
joblib = "1.1.0"
keyring = "23.5.0"
kiwisolver = "1.4.2"
markupsafe = "2.1.1"
marshmallow = "3.15.0"
marshmallow-enum = "1.5.1"
marshmallow-jsonschema = "0.13.0"
matplotlib = "3.5.1"
mypy-extensions = "0.4.3"
natsort = "8.1.0"
numpy = "1.22.3"
packaging = "21.3"
pandas = "1.4.2"
pillow = "9.1.0"
poyo = "0.5.0"
protoc-gen-swagger = "0.1.0"
py = "1.11.0"
pyarrow = "6.0.1"
pyparsing = "3.0.8"
python-dateutil = "2.8.2"
python-json-logger = "2.0.2"
python-slugify = "6.1.1"
pytimeparse = "1.1.8"
pytz = "2022.1"
regex = "2022.3.15"
requests = "2.27.1"
responses = "0.20.0"
retry = "0.9.2"
scikit-learn = "1.0.2"
scipy = "1.8.0"
six = "1.16.0"
sklearn = "0.0"
sortedcontainers = "2.4.0"
statsd = "3.3.0"
tabulate = "0.8.9"
text-unidecode = "1.3"
threadpoolctl = "3.1.0"
typing-extensions = "4.1.1"
typing-inspect = "0.7.1"
urllib3 = "1.26.9"
wheel = "0.37.1"
wrapt = "1.14.0"
xgboost = "1.5.2"
zipp = "3.8.0"
flytekit = "^1.8.1"
flytekitplugins-deck-standard = "^1.8.1"

However, adding the --remote tag now gives me this in the console output

[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes.
[fa06ef244a90e4a87884-n0-0] terminated with exit code (247). Reason [OOMKilled]. Message: 
tar: Removing leading `/' from member names

[eapolinario]

This indicates that one of the pods responsible for running the tasks in that workflow is hitting an out-of-memory error. Can you try increasing the resources for the task (e.g. https://docs.flyte.org/projects/cookbook/en/latest/auto_examples/nlp_processing/word2vec_and_lda.html#word-similarities, notice the use of Resources in the task definition) and re-running? Contributions are welcome!

[dado5688]

Hello, @hududed. I'm interested in this issue, I process by the following step,

curl -sL https://ctl.flyte.org/install | sudo bash -s -- -b /usr/local/bin
pip install flytekit flytekitplugins-deck-standard scikit-learn
flytectl demo start
pyflyte run --remote pima_diabetes/diabetes.py diabetes_xgboost_model

but it return the ModuleNotFoundError

Failed with Unknown Exception <class 'ModuleNotFoundError'> Reason: No module named 'xgboost'
No module named 'xgboost'

so I install the xgboost first, and rerun it but it didn't return the error that you mention above,

[1/1] currentAttempt done. Last Error: USER::Pod failed. No message received from kubernetes. [fa06ef244a90e4a87884-n0-0] terminated with exit code (247). Reason [OOMKilled]. Message: tar: Removing leading `/' from member names

if you don't mind, can I ask more detail about your process ?

[hududed]

Hi @dado5688 sorry I am not actively working on flytekit problems atm - but this should be easily addressed by the Flyte folks in their slack channel.

[kumare3]

Two things, increase memory for the pod, and ensure the docker image has xgboost

[dado5688]

@hududed Ok, I see, thanks for the response !

[dado5688]

Hi, @kumare3 since I'm a new one, don't know much detail behind the process, can I ask a question about I don't understand that my requirement.txt is already has xgboost, but I still need to install it locally before pyflyte run? or there have some steps I miss? (I already build the docker image in local and specify it when pyflyte run)

[github-actions[bot]]

Hello 👋, this issue has been inactive for over 9 months. To help maintain a clean and focused backlog, we'll be marking this issue as stale and will engage on it to decide if it is still applicable. Thank you for your contribution and understanding! 🙏