microsoft / TaskWeaver

A code-first agent framework for seamlessly planning and executing data analytics tasks.
https://microsoft.github.io/TaskWeaver/
MIT License
5.38k stars 689 forks source link

Having trouble getting packages installed for plugins to use #343

Closed TestMannequin closed 6 months ago

TestMannequin commented 6 months ago

I have a plugin that uses the package msgraph-sdk. The plugin works fine when this dependency is removed and output hardcoded.

I am running taskweaver with this syntax: python -m taskweaver -p ./project/   To try and get the package installed I have added msgraph-sdk to the requirements.txt file and rebuilt the docker image: powershell build_executor.ps1

During the build I know this package is being installed from the logs, but when I run taskweaver it is not installed because the plug fails to load, and asking taskweaver if it is installed returns that it is not.

What is the correct way to get a package installed for a plugin to reference?

Environment Information (please complete the following information):

TestMannequin commented 6 months ago

Ahh OK, it's because the command python -m taskweaver -p ./project/ is re-downloading the image and overwriting the latest.

So it seems the best practice is to run the rebuilt docker image directly?

TestMannequin commented 6 months ago

OK, for the next person that has this problem. The help directions to pip install something are not useful, because the command line python -m taskweaver -p ./project/ seems to always use the default docker image, even if you rebuild the ces image.

So to get around this I had to build the taskweaver-all-in-one image, but since on windows had to first convert the line endings to LF from CRLF in the entrypoint.sh file in the all_in_one_container folder before building it.

Summarized: 1) Convert line endings to LF from CRLF in TaskWeaver\docker\all_in_one_container\entrypoint.sh 2) Build docker image using TaskWeaver\scripts\build_all_in_one.ps1 3) Run the docker image per the commands documented well here