search

flexflow / FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving
https://flexflow.readthedocs.io
Apache License 2.0
1.67k stars 224 forks source link

Failed to export the parallelization strategy of FlexFlow (Unity) #690

Closed Ash-Zheng closed 1 year ago

Ash-Zheng commented 1 year ago

Hi,

I am facing some issues when I tried to export the optimal parallelization strategy discovered by FlexFlow (Unity). I have tried both Python and C++ examples using the flag settings --budget=20 --export mlp_strategy.txt. While the training proceeds as expected, the output file of the parallelization strategy is not generated.

Here are the commands I have used:

"$FF_HOME"/build/examples/cpp/MLP_Unify/mlp_unify -ll:gpu 2 -ll:fsize 14000 -ll:zsize 14000 --budget 20 --export mlp_strategy.txt 

./flexflow_python ../examples/python/native/mnist_mlp.py -ll:py 1 -ll:gpu 2 -ll:fsize 8192 -ll:zsize 8192 --export mlp_strategy.txt --search-budget 20 --enable-parameter-parallel -enable-attribute-parallel

How can I correctly export the parallelization strategy? Thanks!

lockshaw commented 1 year ago

@Ash-Zheng Sorry for the delayed response! Strategy export is currently not supported by Unity, but will be enabled in #622. You can follow #403 to receive an update when this is added--we're hoping to merge #622 in the next month or two.