flexflow / FlexFlow

FlexFlow Serve: Low-Latency, High-Performance LLM Serving
https://flexflow.readthedocs.io
Apache License 2.0
1.6k stars 219 forks source link

Add examples for every layer in the python layer API #1297

Closed FelixBrakel closed 1 month ago

FelixBrakel commented 5 months ago

I am considering using FlexFlow for my master thesis so I created this as a test to see what the state of the python API is.

Description of changes: This pull request adds examples for every layer in the Python API. These examples are added under the examples folder as standalone scripts for easy reference and testing. Each example script demonstrates the usage of the respective layer with sample input data and prints the output array. The examples are added under the examples folder instead of the tests folder, as I am not familiar with the process of integrating these tests into the CI pipeline.

How was this created?

I manually created the example for conv2d.py through trial and error. I then provided this as an example to ChatGPT and together with the python code for the layer asked it to create a similar test for said layer. I then made sure that when running the example no exceptions would occur within the example script.

Using this method I obtained the following results:

File Status
mean.py Exception. Stack trace saved to mean.py_error.txt
identity.py Success
spec_inc_multihead_self_attention.py Exception. Stack trace saved to spec_inc_multihead_self_attention.py_error.txt
arg_top_k.py Exception. Stack trace saved to arg_top_k.py_error.txt
tanh.py Success
pow.py Success
concat.py Exception. Stack trace saved to concat.py_error.txt
conv2d.py Success
sigmoid_silu_multi.py Exception. Stack trace saved to sigmoid_silu_multi.py_error.txt
inc_multiquery_self_attention_verify.py Exception. Stack trace saved to inc_multiquery_self_attention_verify.py_error.txt
sigmoid.py Success
max.py Success
sampling.py Exception. Stack trace saved to sampling.py_error.txt
residual_layer_norm.py Exception. Stack trace saved to residual_layer_norm.py_error.txt
add_bias_residual_layer_norm.py Exception. Stack trace saved to add_bias_residual_layer_norm.py_error.txt
reduce_sum.py Success
split.py Exception. Stack trace saved to split.py_error.txt
elu.py Success
cos.py Success
relu.py Success
subtract.py Success
rms_norm.py Exception. Stack trace saved to rms_norm.py_error.txt
scalar_sub.py Success
spec_inc_multiquery_self_attention.py Exception. Stack trace saved to spec_inc_multiquery_self_attention.py_error.txt
gather.py Exception. Stack trace saved to gather.py_error.txt
scalar_add.py Success
inc_multihead_self_attention.py Exception. Stack trace saved to inc_multihead_self_attention.py_error.txt
divide.py Exception. Stack trace saved to divide.py_error.txt
gelu.py Success
batch_norm.py Exception. Stack trace saved to batch_norm.py_error.txt
min.py Success
multihead_attention.py Success
pool2d.py Exception. Stack trace saved to pool2d.py_error.txt
scalar_multiply.py Success
batch_matmul.py Success
sin.py Success
reshape.py Success
scalar_true_divide.py Success
rsqrt.py Success
layer_norm.py Exception. Stack trace saved to layer_norm.py_error.txt
residual_rms_norm.py Exception. Stack trace saved to residual_rms_norm.py_error.txt
transpose.py Exception. Stack trace saved to transpose.py_error.txt
embedding.py Success
dense.py Success
beam_top_k.py Exception. Stack trace saved to beam_top_k.py_error.txt
exp.py Success
multiply.py Success
flat.py Success
reverse.py Exception. Stack trace saved to reverse.py_error.txt
inc_multiquery_self_attention.py Exception. Stack trace saved to inc_multiquery_self_attention.py_error.txt
softmax.py Exception. Stack trace saved to softmax.py_error.txt
dropout.py Exception. Stack trace saved to dropout.py_error.txt
add.py Success
inc_multihead_self_attention_verify.py Exception. Stack trace saved to inc_multihead_self_attention_verify.py_error.txt
argmax.py Exception. Stack trace saved to argmax.py_error.txt

Out of the 55 operators 27 currently crash.

Of course just because the exception doesn't originate from the example script doesn't mean that the example is correct, this is as far as I am able to get on my own however. Merging this PR then would unfortunately require reviewing the 27 crashing examples.

I have attached the crashlogs errors.zip


This change is Reviewable

jiazhihao commented 4 months ago

@eddy16112 Any updates on this PR? Do you think it's ready to be merged?