towhee-io / towhee

Towhee is a framework that is dedicated to making neural data processing pipelines simple and fast.
https://towhee.io
Apache License 2.0
3.16k stars 246 forks source link

[Bug]: When building the triton backend, using custom operators may result in errors #2693

Closed bug400400 closed 4 months ago

bug400400 commented 6 months ago

Is there an existing issue for this?

Current Behavior

@register(name='embedding') class Embedding(NNOperator): def init(self):

def __call__(self, object, type, model):
    return vec

embedding_pipe = ( pipe.input('object') .map('object', 'vec', ops.embedding()) .output('vec') )

towhee.build_docker_image( dc_pipeline=embedding_pipe, image_name='triton_embedding:v1.0', cuda_version='11.7', format_priority=['onnx'], parallelism=4, inference_server='triton' )

but it will go to the hub to download the operator image

Expected Behavior

build success

Steps To Reproduce

@register(name='embedding')
class Embedding(NNOperator):
    def __init__(self):
      ####
    def __call__(self, object, type, model):
        ####
        return vec

embedding_pipe = (
    pipe.input('object')
        .map('object', 'vec', ops.embedding())
        .output('vec')
)

towhee.build_docker_image(
    dc_pipeline=embedding_pipe,
    image_name='triton_embedding:v1.0',
    cuda_version='11.7',
    format_priority=['onnx'],
    parallelism=4,
    inference_server='triton'
)

but it will go to the hub to download the operator

Environment

- Towhee version(e.g. v0.1.3 or 8b23a93):
- OS(Ubuntu or CentOS): Ubuntu 
- CPU/Memory: CPU
- GPU:
- Others:

Anything else?

No response

bug400400 commented 6 months ago

i find ,while build_docker_image,REGISTRY: Dict[str, Any] ,is null。

junjiejiangjjj commented 6 months ago

Only supports hub operators and lambda function when using triton backend

bug400400 commented 6 months ago

Only supports hub operators and lambda function when using triton backend

Can this feature be added?I think this feature is very common in projects. I have found a temporary solution to avoid it. I need to add @ register code to the/usr/local/bin/triton_builder and model.py files

stale[bot] commented 5 months ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. Rotten issues close after 30d of inactivity. Close the stale issues and pull requests after 7 days of inactivity. Reopen the issue with /reopen.