search

hpcaitech / EnergonAI

Large-scale model inference.
Apache License 2.0
630 stars 90 forks source link

Doesn't run gpt reference? #196

Open YuchengWang opened 1 year ago

YuchengWang commented 1 year ago

root@2d8fec1401d1:/workspace/EnergonAI# python examples/gpt/gpt_batch_server.py Traceback (most recent call last): File "/workspace/EnergonAI/examples/gpt/gpt_batch_server.py", line 7, in from energonai.engine import InferenceEngine ImportError: cannot import name 'InferenceEngine' from 'energonai.engine' (/opt/conda/lib/python3.9/site-packages/energonai/engine.py)

When I modified inferenceEngine. Another error ocurres:

root@2d8fec1401d1:/mnt/EnergonAI# python examples/gpt/gpt_batch_server.py Traceback (most recent call last): File "/mnt/EnergonAI/examples/gpt/gpt_batch_server.py", line 8, in from energonai.legacy_batch_mgr.dynamic_batch_manager import Dynamic_Batch_Manager File "/opt/conda/lib/python3.9/site-packages/energonai/legacy_batch_mgr/init.py", line 1, in from .worker_server import launch_worker ModuleNotFoundError: No module named 'energonai.legacy_batch_mgr.worker_server'

ver217 commented 1 year ago

Hi, please see https://github.com/hpcaitech/EnergonAI/tree/main/examples/bloom or https://github.com/hpcaitech/EnergonAI/tree/main/examples/opt and simply replace the model with GPT.