aws-taylor
commented
1 month ago Thanks @Bartosz-G, we're taking a look. This may be the same issue as https://github.com/aws-neuron/aws-neuron-sdk/issues/891
Open Bartosz-G opened 1 month ago
aws-taylor
commented
1 month ago Thanks @Bartosz-G, we're taking a look. This may be the same issue as https://github.com/aws-neuron/aws-neuron-sdk/issues/891
Description
When tracing the model with
compiler_args = ['--neuroncore-pipeline-cores', '4']the tracing passes but traced model cannot be loaded back in due to the NEFF error (below). When tracing the same model without the following flag the model is correctly loaded in and runs on Inferentia core.Code:
Error When loading back the model:
DEBUG logs of the final job:
Instance:
ENV:
Model:
RTDETR - https://github.com/lyuwenyu/RT-DETR
Steps to reproduce:
from src.core import YAMLConfig from src.solver import TASKS from src.core.yaml_utils import load_config
config_name = 'rtdetr_r18vd_6x_coco.yml' config_path = os.path.join('configs', 'rtdetr', config_name)
cfg = YAMLConfig( config_path )
rtdetr = cfg.model rtdetr.deploy()
img = torch.rand(1, 3, 640, 640)
with torch.no_grad(): output = rtdetr(img)
neuron_model_path = 'rtdetr480_neuron_test.pt' model_neuron = torch.neuron.trace(rtdetr, img, compiler_args = ['--neuroncore-pipeline-cores', '4', '--verbose', 'DEBUG']) model_neuron.save(neuron_model_path)
os.environ['NEURON_RT_LOG_LEVEL'] = 'WARN' rtdetr = torch.jit.load(neuron_model_path) img = torch.rand(1, 3, 480, 480)
Warm-up
output = rtdetr(img)