rockchip-linux / rknn-toolkit

BSD 3-Clause "New" or "Revised" License
817 stars 174 forks source link

Model input different than images #241

Open tucanae47 opened 2 years ago

tucanae47 commented 2 years ago

Hello all, I would like to know if is possible to load a model that does not use images as inputs (and instead text, scalars, or audio), and how can you do this, as an example im trying to load a pytorch LSTM model for timeseries prediction but im not able to do it yet, i dont know how to set the parameter input_size_list i.e ret = rknn.load_pytorch(model=PT_MODEL, input_size_list=[[1,1,6]]) and also the same for rknn.config of the RKNN object rknn.config(channel_mean_value='0 0 0 255', reorder_channel='0 1 2', batch_size=1) and I havent found information about this in the documentation

here is the complete code example im using

import os
import urllib
import traceback
import time
import sys
import numpy as np
import cv2
from rknn.api import RKNN

if __name__ == '__main__':
    PT_MODEL = 'lstm.pt'
    RKNN_MODEL = 'lstm.rknn'

    # Create RKNN object
    rknn = RKNN()
    print('--> config model')
    rknn.config(channel_mean_value='0 0 0 255', reorder_channel='0 1 2', batch_size=1)

    # Load tensorflow model
    print('--> Loading model')
    ret = rknn.load_pytorch(model=PT_MODEL, input_size_list=[[1,1,6]])
    if ret != 0:
        print('Load failed!')
        exit(ret)

    # Build model
    print('--> Building model')
    ret = rknn.build(do_quantization=False)  # pre_compile=True
    if ret != 0:
        print('Build  failed!')
        exit(ret)
    print('done')

    # Export rknn model
    print('--> Export RKNN model')
    ret = rknn.export_rknn(RKNN_MODEL)
    if ret != 0:
        print('Export lstm.rknn failed!')
        exit(ret)
    print('done')
    rknn.release()

and the current output im getting using version 1.7.1 of rknn toolkit and platform RV1126 soc

E Catch exception when loading pytorch model: lstm.pt!
E Traceback (most recent call last):
E   File "rknn/base/RKNNlib/converter/convert_pytorch_new.py", line 3967, in rknn.base.RKNNlib.converter.convert_pytorch_new.convert_pytorch.load
E   File "rknn/base/RKNNlib/converter/pytorch_util.py", line 75, in rknn.base.RKNNlib.converter.pytorch_util.check_model_pt_version
E   File "rknn/base/RKNNlib/converter/pytorch_util.py", line 76, in rknn.base.RKNNlib.converter.pytorch_util.check_model_pt_version
E   File "/home/tuc/.pyenv/versions/3.5.5/lib/python3.5/zipfile.py", line 1026, in __init__
E     self._RealGetContents()
E   File "/home/tuc/.pyenv/versions/3.5.5/lib/python3.5/zipfile.py", line 1093, in _RealGetContents
E     raise BadZipFile("File is not a zip file")
E zipfile.BadZipFile: File is not a zip file
E Please feedback the detailed log file <log_feedback_to_the_rknn_toolkit_dev_team.log> to the RKNN Toolkit development team.
sahi1422 commented 2 years ago

@tucanae47 , did you find clear documentation/explanation on how to set input_size_list of scalar input, gray scale images, etc?