pulp-platform / pulp-dronet

A deep learning-powered visual navigation engine to enables autonomous navigation of pocket-size quadrotor - running on PULP
Apache License 2.0
500 stars 163 forks source link

How to use your best weights #9

Closed Yingxiu-Chang closed 2 months ago

Yingxiu-Chang commented 3 years ago

@DP8 Hi, I'm reading your work, "DroNet Learning to Fly by Driving" and "An Open Source and Open Hardware Deep Learning-powered Visual Navigation Engine". I was inspired that you use DroNet to fly nano UAV in the lab environments. But I still meet some questions. I was using the following code to predict a single image. The model and the best weights I used was from https://github.com/uzh-rpg/rpg_public_dronet. In addition, I tested the single image from your Himax_Dataset-master/test_2/frame_26.pgm. It's really good to test the outdoor images, but hard to get the correct results for indoor sceneries. I was using the keras to construct the model, so I can't use your hex weights from your pulp-dronet/weights/binary/. So may I know how you trained your DroNet for the pulp-dronet. Did you use keras or any other dependencies and packages? Or if you can provide the best weights of pulp-dronet which you originally generated .hex files like h5py or so forth?

from PIL import Image
import utils
import numpy as np
import cv2

model = utils.jsonToModel("model/model_struct.json")
model.load_weights("model/best_weights.h5")
model.summary()

def central_image_crop(img, crop_width=150, crop_heigth=150):

    half_the_width = int(img.shape[1] / 2)
    img = img[img.shape[0] - crop_heigth: img.shape[0],
          half_the_width - int(crop_width / 2):
          half_the_width + int(crop_width / 2)]
    return img

def read_img():
    img = np.array(Image.open("./model/Himax_Dataset-master/test_2/frame_26.pgm"))  
    cv2.imshow('', img)
    cv2.waitKey(0)
    img_central = central_image_crop(img, 200, 200) 
    img_01 = np.asarray(img_central, dtype=np.float32) * np.float32(1.0/255.0) 
    img_3d = np.expand_dims(img_01, axis=0)  
    im_4d = img_3d[:, :, :, np.newaxis]    
    return im_4d

im = read_img()    
outs = model.predict([im])
steer, coll = outs[0][0], outs[1][0]
print("Steer angle= ", steer)
print("Collision prob= ", coll)