Jetbot very biased towards right side

[abuelgasimsaadeldin]

Guys, I have a Waveshare Jetbot and I trained the model really well but in a straight road my model decides to turn right and it very biased towards the right side, please help me guys it's an emergency. Not sure if there is something wrong with the x slider calculations or what, I am really confused at this moment. Thanks.

[abuelgasimsaadeldin]

@jaybdub @tokk-nv, your expertise and help in solving this issue would be very much appreciated. This only happens during the road following live demo.

[jaybdub]

Hi @abuelgasimsaadeldin ,

Thanks for reaching out.

This could be from various issues. One thing to quickly check is that the neural network is predicting as expected.

Are you able to place the robot (without motors running) in various positions on the track (centered, offset left, offset right, tilted left, tilted right, etc.) and verify that the output steering value is as expected?

If so this might indicate the issue is in the vehicle dynamics.

Best, John

[abuelgasimsaadeldin]

Hi @jaybdub ,

Thanks for the reply, I have checked the neural network predictions while turning off the motors and it is neural network predictions that point to the right side all time even though it has been trained with over 400 images (I kept on increasing) with proper data variance (different angles, different lighting conditions, different offsets from center etc.) on the road and to follow the correct path.

Usually I would collect data (on my pervious JetBot and on my previous track) using just the gamepad controller, however, on this new JetBot, since the track is a bit more complex, I decided to collect images using the mouse (more convenient) and I thought that maybe that would have been the issue, or it could just be that the new track is a bit too complex?

I'm confused at this point, do I need to collect 1000s of training images for it to work properly? Was that ever the case before?

Attached herewith is an image of my new track for your reference.

Best Regards, Abu

[abuelgasimsaadeldin]

Hi @jaybdub @tokk-nv, I am still facing this issue even with another track that I have. So I confirmed that it is not an issue with this track specifically. (my model is still biased towards the right side almost everytime).

I'm also sure that there is nothing wrong with my training data, I have collected a total of about 150 training images (using mouse) with variations in the data and with different lighting conditions and different offsets etc. I am thinking that the problem has to do with the training and so I would like to ask a few questions.

What is the desirable final loss that should be obtained? What are the hyperparameters that I should tune in order to achieve good results (eg. I set batch size to 8 and num workers to 0 for this task, is that ok?) and what are hyperparameters that you would suggest I should play with in order to achieve my desired output? Your help would be highly appreciated.

Thank you and Best Regards, Abu

[abuelgasimsaadeldin]

I found out the issue was coming from the Dataset Instance in train_model from the Waveshare Repo:

    """Gets the x value from the image filename"""
    return (float(int(path[3:6])) - 50.0) / 50.0

def get_y(path):
    """Gets the y value from the image filename"""
    return (float(int(path[7:10])) - 50.0) / 50.0

class XYDataset(torch.utils.data.Dataset):

    def __init__(self, directory, random_hflips=False):
        self.directory = directory
        self.random_hflips = random_hflips
        self.image_paths = glob.glob(os.path.join(self.directory, '*.jpg'))
        self.color_jitter = transforms.ColorJitter(0.3, 0.3, 0.3, 0.3)

    def __len__(self):
        return len(self.image_paths)

    def __getitem__(self, idx):
        image_path = self.image_paths[idx]

        image = PIL.Image.open(image_path)
        x = float(get_x(os.path.basename(image_path)))
        y = float(get_y(os.path.basename(image_path)))

        if float(np.random.rand(1)) > 0.5:
            image = transforms.functional.hflip(image)
            x = -x

        image = self.color_jitter(image)
        image = transforms.functional.resize(image, (224, 224))
        image = transforms.functional.to_tensor(image)
        image = image.numpy()[::-1].copy()
        image = torch.from_numpy(image)
        image = transforms.functional.normalize(image, [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])

        return image, torch.tensor([x, y]).float()

dataset = XYDataset('dataset_xy', random_hflips=False)

I replaced it with that from the NVIDIA-AI-IOT Jetbot notebook and everything worked just fine:

def get_x(path, width):
    """Gets the x value from the image filename"""
    return (float(int(path.split("_")[1])) - width/2) / (width/2)

def get_y(path, height):
    """Gets the y value from the image filename"""
    return (float(int(path.split("_")[2])) - height/2) / (height/2)

class XYDataset(torch.utils.data.Dataset):

    def __init__(self, directory, random_hflips=False):
        self.directory = directory
        self.random_hflips = random_hflips
        self.image_paths = glob.glob(os.path.join(self.directory, '*.jpg'))
        self.color_jitter = transforms.ColorJitter(0.3, 0.3, 0.3, 0.3)

    def __len__(self):
        return len(self.image_paths)

    def __getitem__(self, idx):
        image_path = self.image_paths[idx]

        image = PIL.Image.open(image_path)
        width, height = image.size
        x = float(get_x(os.path.basename(image_path), width))
        y = float(get_y(os.path.basename(image_path), height))

        if float(np.random.rand(1)) > 0.5:
            image = transforms.functional.hflip(image)
            x = -x

        image = self.color_jitter(image)
        image = transforms.functional.resize(image, (224, 224))
        image = transforms.functional.to_tensor(image)
        image = image.numpy()[::-1].copy()
        image = torch.from_numpy(image)
        image = transforms.functional.normalize(image, [0.485, 0.456, 0.406], [0.229, 0.224, 0.225])

        return image, torch.tensor([x, y]).float()

dataset = XYDataset('dataset_xy', random_hflips=False)

Not sure the reason why though, @jaybdub would you mind clarrifying?

Best, Abu