Error while running FWE demo on Raspberry Pi

[totino5]

Describe the bug While trying to reproduce the demo with Raspberry Pi. Everything runs fine until the command prompt output this:

An error occurred (ValidationException) when calling the Query operation: line 1:78: Column 'vehicleName' does not exist

Note: If you discover a potential security issue in this project we ask that you notify AWS/Amazon Security via our vulnerability reporting page. Please do not create a public github issue.

To Reproduce (https://github.com/aws/aws-iot-fleetwise-edge/blob/main/docs/rpi-tutorial/raspberry-pi-tutorial.md#step-6-deploy-a-campaign-to-the-raspberry-pi) I followed this guide step by step.

Expected behavior Timestream is populated with data and is queryable

Logs

ubuntu@ip-xxx-xxx-xxx:~/aws-iot-fleetwise-edge/tools/cloud$ ./demo.sh --vehicle-name fwdemo-rpi --campaign-file campaign-obd-heartbeat.json
===================================
AWS IoT FleetWise Cloud Demo Script
===================================
Date: 2024-09-17T15:25:30+0000
Disambiguator: caaf0f71
Vehicle name: fwdemo-rpi
Fleet Size: 1
Vehicles: fwdemo-rpi
Checking AWS CLI version...
aws-cli/2.17.52
Getting AWS account ID...
339713082024
Getting account registration status...
REGISTRATION_SUCCESS
Account is already registered
Creating Timestream database...
arn:aws:timestream:us-east-1:339713082024:database/IoTFleetWiseDB-caaf0f71
Creating Timestream table...
Creating service role...
arn:aws:iam::339713082024:role/IoTFleetWiseServiceRole-us-east-1-caaf0f71
Waiting for role to be created...
Creating service role policy...
arn:aws:iam::339713082024:policy/IoTFleetWiseServiceRole-us-east-1-caaf0f71-policy
Waiting for policy to be created...
Attaching policy to service role...
Deleting vehicle fwdemo-rpi if it already exists...
arn:aws:iotfleetwise:us-east-1:339713082024:vehicle/fwdemo-rpi
Checking for existing signal catalogs...
Reusing existing signal catalog: arn:aws:iotfleetwise:us-east-1:339713082024:signal-catalog/fwdemo-7fda63ff-signal-catalog
Updating Vehicle node in signal catalog...
arn:aws:iotfleetwise:us-east-1:339713082024:signal-catalog/fwdemo-7fda63ff-signal-catalog
Updating OBD signals in signal catalog...
arn:aws:iotfleetwise:us-east-1:339713082024:signal-catalog/fwdemo-7fda63ff-signal-catalog
Updating DBC signals in signal catalog...
arn:aws:iotfleetwise:us-east-1:339713082024:signal-catalog/fwdemo-7fda63ff-signal-catalog
Updating color attribute
arn:aws:iotfleetwise:us-east-1:339713082024:signal-catalog/fwdemo-7fda63ff-signal-catalog
Creating model manifest...
arn:aws:iotfleetwise:us-east-1:339713082024:model-manifest/fwdemo-rpi-caaf0f71-model-manifest
Updating attribute in model manifest...
arn:aws:iotfleetwise:us-east-1:339713082024:model-manifest/fwdemo-rpi-caaf0f71-model-manifest
Activating model manifest...
arn:aws:iotfleetwise:us-east-1:339713082024:model-manifest/fwdemo-rpi-caaf0f71-model-manifest
Creating decoder manifest with OBD signals...
arn:aws:iotfleetwise:us-east-1:339713082024:decoder-manifest/fwdemo-rpi-caaf0f71-decoder-manifest
Adding DBC signals to decoder manifest...
arn:aws:iotfleetwise:us-east-1:339713082024:decoder-manifest/fwdemo-rpi-caaf0f71-decoder-manifest
Activating decoder manifest...
arn:aws:iotfleetwise:us-east-1:339713082024:decoder-manifest/fwdemo-rpi-caaf0f71-decoder-manifest
Waiting for decoder manifest to become active...
arn:aws:iotfleetwise:us-east-1:339713082024:decoder-manifest/fwdemo-rpi-caaf0f71-decoder-manifest
Creating vehicle fwdemo-rpi...
Creating fleet...
arn:aws:iotfleetwise:us-east-1:339713082024:fleet/fwdemo-rpi-caaf0f71-fleet
Associating vehicle fwdemo-rpi...
Creating campaign from campaign-obd-heartbeat.json...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-caaf0f71-campaign
Waiting for campaign to become ready for approval...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-caaf0f71-campaign
Approving campaign...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-caaf0f71-campaign
Suspending campaign...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-caaf0f71-campaign
Resuming campaign...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-caaf0f71-campaign
Waiting until status of vehicle fwdemo-rpi is healthy...
Waiting 30 seconds for data to be collected...
The DB Name is IoTFleetWiseDB-caaf0f71
The DB Table is VehicleDataTable
Querying Timestream for vehicle fwdemo-rpi...

An error occurred (ValidationException) when calling the Query operation: line 1:78: Column 'vehicleName' does not exist

Environment (please complete the following information):

• Local Machine: Windows
• Development Machine: Amazon EC2 from CloudFormation stack template shown in the guide
• Raspberry Pi 4 Model B
• 2-Channel Isolated CAN Bus Expansion HAT For Raspberry Pi, MCP2515 + SI65HVD230(or SN65HVD230)

Additional context Development machine is intended like Amazon EC2 instance? Or the Raspberry? I am also a bit confused about that.

[olgaakulinushkina]

Hello @totino5

This error indicates that Edge agent isn't collecting any data. If you check the Edge logs, you most likely won't see the log indicating the upload:

FWE data ready to send <...>

Let me provide more details about the setup. The development machine is basically needed to build the Edge agent code, set it up and use AWS CLI to use the FleetWise service. It can either be an EC2 from the guide example or a local Intel x86_64 (amd64) machine.

The Edge agent is then deployed to the Raspberry pi and runs there. It consumes data over CAN from the data source, that for that particular guide is supposed to be a real vehicle. You could also try to simulate signals demoed in this guide on Raspberry pi.

However, if you want to adapt the demo to your use case and your device, you would need to create fleetwise resources differently to suit your setup (perhaps use your custom DBC file).

Let me know if I can support further.

[totino5]

Hi @olgaakulinushkina

Thank you for the reply. Very new to this topic but now I understand the role of the developmente machine. So it is basically used to compile and set up Edge Agent. Next, the zip file produced from the EC2 (in my case) is transferred to the Raspberry for deployment on the board, am I right?

I don't have a real vehicle in my case, so I need simulated signals, but I don't understand how to get these signals from the guide. Could I use the tutorial here? https://docs.aws.amazon.com/iot-fleetwise/latest/developerguide/fleetwise-getting-started.html . Or am I missing something from the tutorial on the Raspberry to get the simulated signals?

Thanks for your support.

[hefroy]

Hey @totino5, you could setup a CAN simulator also on the Raspberry Pi board, to test the setup before you were to connect it with a real vehicle. To do that you can do the following:

1. Physically connect the CAN0 and CAN1 interfaces together on the CAN Bus Expansion HAT using two pieces of wire. (This will allow CAN messages to be sent on the can0 interface, otherwise without physical acknowledgements nothing can be sent.)
   • Connect CAN0 H to CAN1 H
   • Connect CAN0 L to CAN1 L
2. On the Raspberry Pi, install the simulator dependencies:

   sudo apt update
   sudo apt install -y python3 python3-pip
   pip3 install \
   wrapt==1.10.0 \
   cantools==36.4.0 \
   prompt_toolkit==3.0.21 \
   python-can==3.3.4 \
   can-isotp==1.7 \
   matplotlib==3.4.3

3. Run the simulator on the can0 interface:

   cd ~/aws-iot-fleetwise-deploy/tools/cansim
   python3 cansim.py -i can0

4. Try running the demo again, in the FWE log you should see FWE data ready to send messages.

[totino5]

Thank you very much @hefroy . I will try these instructions in the next day! I was missing the simulator part that you suggets.

[hefroy]

No problem, thanks for the feedback - we will add the above advice to the RPi guide. I'll close this ticket for now, but feel free to re-open it if you have any more issues.

[totino5]

Hi, thank you again for your suggestions, now the demo works properly. Just two other things I want to ask:

1. Where to check FWE logs? Or these are the logs generated in the command prompt where I launch the demo?

2. I lanched the demo 3 times with the simulator but no data are visible on the HTML file generated. Any advice on this?

[olgaakulinushkina]

Hi @totino5

1. To check Edge logs, run

sudo journalctl -fu fwe@0 --output=cat

on raspebery pi. (step 7 here)

2. Do you get the data in timestream? You can check under folder collected_data in the demo folder ~/aws-iot-fleetwise-edge/tools/cloud. Collected and downloaded samples should be there. Can you confirm whether anything is in TS and downloaded?

[totino5]

Hi @olgaakulinushkina

Thanks again, I can see logs with your suggestions. After running the demo again, I can confirm that there are two files on ~/aws-iot-fleetwise-edge/tools/cloud/collected-data. One is "fwdemo-rpi.html", the other is "fwdemo-rpi-timestream-result.json". The JSON file is populated with data, so there might be a problem with HTML one. I can also see the data on TimeStream in the console.

[totino5]

Hi, thanks for your previous help.

Now I am going to test this setup but connecting the CAN Hat module to an Invidia reComputer Jetson that supports CAN (so real data, not simulated like before) and that is actually used as TCU in a small robot capable of autonomous drive. I have a custom .dbc file and I want to receive data relative to the IMU sensor in the Jetson.

Again, sorry but I am very newbie to this topic, so can you provide detailed instructions on how to setup the raspberry tutorial with real vehicle and my .dbc file (or also "hscan.dbc" present in the guide) ?

I also created signals catalog, vehicle model and campaign relative to my .dbc file on the AWS console.

[VB011201]

Hi @totino5 , so you can try this:

[Step 6: Deploy a campaign to the Raspberry Pi] In its step 2 just add your custom .dbc file with the argument --dbc-file <path to file> and custom-campaign.json with argument like --campaign-file <path to campaign file>, some thing like : ./demo.sh --vehicle-name fwdemo-rpi --campaign-file custom-campaign.json --dbc-file custom_file.dbc

Note: custom-campaign.json needs to be written which contains the campaign with the signals present based on custom_file.dbc

[totino5]

Hi @VB011201, thanks for your instructions.

I imported my .dbc file and write custom-campaign.json. My campaign file is like campaign-obd-heartbeat.json except for the signals that I want to read from my vehicle. The JSON file is this:

{
    "compression": "SNAPPY",
    "diagnosticsMode": "SEND_ACTIVE_DTCS",
    "spoolingMode": "TO_DISK",
    "collectionScheme": {
        "timeBasedCollectionScheme": {
            "periodMs": 10000
    }
  },
    "signalsToCollect": [
        {
            "name": "AngularSpeedX"
        },
        {
            "name": "AngularSpeedY"
        },
        {
            "name": "AngularSpeedZ"
        },
        {
            "name": "LateralAcceleration"
        },
        {
            "name": "LongitudinalAcceleration"
        },
        {
            "name": "VerticalAcceleration"
        },
        {
            "name": "Roll"
        },
        {
            "name": "Pitch"
        },
        {
            "name": "Yaw"
        }
    ]
}

I also added both files to ../aws-iot-fleetwise-edge/tools/cloud but when I run the demo script, I get this error:

...
...
Creating campaign from campaign-config.json...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-9cbfceb0-campaign
Waiting for campaign to become ready for approval...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-9cbfceb0-campaign
Approving campaign...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-9cbfceb0-campaign
Suspending campaign...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-9cbfceb0-campaign
Resuming campaign...
arn:aws:iotfleetwise:us-east-1:339713082024:campaign/fwdemo-rpi-9cbfceb0-campaign
Waiting until status of vehicle fwdemo-rpi is healthy...
Waiting 30 seconds for data to be collected...
The DB Name is IoTFleetWiseDB-9cbfceb0
The DB Table is VehicleDataTable
Querying Timestream for vehicle fwdemo-rpi...

An error occurred (ValidationException) when calling the Query operation: line 1:78: Column 'vehicleName' does not exist

After tried to inspect this issue, I think there might be some problem linking my vehicle information to Timestream but I am not able to fix this. Any help on this?

[VB011201]

@totino5 thanks for sharing the logs. Can you please verify if you are starting your edge agent?

[totino5]

Hi @VB011201 ,

Yes it's running because if I launch:

sudo journalctl -fu fwe@0 --output=cat

I can see:

[INFO ] [AwsIotConnectivityModule.cpp:204] [operator()()]: [Connection completed successfully. ClientId: fwdemo-rpi, SessionExpiryIntervalSec: 3600, ServerKeepAliveSec: 60, RejoinedSession: false]

[olgaakulinushkina]

Hi @totino5,

can you see any CHECKIN logs from edge agent? if yes, can you share them? At this step they should have the decoder manifest listed there

[totino5]

Hi @olgaakulinushkina ,

Do you mean CHECKIN logs generated when launching sudo journalctl -fu fwe@0 --output=cat ?

[olgaakulinushkina]

yes

[totino5]

No, I can’t see any CHECKIN logs while running the demo script

[totino5]

Hi all, I can confirm that after cleaning some resource on the console and re-running the whole tutorial, I can finally see data on Timestream. Thank you everyone for the support :)