Trouble Opening a Flowgraph Scenario Using Python

[Guaziboi]

So I have a python script that opens any of the pre existing scenarios that are shipped with BeamNG and logs the user's input data and outputs it all into and Excel spreadsheet. The script runs perfectly fine with any of said scenarios. I'm at the point where I need to create my own custom scenarios to run tests on the user's reaction times and I'm going with the FlowGraph route. I have a test "scenario" called spawn_test_3 made, even clicked on "save as scenario", and moved both the spawn_test_3.json and spawn_test_3.flow.json files into the map folder they're supposed to be. However, my script can't seem to find either file which results in the script not working.

I've even tried to open downloaded scenarios from the repository and those opened up just fine. My code should be pasted below. Any ideas are appreciated. Thank you

import threading
import time
import matplotlib.pyplot as plt
import numpy as np
import openpyxl
import pandas as pd
import seaborn as sns
import os
from beamngpy import BeamNGpy, Scenario, Vehicle, angle_to_quat
from beamngpy.sensors import Electrics

sns.set()  # Make seaborn set matplotlib styling

# Instantiate a BeamNGpy instance the other classes use for reference & communication
# This is the host & port used to communicate over
# beamng = BeamNGpy('localhost', 64256)
beamng = BeamNGpy('localhost', 64256, home='C:/BeamNG.tech.v0.28.2.0', user='C:/Users/mtith2')
beamng.open()

scenario = Scenario('Utah', 'spawn_test_3.flow', path='/levels/Utah/scenarios/spawn_test_3.flow.json')

beamng.scenario.load(scenario)
# beamng.scenario.start()  # After loading, the simulator waits for further input to actually start

vehicle = next(iter(scenario.vehicles.values()))
electrics = Electrics()
vehicle.sensors.attach('electrics', electrics)

positions = []
directions = []
wheel_speeds = []
throttles = []
brakes = []
steerings = []  # New list to store steering values
elapsed_times = []  # New list to store elapsed time

vehicle.sensors.poll()
sensors = vehicle.sensors

print('The vehicle position is:')
print(vehicle.state['pos'])

print('The vehicle direction is:')
print(vehicle.state['dir'])

print('The wheel speed is:')
print(sensors['electrics']['wheelspeed'])

print('The throttle intensity is:')
print(sensors['electrics']['throttle'])

print('The brake intensity is:')
print(sensors['electrics']['brake'])

# Function to update vehicle controls based on user input
def update_controls():
    while True:
        user_input = input("Enter throttle, brake, and steering values (separated by space), or 'q' to quit: ")
        if user_input.lower() == 'q':
            # Quit the program
            beamng.close()  # End the scenario and close the connection to the simulator
            return False
        inputs = user_input.split()
        if len(inputs) != 3:
            print("Invalid input. Please enter throttle, brake, and steering values.")
            continue
        try:
            throttle = float(inputs[0])
            brake = float(inputs[1])
            steering = float(inputs[2])
            if throttle < 0 or throttle > 1 or brake < 0 or brake > 1:
                print("Throttle and brake values should be between 0 and 1.")
                continue
            if steering < -1 or steering > 1:
                print("Steering value should be between -1 and 1.")
                continue
            vehicle.control(throttle=throttle, brake=brake, steering=steering)
            break
        except ValueError:
            print("Invalid input. Throttle, brake, and steering values should be numeric.")

# Function to collect sensor data
# Function to collect sensor data
def collect_data(start_time):
    while True:
        time.sleep(0.5)
        current_time = time.time() - start_time  # Calculate the elapsed time
        vehicle.sensors.poll()  # Polls the data of all sensors attached to the vehicle
        sensors = vehicle.sensors

        position = vehicle.state['pos']
        direction = vehicle.state['dir']

        positions.append([position[0], position[1], position[2]])  # Store XYZ values separately
        directions.append([direction[0], direction[1], direction[2]])  # Store XYZ values separately
        wheel_speeds.append(sensors['electrics']['wheelspeed'])
        throttles.append(sensors['electrics']['throttle'])
        brakes.append(sensors['electrics']['brake'])
        steerings.append(sensors['electrics']['steering'])
        elapsed_times.append(current_time)  # Append the elapsed time

# Create separate threads for user input and data collection
input_thread = threading.Thread(target=update_controls)
data_thread = threading.Thread(target=collect_data, args=(time.time(),))

# Start the threads
input_thread.start()
data_thread.start()

# Wait for both threads to finish
input_thread.join()
data_thread.join()

beamng.close()  # End the scenario and close the connection to the simulator

# Create a DataFrame using the collected data
data = {
    'Elapsed Time': elapsed_times,
    'Position_X': [pos[0] for pos in positions],
    'Position_Y': [pos[1] for pos in positions],
    'Position_Z': [pos[2] for pos in positions],
    'Direction_X': [dir[0] for dir in directions],
    'Direction_Y': [dir[1] for dir in directions],
    'Direction_Z': [dir[2] for dir in directions],
    'Wheel Speed': wheel_speeds,
    'Throttle': throttles,
    'Brake': brakes,
    'Steering': steerings
}
df = pd.DataFrame(data)

# Save the DataFrame to an Excel file with a new number appended to the filename
excel_filename = 'vehicle_data'
excel_file_path = ''
counter = 1
while True:
    excel_file_path = f'C:/pythonProject9/{excel_filename}_{counter}.xlsx'
    if not os.path.exists(excel_file_path):
        break
    counter += 1

with pd.ExcelWriter(excel_file_path, engine='openpyxl') as writer:
    df.to_excel(writer, index=False)

plt.plot(throttles, 'b-', label='Throttle')
plt.plot(brakes, 'r-', label='Brake')
plt.plot(steerings, 'g-', label='Steering')
plt.legend()
plt.show()
plt.clf()

x = [p[0] for p in positions]
y = [p[1] for p in positions]
plt.plot(x, y, '.')
plt.axis('square')
plt.show()
plt.clf()

angles = [np.arctan2(d[1], d[0]) for d in directions]
r = wheel_speeds
plt.subplot(111, projection='polar')
plt.scatter(angles, r)
plt.show()

[aivora-beamng]

Hi, if you have the two flowgraph scenario files (.flow.json and .json) in the correct location (%localappdata%\BeamNG.tech\0.28\levels\Utah\scenarios, then it should suffice to change only this line:

scenario = Scenario('Utah', 'spawn_test_3.flow', path='/levels/Utah/scenarios/spawn_test_3.json')

as BeamNGpy loads the json scenario, not the .flow.json scenario.

You can also try a scenario I tested this with, extract the two files in spawn_test.zip to %localappdata%\BeamNG.tech\0.28\flowgraphEditor and then change the loading code to

scenario = Scenario('italy', 'spawn_test_3.flow', path='/flowgraphEditor/spawn_test.json')

That seems to work for me with your code. Let me know if you still have issues.

[Guaziboi]

Unfortunately still having the same issue where the script can't my files, even with your files. Could it be an issue with where BeamNG is saved in my system? The produced error codes should be below.

Traceback (most recent call last): File "C:\Users\mtith2\AppData\Roaming\JetBrains\PyCharmCE2023.1\scratches\scratch_17.py", line 23, in beamng.scenario.load(scenario) File "C:\pythonProject9\venv\Lib\site-packages\beamngpy\api\beamng\scenario.py", line 165, in load self._send(data).ack('MapLoaded') File "C:\pythonProject9\venv\Lib\site-packages\beamngpy\connection\connection.py", line 275, in ack message = self.recv() ^^^^^^^^^^^ File "C:\pythonProject9\venv\Lib\site-packages\beamngpy\connection\connection.py", line 269, in recv raise message beamngpy.logging.BNGValueError: Scenario not found: "/flowgraphEditor/spawn_test.json" 10.71800|E|GELua.TechCom| Error reading from socket: closed 10.71907|E|GELua.TechCom| Error reading from socket: tcp{client}: 000001A4B14FB028 - closed

Process finished with exit code 1

[aivora-beamng]

Ah, I see you have also overriden the userpath of the simulator, so the correct path for these files would be C:/Users/mtith2/0.28/flowgraphEditor or C:/Users/mtith2/0.28/levels/Utah/scenarios/, respectively. Please try with these paths (or remove the user=... argument.

[Guaziboi]

That opened it, thank you so much! But another problem comes up now. The rest of my code doesn't execute after the scenario is loaded up. It works just fine for any of the other pre-installed/downloaded scenarios. Is this a result of my scenario not being a competed one or does it have something to do with the script?

Edit: It also works just fine with the scenario you provided

[aivora-beamng]

Hi, can you send me your scenario file (the .flow.json file) either here or to tech@beamng.gmbh? Thank you.

[Guaziboi]

Here they are. So the main thing in the scenario is to just drive straight through a trigger to spawn an obstacle. That part works fine but the script stops doing its thing after the scenario is loaded. For example, if the user input is "q" in the run window, the script stops and shows all the data it collected. This doesn't happen when running the scenario.

On Thu, Jul 27, 2023 at 3:59 AM Adam Ivora @.***> wrote:

Hi, can you send me your scenario file (the .flow.json file) either here or to @.***? Thank you.

— Reply to this email directly, view it on GitHub https://github.com/BeamNG/BeamNGpy/issues/228#issuecomment-1653194080, or unsubscribe https://github.com/notifications/unsubscribe-auth/BAXV4FFYZR4XOMUVWJUX443XSIUWNANCNFSM6AAAAAA2TJH3XY . You are receiving this because you authored the thread.Message ID: @.***>

[aivora-beamng]

Hi, I don't see the file, unfortunately. GitHub should allow to upload a zip archive including the flow.json file to this thread, so you may try that?

[Guaziboi]

My bad, they should be attached now. SpawnV2Scenario.zip

[aivora-beamng]

Hi, try this version, I added a custom loading screen to the Flowgraph. It looks that sometimes BeamNGpy hangs when the custom screen is not there, we will look into that.

SpawnV2Scenario.zip

Let me know how this one works for you :)

[Guaziboi]

It still doesn't seem to work. And I think the file my be the same one I sent. Also, how would I set up the flow graph so that when I press R the whole thing resets?

[aivora-beamng]

Hi, sorry, I must have mixed up the files somehow. Try this one: SpawnV2Scenario_updated.zip

Or edit your flowgraph as in this screenshot (check the "customLoadingScreen" checkbox and then add the "Hide the loading screen" node):

I also had to edit the script a little bit to not crash, but I managed to get these plots you are making to show with this version and that scenario:

import os
import threading
import time

import matplotlib.pyplot as plt
import numpy as np
import openpyxl
import pandas as pd
import seaborn as sns
from beamngpy import BeamNGpy, Scenario, Vehicle, angle_to_quat
from beamngpy.sensors import Electrics

sns.set()  # Make seaborn set matplotlib styling

# Instantiate a BeamNGpy instance the other classes use for reference & communication
# This is the host & port used to communicate over
beamng = BeamNGpy('localhost', 64256, home='C:/BeamNG.tech.v0.28.2.0', user='C:/Users/mtith2')
beamng.open()

scenario = Scenario('Utah', 'spawn_test_3.flow', path='/flowgraphEditor/SpawnV2Scenario.json')

beamng.scenario.load(scenario)
beamng.scenario.start()  # After loading, the simulator waits for further input to actually start

vehicle = next(iter(scenario.vehicles.values()))
electrics = Electrics()
vehicle.sensors.attach('electrics', electrics)

positions = []
directions = []
wheel_speeds = []
throttles = []
brakes = []
steerings = []  # New list to store steering values
elapsed_times = []  # New list to store elapsed time

vehicle.sensors.poll()
sensors = vehicle.sensors

print('The vehicle position is:')
print(vehicle.state['pos'])

print('The vehicle direction is:')
print(vehicle.state['dir'])

print('The wheel speed is:')
print(sensors['electrics']['wheelspeed'])

print('The throttle intensity is:')
print(sensors['electrics']['throttle'])

print('The brake intensity is:')
print(sensors['electrics']['brake'])

# Function to update vehicle controls based on user input
def update_controls():
    global vehicle
    while True:
        user_input = input("Enter throttle, brake, and steering values (separated by space), 'r' to restart, or 'q' to quit: ")
        if user_input.lower() == 'r':
            beamng.scenario.restart()
            vehicle = next(iter(scenario.vehicles.values()))
            electrics = Electrics()
            vehicle.sensors.attach('electrics', electrics)
            continue
        if user_input.lower() == 'q':
            # Quit the program
            beamng.close()  # End the scenario and close the connection to the simulator
            return False
        inputs = user_input.split()
        if len(inputs) != 3:
            print("Invalid input. Please enter throttle, brake, and steering values.")
            continue
        try:
            throttle = float(inputs[0])
            brake = float(inputs[1])
            steering = float(inputs[2])
            if throttle < 0 or throttle > 1 or brake < 0 or brake > 1:
                print("Throttle and brake values should be between 0 and 1.")
                continue
            if steering < -1 or steering > 1:
                print("Steering value should be between -1 and 1.")
                continue
            vehicle.control(throttle=throttle, brake=brake, steering=steering)
        except ValueError:
            print("Invalid input. Throttle, brake, and steering values should be numeric.")

# Function to collect sensor data
def collect_data(start_time):
    while True:
        time.sleep(0.5)
        current_time = time.time() - start_time  # Calculate the elapsed time
        if vehicle.is_connected():
            vehicle.sensors.poll()  # Polls the data of all sensors attached to the vehicle
        sensors = vehicle.sensors

        position = vehicle.state['pos']
        direction = vehicle.state['dir']

        positions.append([position[0], position[1], position[2]])  # Store XYZ values separately
        directions.append([direction[0], direction[1], direction[2]])  # Store XYZ values separately
        wheel_speeds.append(sensors['electrics']['wheelspeed'])
        throttles.append(sensors['electrics']['throttle'])
        brakes.append(sensors['electrics']['brake'])
        steerings.append(sensors['electrics']['steering'])
        elapsed_times.append(current_time)  # Append the elapsed time

# Create separate threads for user input and data collection
input_thread = threading.Thread(target=update_controls)
data_thread = threading.Thread(target=collect_data, args=(time.time(),))

# Start the threads
input_thread.start()
data_thread.start()

# Wait for both threads to finish
input_thread.join()
data_thread.join()

beamng.close()  # End the scenario and close the connection to the simulator

# Create a DataFrame using the collected data
data = {
    'Elapsed Time': elapsed_times,
    'Position_X': [pos[0] for pos in positions],
    'Position_Y': [pos[1] for pos in positions],
    'Position_Z': [pos[2] for pos in positions],
    'Direction_X': [dir[0] for dir in directions],
    'Direction_Y': [dir[1] for dir in directions],
    'Direction_Z': [dir[2] for dir in directions],
    'Wheel Speed': wheel_speeds,
    'Throttle': throttles,
    'Brake': brakes,
    'Steering': steerings
}
df = pd.DataFrame(data)

# Save the DataFrame to an Excel file with a new number appended to the filename
excel_filename = 'vehicle_data'
excel_file_path = ''
counter = 1
while True:
    excel_file_path = f'{excel_filename}_{counter}.xlsx'
    if not os.path.exists(excel_file_path):
        break
    counter += 1

with pd.ExcelWriter(excel_file_path, engine='openpyxl') as writer:
    df.to_excel(writer, index=False)

plt.plot(throttles, 'b-', label='Throttle')
plt.plot(brakes, 'r-', label='Brake')
plt.plot(steerings, 'g-', label='Steering')
plt.legend()
plt.show()
plt.clf()

x = [p[0] for p in positions]
y = [p[1] for p in positions]
plt.plot(x, y, '.')
plt.axis('square')
plt.show()
plt.clf()

angles = [np.arctan2(d[1], d[0]) for d in directions]
r = wheel_speeds
plt.subplot(111, projection='polar')
plt.scatter(angles, r)
plt.show()

With regards to your second question, that depends whether you mean pressing R in the Python console (1.), or inside the simulator (2.): 1) I added the support for that in your script, it should be enough to call beamng.scenario.restart(). However, the vehicle is respawned with a different connection port, so you need to reconnect it. We will try to make this experience simpler in the future updates. 2) You can add a "Restart project" node to the project that is activated by a "Flowgraph Custom Action": That will make the Flowgraph scenarios restart on Ctrl+Space by default, but you can add a custom binding for this action in the settings.

[Guaziboi]

Thank you! Mostly everything seems to be working properly. However, python will sometimes connect to the ball and collect its data instead of the car. I've tried to delay the ball spawning so that the script can only connect to the car, but no luck. I think that this line needs to be modified

25) vehicle = next(iter(scenario.vehicles.values()))

[aivora-beamng]

Ah, I see. You should be able to fix that by specifying the name of the vehicle in the Flowgraph scenario like this (I set the name to player_vehicle, but you can set it to any value you want):

Then, instead of vehicle = next(iter(scenario.vehicles.values())) which picks the first vehicle that is available, you can access the specified vehicle by name using:

vehicle = scenario.vehicles['player_vehicle']

[Guaziboi]

My bad, a little late on the response. Everything works as expected. Thank you so much!

[lsm2842035890]

hello friends,I loaded the gridsmall map and paved a simple road. I want the vehicle to run according to my trajectory. Why does the vehicle not move?