AIasd / ADFuzz

An open-source software package for fuzzing autonomous driving systems in high-fidelity simulators
MIT License
30 stars 9 forks source link

Agent '.....' is missing bridge #8

Closed YuqiHuai closed 2 years ago

YuqiHuai commented 2 years ago

Hi,

For some reason when running ADFuzz I have been seeing bridge throwing errors

Agent '9c067880-ceaa-49f5-a705-e9e6365be6ab' is missing bridge client

For the section "Other preparation" in the readme file, do we have to change every ID specified here?

    if ego_car_model == 'apollo_6_with_signal':
        model_id = '9272dd1a-793a-45b2-bff4-3a160b506d75'
    elif ego_car_model == 'apollo_6_modular':
        model_id = '2e9095fa-c9b9-4f3f-8d7d-65fa2bb03921'
    elif ego_car_model == 'apollo_6_modular_2gt':
        model_id = 'f0daed3e-4b1e-46ce-91ec-21149fa31758'
    elif ego_car_model == 'apollo_6':
        model_id = 'c354b519-ccf0-4c1c-b3cc-645ed5751bb5'
    else:
        print('ego car model is invalid:', ego_car_model)
        raise

I used to be able to run ADFuzz by using apollo_6_modular without changing anything, but could not do so today. Is it because the vehicle ID changed in LGSVL?

AIasd commented 2 years ago

Yes, you need to change the id to yours for each one you want to use.

YuqiHuai commented 2 years ago

Thanks for the answer!

YuqiHuai commented 2 years ago

I actually have another question. Until last week, apollo_6_modular with the provided ID is working, but it seems like this ID is missing this week. Is it because the LGSVL user with that car ID deleted the configuration from LGSVL?

YuqiHuai commented 2 years ago

I also have a minor suggestion, it would probably make things easier if you keep model_id all at the same place, so something like the following so we don't need to go to multiple places to make updates

vehicle_models = {
    'apollo_6_with_signal': {
        'id': '9272dd1a-793a-45b2-bff4-3a160b506d75',
        'modules': [
            'Localization',
            'Perception',
            'Transform',
            'Routing',
            'Prediction',
            'Planning',
            'Camera',
            # 'Traffic Light',
            'Control'
        ]
    },
    'apollo_6_modular': {
        'id': '2e9095fa-c9b9-4f3f-8d7d-65fa2bb03921',
        'modules': [
            'Localization',
            # 'Perception',
            'Transform',
            'Routing',
            'Prediction',
            'Planning',
            # 'Camera',
            # 'Traffic Light',
            'Control'
        ]
    },
    'apollo_6_modular_2gt': {
        'id': 'f0daed3e-4b1e-46ce-91ec-21149fa31758',
        'modules': [
            'Localization',
            # 'Perception',
            'Transform',
            'Routing',
            'Prediction',
            'Planning',
            # 'Camera',
            # 'Traffic Light',
            'Control'
        ]
    },
    'apollo_6': {
        'id': '4901c017-0c18-4522-ba61-ce76cf0f9f91',
        'modules': [
            'Localization',
            'Perception',
            'Transform',
            'Routing',
            'Prediction',
            'Planning',
            'Camera',
            'Traffic Light',
            'Control'
        ]
    }
}

def get_modules_for_id(_id: str):
    for k in vehicle_models:
        if vehicle_models[k]['id'] == _id:
            return vehicle_models[k]['modules']
    raise Exception('unknown model_id: '+ _id)

Then in svl_specific.py we have something like the snippet below rather than long if statements

    if ego_car_model not in vehicle_models:
        print('ego car model is invalid:', ego_car_model)
        raise
    model_id = vehicle_models[ego_car_model]['id']

in simulation_utils.py we can have

modules = get_modules_for_id(model_id)
AIasd commented 2 years ago

Hi @YuqiHuai , thank you for the great suggestion! I have updated the scripts as suggested.

AIasd commented 2 years ago

I believe that id ('2e9095fa-c9b9-4f3f-8d7d-65fa2bb03921' for apollo_6_modular) should be belong to the SVL official content and I can still use it... So I am not sure what happened regarding that.

YuqiHuai commented 2 years ago

Got it. I checked ID carefully yesterday and it is indeed there. What happened was, I was calling ADFuzz using apollo 6 modular, then in the try catch block that attempts to add vehicle 3 times, when the script is adding the ego car to the simulator, it says Agent id xxxxx is missing, and xxxxx is different and random for 3 tries, which is super weird.

YuqiHuai commented 2 years ago

Thanks for adding the suggested script! Very easy to add new vehicles now.

YuqiHuai commented 2 years ago

I added some print statements like following

while times < 3:
        try:
            state = lgsvl.AgentState()
            state.transform = start
            print('model_id', model_id) # added
            ego = sim.add_agent(model_id, lgsvl.AgentType.EGO, state)
            ego.connect_bridge(BRIDGE_HOST, BRIDGE_PORT)
            ego.on_collision(on_collision)
            ..........
        except Exception as e:
            print(e) # added
            print('fail to spin up apollo, try again!')
            times += 1

and got the following output

finish binding sockets
model_id c354b519-ccf0-4c1c-b3cc-645ed5751bb5
Agent 'a76fae48-8642-4a8a-a723-8ed2fe8dcb09' is missing bridge client
fail to spin up apollo, try again!
model_id c354b519-ccf0-4c1c-b3cc-645ed5751bb5
Agent 'b90edcfb-10ea-4671-b1bb-59e6faed5761' is missing bridge client
fail to spin up apollo, try again!
model_id c354b519-ccf0-4c1c-b3cc-645ed5751bb5
Agent '88480a6d-e6ab-44a6-a55a-406506222852' is missing bridge client
fail to spin up apollo, try again!

Do you know what might have gone wrong for sim.add_agent to keep using random model_id?

AIasd commented 2 years ago

The problem might be caused by the access issue. "c354b519-ccf0-4c1c-b3cc-645ed5751bb5" was created by me so you may not have access to it. Does it appear on your page? Also, did you see similar issues if you use '9272dd1a-793a-45b2-bff4-3a160b506d75' or 'f0daed3e-4b1e-46ce-91ec-21149fa31758'? Those were also created by me.

YuqiHuai commented 2 years ago

I tried 4901c017-0c18-4522-ba61-ce76cf0f9f91 which is apollo 6 created by me and I saw the same issue. Can it be LGSVL failed to retrieve vehicle information from svl cloud?

AIasd commented 2 years ago

Sorry for the late reply! You can probably try a minimal script as shown inhttps://www.svlsimulator.com/docs/python-api/dreamview-api/ with your vehicle id. If the same issue happens, you can open an issue on svl's github and see what they say about it.

YuqiHuai commented 2 years ago

Hi! Thanks for the suggestion! I guess there must be weird issues. Today I tried again and this issue no longer exists, the latency problem also does not exist anymore.