autowarefoundation / autoware.universe

https://autowarefoundation.github.io/autoware.universe/
Apache License 2.0
964 stars 630 forks source link

Tracking tests disabled due to lack of gpu capable machines #7724

Open xmfcx opened 3 months ago

xmfcx commented 3 months ago

Checklist

Description

Story behind can be tracked from:

Basically the tests disabled here:

When we upgrade our testing infrastructure to have CUDA capable GPUs, we can re-enable these tests.

cc. @knzo25 @mitsudome-r

Sister issue:

Purpose

To make CI pass on no-gpu machines. (Especially github hosted runners)

Possible approaches

Definition of done

Table of tracked tests

Link to commented section Package Name Related PR
https://github.com/autowarefoundation/autoware.universe/blob/e96334ceb5d31e263d2f7652760a2632366654ba/perception/lidar_centerpoint/CMakeLists.txt#L163-L167 lidar_centerpoint https://github.com/autowarefoundation/autoware.universe/pull/7722
https://github.com/autowarefoundation/autoware.universe/blob/891b23172fa7a8e98bf368a7f2049bccf7ccb15e/perception/lidar_centerpoint/CMakeLists.txt#L169-L207 lidar_centerpoint https://github.com/autowarefoundation/autoware.universe/pull/7731
xmfcx commented 3 months ago

@mitsudome-r if somehow we gain access to gpu capable machines either by:

We can use this PR as reference:

I will also create a documentation on how I set up the self hosted runners with custom init scripts.

stale[bot] commented 1 month ago

This pull request has been automatically marked as stale because it has not had recent activity.

knzo25 commented 1 month ago

@xmfcx Is there any updates on this issue? We would like to have the cuda tests enabled (https://github.com/autowarefoundation/autoware.universe/blob/main/perception/autoware_lidar_centerpoint/CMakeLists.txt)

xmfcx commented 3 weeks ago

@knzo25 please have a peek at:

Through this new system, we have access to gpu1.small machines with 15 GB, 4vCPU and an unknown GPU, I didn't make any tests.

Full list: https://aws.amazon.com/codebuild/pricing/ (select us east n. virginia)

But this machine is pricier.

We only need GPU for tests, not for building with CUDA.

So one option would be to separate build and test in the workflow and test on this new machine.

But it requires someone to work on this task.