✨ Surge GPU access

[bcrawford-moj]

Describe the feature request.

Kim Brett (@K1Br) and I are running a team office day where we will set our team's a challenge involving using large language models on the AP. There will be 10-15 participants who will all want to use GPU simultaneously, for approximately an hour in the morning (around 10am) and an hour in the afternoon (around 3pm). It will be on Wednesday 11th September.

@julialawrence - tagging you for sight as per our discussion.

Describe the context.

No response

Value / Purpose

Team building and learning and development in the field of prompt engineering and using large language models on the AP.

User Types

Data scientists

[julialawrence]

@jacobwoffenden and I brought up this in our conversation with AWS regarding instance size capacity and one of the way suggested we ensure sufficient capacity is available is by spinning the needed instances the day before. I think that should be fairly trivial. We can set the minimum on the ASG to 5-6 the night before which should stop the cluster autoscaler from scaling then down.

[julialawrence]

Good morning @bcrawford-moj. We have set up a new release called Visual Studio Code 2.0.0 (GPU) (Hackathon) for you to use tomorrow. The maximum node count of our scaling group is 10 which, as each node can accommodate 4 users, covers you up to 40 participants. This release is going to be available only for the Hackathon. If any users want access to GPU-enabled compute after the hackathon, they'll need to raise a support request. We have bumped the initial size of the node group to 2 to ease the ramp up a bit but nodes take a bit of time to scale, so might be worthwhile getting people spinning up their tooling as soon as they get on site. Let us know if you encounter any issues.

[bcrawford-moj]

Thanks @julialawrence!

[parminder-thindal-moj]

@julialawrence piggy backing off this request, could this instance be recreated by September 30th?

Myself, @laura-williams2 @vmohanram had a clash when this team hackathon day took place, and have set aside some time on September 30th to go through the exercises.

[julialawrence]

If this is just 3 of you you could raise a support request to get access to the GPU backed vscode since 3 people can be accommodated with our default capacity.

[parminder-thindal-moj]

Sorry it will 5 of us now.

• myself - @parminder-thindal-moj
• @laura-williams2
• @vmohanram
• @lewissheppard
• @ldmoj4

Do you still want me to raise a support request?

Thanks

[julialawrence]

Apologies for missing this. 5 people could be accommodated using the support request. So if you could raise one please you should get acceaa.

[simon-pope]

@julialawrence @parminder-thindal-moj with the support request raised can this Feature Request be closed? thanks