search

deepjavalibrary / djl-serving

A universal scalable machine learning model deployment solution
Apache License 2.0
182 stars 59 forks source link

[ci][fix] don't use env vars for llm integ test as it causes issues w… #2068

Closed siddvenk closed 3 weeks ago

siddvenk commented 3 weeks ago

…ith later tests

Description

Moving the gpt2 test to serving.properties - using environment variables persists to following tests and causes multiple models to be loaded.

We already have a gpt2 config that matches the env vars, so just using that.

See this test run https://github.com/deepjavalibrary/djl-serving/actions/runs/9518222203/job/26238764686. The mpt test is loading gpt2 model first, causing oom when mpt test runs