Open andreyvelich opened 1 month ago
@andreyvelich: This request has been marked as suitable for new contributors.
Please ensure the request meets the requirements listed here.
If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-good-first-issue
command.
Hello @andreyvelich I am working on writing unit test for get_job, so should I seperate the test_data both for create_job and get_job individually or combine them into single test_data by adding one more identifier to each sample which tells for which function ( create_job or get_job) the test is for?
Example test data -
( "create_job_valid_flow_to_create_job_using_image", "create_job", { "name": "test-job", "namespace": "test", "base_image": "docker.io/test-training", "num_workers": 2, }, "success", )
( "get_job_valid", "get_job", {"namespace": "test", "job_name": "valid_job"}, {"mock": "job_response"} ),
Thank you for your contribution @ayushrakesh!
I think it is easier to separate tests data for create_job
and get_job
API, so it will be easier for us to maintain those lists.
WDYT @droctothorpe @deepanker13 @kubeflow/wg-training-leads ?
Thank you for your contribution @ayushrakesh!
I think it is easier to separate tests data for
create_job
andget_job
API, so it will be easier for us to maintain those lists.WDYT @droctothorpe @deepanker13 @kubeflow/wg-training-leads ?
As my understanding, this is unit tests, not integration tests. So, dedicated test data every for functions would be better.
In general, if you can recycle fixtures to reduce redundancy, that's usually advisable; whether or not that's possible depends on the expected IO of the two functions; create_jobs
doesn't return anything (though maybe it should), so I'm not sure if it makes sense to consolidate the fixtures in this context.
@andreyvelich Thank you for guiding me in contributing to Training-Operator and Katib.
I'll write some test cases for training_client_test.py
as soon as I understand the whole logics and the source code of Training-Operator.
Thank you @Electronic-Waste.
So, maybe @ayushrakesh can create unit test for get_job
API and @Electronic-Waste can work on
wait_for_job_conditions
API.
I'm interested in working on the unit test for get_job_pods, any objection?
Sure, thank you for your time @YosiElias! /assign @ayushrakesh @Electronic-Waste @YosiElias
Having a different list will make code more readable, Also should we create a folder and have a test file for each function to make it more organised? @andreyvelich
Having a different list will make code more readable, Also should we create a folder and have a test file for each function to make it more organised? @andreyvelich
@deepanker13 Usually, test file have the same name as the actual file with _test
suffix,
E.g. you can check the KFP client example: https://github.com/kubeflow/pipelines/tree/master/sdk/python/kfp/client.
Will the number of test cases be too many to keep in a single training_client_test
module?
I was thinking of moving test data, fixtures and utility functions from the katib_client_test module, for example, and putting these in newly created test_data
, conftest
and utils
modules.
That way, I was thinking that the code may be easier to maintain in the katib_client_test
module.
And katib_client_test
and its associated test_data
, conftest
and utils
modules would be moved to a newly created sdk/python/v1beta1/kubeflow/katib/tests
folder.
Please let me know your thoughts.
@tariq-hasan I think, we discussed before with @droctothorpe that we want to keep _test
file alongside with actual files: https://github.com/kubeflow/katib/issues/2184#issuecomment-1660475384.
I understand that this cause file to grow, but we are already doing the same for go unit tests. For example: https://github.com/kubeflow/katib/blob/master/pkg/webhook/v1beta1/experiment/validator/validator_test.go.
I feel that the test data should be associated with function that we test: get_job
, create_job
, train
, etc.
KFP follows the same way: https://github.com/kubeflow/pipelines/tree/master/sdk/python/kfp/client.
Any concerns you see with that @tariq-hasan ?
That sounds good. Thanks for the clarification.
I'm interested in working on the unit test for get_job_pod_names and update_job, any objection?
Absolutely, thank you @YosiElias!
@andreyvelich UTs for wait_for_job_conditions
have been added now! PTALđź‘€ when you are available.
Also, can I work on get_job_conditions
and related functions whose names begin with is_job_
?
Hey @andreyvelich,
I can take a look at get_job
and delete_job
We need to add more unit tests for
TrainingClient
APIs by updating thetraining_client_test.py
. That should help us to detect bugs like this one: https://github.com/kubeflow/training-operator/pull/2160. Let's use this issue to track unit tests for various public APIs.Feel free to submit PR to add unit tests for one of the following APIs, just comment on this issue on which unit tests you are working on:
create_job
wait_for_job_conditions
https://github.com/kubeflow/training-operator/pull/2196get_job
https://github.com/kubeflow/training-operator/pull/2205list_jobs
get_job_conditions
is_job_created
is_job_running
is_job_restarting
is_job_succeeded
is_job_failed
get_job_pods
https://github.com/kubeflow/training-operator/pull/2175get_job_pod_names
https://github.com/kubeflow/training-operator/pull/2192get_job_logs
update_job
https://github.com/kubeflow/training-operator/pull/2192delete_job
https://github.com/kubeflow/training-operator/pull/2232cc @kubeflow/wg-training-leads @deepanker13 @Electronic-Waste @tariq-hasan
/good-first-issue /area sdk /area testing