[Good First Issue]: Enable parallel execution of TensorFlow Layer 1 python tests

mryzhov commented 11 months ago

Context

Current CI (https://github.com/openvinotoolkit/openvino/blob/c0381ab08d2fcdc51c621196e935aa02c64a4b50/.github/workflows/linux.yml#L869) runs all the python tests sequentially one by one and it takes too much time. The idea is to run all those tests in parallel by using pytest-xdist plugin, but tests should be modified for that. Please find bellow the common tests issues:

Random inputs generation as parametrization arguments. Solution: all random test arguments should be generated inside of the tests
Tests create, use and remove the same artifacts (temporary data, models, etc). Hot fix solution: all test data should be generated in the system temporary directory (TMPDIR) which can be unique for each thread. Long term solution: all test data should be generated in the unique temporary directory or with unique names.

What needs to be done?

Modify TensorFlow Layer 1 tests to solve issues described in the Context. Tests are located here - https://github.com/openvinotoolkit/openvino/tree/master/tests/layer_tests/tensorflow_tests
Enable parallel execution of those test in the CI (https://github.com/openvinotoolkit/openvino/blob/c0381ab08d2fcdc51c621196e935aa02c64a4b50/.github/workflows/linux.yml#L1048)

Example Pull Requests

https://github.com/openvinotoolkit/openvino/pull/20613

Resources

Contact points

@mryzhov @akashchi

Ticket

No response

rajatkrishna commented 10 months ago

.take

github-actions[bot] commented 10 months ago

Thank you for looking into this issue! Please let us know if you have any questions or require any help.

p-wysocki commented 10 months ago

Hello @rajatkrishna! Thank you for taking a look, please let us know if you have any questions. Just yesterday our CONTRIBUTING.md has been updated with a technical guide - I highly recommend checking it out. :)

p-wysocki commented 10 months ago

Hi @rajatkrishna, are you still working on that issue?

I am happy to announce that we have created a channel dedicated to Good First Issues support on our Intel DevHub Discord server! Join it to receive support, engage in discussions, ask questions and talk to OpenVINO developers.

rajatkrishna commented 10 months ago

Hey @p-wysocki, thanks for letting me know about the support channel on the Intel DevHub Discord server. I'm currently facing an issue building the Python wheel. The build fails with the following error:

[100%] Run 'fdupes' checks for wheel openvino-2024.0.0-13858-cp310-cp310-manylinux_2_35_x86_64.whl
CMake Error at /home/rajat/gsoc/openvino/src/bindings/python/wheel/fdupes_check.cmake:61 (message):

  /home/rajat/gsoc/openvino/build/src/bindings/python/wheel/openvino-2024.0.0/openvino/tools/benchmark/__init__.py

make[2]: *** [src/bindings/python/wheel/CMakeFiles/ie_wheel.dir/build.make:74: src/bindings/python/wheel/fdupes_report.txt] Error 1
make[2]: *** Deleting file 'src/bindings/python/wheel/fdupes_report.txt'
make[1]: *** [CMakeFiles/Makefile2:4675: src/bindings/python/wheel/CMakeFiles/ie_wheel.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

Would the Discord channel be the right place to bring this up and get some help troubleshooting?

p-wysocki commented 9 months ago

The person who could be asked about this error (the check author) is currently on vacation, but I've faced similar errors before - you can implement a workaround by commenting out these lines: https://github.com/openvinotoolkit/openvino/blob/master/src/bindings/python/wheel/fdupes_check.cmake#L60-L62

I wouldn't worry about it too much, just don't commit the workaround, it should work just fine in our CI.

Would the Discord channel be the right place to bring this up and get some help troubleshooting?

These type of questions are always welcome on Discord. :) Feel free to join and ask us anything - we're there to help. Chat may be more convenient than GitHub Issues.

openvinotoolkit / openvino