search

VowpalWabbit / vowpal_wabbit

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques such as online, hashing, allreduce, reductions, learning2search, active, and interactive learning.
https://vowpalwabbit.org
Other
8.49k stars 1.93k forks source link

Running of example test in the Code #2343

Closed RituRajSingh878 closed 3 years ago

RituRajSingh878 commented 4 years ago

I am opening this issue for the implementation of scripts so that we can run the test directly by some command(maybevw runalltest )and in this test, it should run the example test as well as the test file. (Currently we can't test examples by this command python -m pytest ./python/tests/) example test - https://github.com/VowpalWabbit/vowpal_wabbit/blob/bcd47ad60dabba63504b6fd8c691b20f1624caf5/python/vowpalwabbit/sklearn_vw.py#L697

We are adding an example test in the code for understanding purposes. Still, it can also be helpful to test the code by addition of unit tests so any user can understand the code easily, and also any developer can understand the code and its usage. It will be helpful in the dubbing or addition of the code because we can see that what we are loosing from the addition/removal of the code by running the test and checking its failing. The example test is more specific about that function or the class in which it is defined so it will be helpful for the above issues.

I will make a pr for the same after the discussion.

jackgerrits commented 4 years ago

Is the ask here to run Python examples as part of CI? I don't understand the issue with ipython? If there are two issues, please open them separately so they can be tracked easier.

RituRajSingh878 commented 4 years ago

It would be great to run the example test on Cl. Also, I am saying for the developer if we have some command for testing on a local machine it will be great. And with that command, we can test the test file and the example present in the code.

Smit-create commented 4 years ago

This may be related https://www.sphinx-doc.org/en/master/usage/extensions/doctest.html

jackgerrits commented 4 years ago

So the feature request is to be able to test that examples in documentation are valid as part of CI?

RituRajSingh878 commented 4 years ago

So the feature request is to be able to test that examples in documentation are valid as part of CI?

yes

RituRajSingh878 commented 4 years ago

You have opened a pr for the running doc on cl for every commits. After your pr, I will add the doc8 style check for the doc(I will learn about this as I am not familiar with cl).

What is checked:
    - invalid rst format - D000
    - lines should not be longer than 79 characters - D001
      - RST exception: line with no whitespace except in the beginning
      - RST exception: lines with http or https urls
      - RST exception: literal blocks
      - RST exception: rst target directives
    - no trailing whitespace - D002
    - no tabulation for indentation - D003
    - no carriage returns (use unix newlines) - D004
    - no newline at end of file - D005
jackgerrits commented 4 years ago

The CI I have created is for documentation general, not to enforce requirements on them. That should be done as part of PR validation.

RituRajSingh878 commented 4 years ago

Can I add doc8 checks here? https://github.com/VowpalWabbit/vowpal_wabbit/blob/master/build-linux.sh