search

rlaager / docsis

This program encodes a DOCSIS binary configuration file from a human-readable text configuration file.
http://docsis.sourceforge.net
GNU General Public License v2.0
115 stars 70 forks source link

Tests fail for 1_0, 1_1 and 20 #9

Closed rapgro closed 9 years ago

rapgro commented 9 years ago

Please fix.

AdrianSimionov commented 9 years ago

Raphael, can you please be more specific on what it is failing?

rapgro commented 9 years ago

Adrian, thanks for your reply. First, you should modify the script to continue the execution of all tests neither if some are failing. Otherwise I have many issues skipping the failing tests each by removing them from file system individually and restart the whole process again.

rapgro commented 9 years ago

I mean this script: https://github.com/AdrianSimionov/docsis/blob/master/tests/RunTests.sh

AdrianSimionov commented 9 years ago

Raphael, this script should stop, this is used by Travis. If at least one regression test will fail, I know my new push made something wrong and I want Travis to fail that build and let me know.

It might be that some of the tests are failing for you because you do not have the non-free MIBs installed, I will add that as a requirement in README. Except that, if some are failing, you should not remove them, you should check why are failing.

This tests are not failing on my Ubuntu 14.10 x86_64, On Debian 7.7-sparc and Travis-CI VMs. If you have a machine where this are failing after installing the non-free MIBs, I would like to take a look and see exactly why are failing.

rapgro commented 9 years ago

Okay, that sounds reasonable. Maybe you could add an option / parameter to the script to not fail and just skip failed test to continue with the other tests? Write the fail and reason into a log.

AdrianSimionov commented 9 years ago

Raphael, maybe it would be good to understand what you are trying to achieve. I use that regression tests to see if I break something when I add code. This is the reason why I want them to fail.

I suppose that you are looking into something else, maybe code coverage?

If not, please let me know also, maybe there are other better ways to improve my tests. Adding an argument is a solution I will add, it makes sense to me at this time.

rapgro commented 9 years ago

Goal: Official package for Fedora. There could be tests run in the %check section of any spec file that is needed to rpmbuild a package.

AdrianSimionov commented 9 years ago

Unfortunately I am not familiar with packaging of any OS. I was planning to become familiar with DEB packaging when I finish the software itself, packaging has very low priority for me at this point as there are many experts on that on the market.

I created this script just to avoid adding breaking something and it helped me several times.

Let me know if there are any other major features you want to prioritize.

rapgro commented 9 years ago

Well, I think DOCSIS 1.x and 2 seem to be obsolete anyways, at least for EuroDOCSIS (in Germany). The tests for 3.0+ seem to work fine. So I'll just skip those tests for <3.0 as not required for the package. Feel free to close this issue if you think that's a goable way.

AdrianSimionov commented 9 years ago

I tested this on Fedore 21, and the tests fail because IETF Mibs are not installed. Once I installed the mib files, all tests are working. I consider this a local configuration issue rather than a bug in docsis utility.

I think this can be closed.

rapgro commented 9 years ago

Thanks for the explanation. It is not possible to ship the nonfree files in Fedora and therefore those tests can not run, but I guess that is no problem.