search

apjanke / octave-testify

New BIST (Built-In Self Test) functions for GNU Octave
GNU General Public License v3.0
4 stars 2 forks source link

doctest tests fail with "'is_octave' undefined" #43

Closed apjanke closed 5 years ago

apjanke commented 5 years ago

Running doctest on several of the packages fails with this error:

testify.ForgePkgTester: Error while testing package financial: 'is_octave' undefined near line 19 column 5

Log: test_forge_pkgs.log

Affected packages:

$ grep is_octave octave-testify-ForgePkgTester-2019-03-18_10-49-40/test_forge_pkgs.log
testify.ForgePkgTester: Error while testing package doctest: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package financial: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package fits: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package ga: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package general: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package generate_html: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package geometry: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package image: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package interval: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package lssa: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package ltfat: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package mapping: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package miscellaneous: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package mvn: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package nan: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package netcdf: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package ncarray: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package nurbs: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package octclip: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package optics: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package optiminterp: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package queueing: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package secs1d: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package secs3d: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package sockets: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package stk: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package symbolic: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package tsa: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package vibes: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package windows: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package zeromq: 'is_octave' undefined near line 19 column 5

I turned doctest off by default in https://github.com/apjanke/octave-testify/commit/3640205e022cbbb44d2c6cea5e9b2962859ba652 because of this.

is_octave() is a private function inside doctest. Maybe it's being called from an eval context or something like that? Or maybe I'm accidentally unloading the doctest pkg during the tests?

Figure out a simpler repro so we can report it to the doctest project.

apjanke commented 5 years ago

Looks like this is a problem with Testify, not doctest. When I try to reproduce with doctest alone, it works fine.


>> pkg install -forge financial io statistics
>> pkg load doctest
>> cd '/Users/janke/Library/Application Support/Octave.app/5.1.0/pkg/financial-0.5.3'
>> ls
>> doctest .
[...]
Summary:

   FAIL    1/3

101/102 targets passed, 100 without tests.

>>
apjanke commented 5 years ago

Works okay with just __test_pkgs__('financial', struct("doctest",true)).

apjanke commented 5 years ago

Aha! The doctest package itself is the first one to fail.


$ grep is_octave octave-testify-ForgePkgTester-2019-03-18_10-49-40/test_forge_pkgs.log
testify.ForgePkgTester: Error while testing package doctest: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package financial: 'is_octave' undefined near line 19 column 5
testify.ForgePkgTester: Error while testing package fits: 'is_octave' undefined near line 19 column 5

Maybe I'm unloading the doctest package after testing it.

apjanke commented 5 years ago

Fixed in https://github.com/apjanke/octave-testify/commit/02759c62dba9ca43e5e87509d0726b8827c630d8.