gnudatalanguage / gdl

GDL - GNU Data Language
GNU General Public License v2.0
274 stars 61 forks source link

At least one case where our testsuite is faulty #1863

Open GillesDuvert opened 1 month ago

GillesDuvert commented 1 month ago

in #1860 there was a mistake in saverestore.cpp that made test_saverestore.pro exit with error:

% SAVE: GDL cannot yet write such large SAVE files. Try with less values. File /tmp/GDL_save_test_numeric1.sav is invalid, remove it.
% Execution halted at: TEST_SAVE_NUMERIC   119 /home/gildas/gdl/testsuite/test_save_restore.pro
%                      TEST_SAVE_RESTORE   210 /home/gildas/gdl/testsuite/test_save_restore.pro
%                      $MAIN$          

(this, when run manually). however, the test was reported as "passed" as https://github.com/gnudatalanguage/gdl/actions/runs/9962944994/job/27527857548?pr=1860 shows. No need to add that his kind of behaviour is very dangerous. Help needed.

alaingdl commented 1 month ago

not that simple : it is possible to put res=EXECUTE("SAVE, [...]") everywhere then testing "res" but usually I tested deeply a change before putting in the github.

Another solution would be to test that the file is generated or not.

Not that simple to decide for a trade-off between hyper basic tests and too much tests

GillesDuvert commented 1 month ago

I suggest replacing execute() by gdl_execute() where gdl_execute calls execute with res=execute() an exit 1 if res is not ok.