forge tester: Test on Linux and Windows

apjanke commented 5 years ago

I've done all the development on macOS. It'd be a shame if it was borked for some dumb reason on macOS or Windows, which is what most of the developers use. Test it on those platforms before presenting to the mailing list.

apjanke commented 5 years ago

Make sure to test it using the pkg install <github-repo-url> instead of a local repo checkout, too, since that's what other devs will be using.

apjanke commented 5 years ago

Well, darn. On Linux, the Flatpak sandboxing seems to make the temp directory not visible in the main filesystem, so the result log files are not readily visible.

apjanke commented 5 years ago

It's running on Linux. The code itself seems good; it's just that Octave crashes partway through the test sequence, during testing level-set.

Example output: octave-testify-ForgePkgTester-2019-03-16_19-08-14.zip

apjanke commented 5 years ago

Well, it fails on Windows. pkg uninstall <package> won't work.

Testing package arduino 0.3.0
warning: isdir is obsolete; use isfolder or dir_in_loadpath instead
Processing files in C:\Octave\OCTAVE~1.0\mingw64\share\octave\packages\arduino-0.3.0:

  addon.m ..................................................... PASS      0/1
                                                                  FAIL    1
  isarduino.m ................................................. PASS      4/5
                                                                  FAIL    1
  listArduinoLibraries.m ...................................... PASS      1/1
  scanForArduinos.m ........................................... PASS      1/2
                                                                  FAIL    1
  scanI2Cbus.m ................................................ PASS      0/1
                                                                  FAIL    1

The following files in C:\Octave\OCTAVE~1.0\mingw64\share\octave\packages\arduino-0.3.0 have no t
ests:

arduinosetup.m

TESTS FAILED!
4 failures in tests for packages: arduino

testify.ForgePkgTool: pkg unload arduino instrument-control
testify.ForgePkgTool: pkg uninstall arduino instrument-control
error: couldn't delete directory C:\Octave\OCTAVE~1.0\mingw64\lib\octave\packages\instrument-cont
rol-0.3.1\x86_64-w64-mingw32-api-v53: Permission denied

Looks like it's because of loaded libs, probably oct-files. Windows holds on to those files so you can't delete them until they're unloaded.

apjanke commented 5 years ago

Okay, it's running all the way through on Linux now. Going to set up an overnight test run to get a complete test suite result for the Linux side. Firing up an overnight macOS run, too, so I have a test result from the latest GitHub master there, too.

apjanke commented 5 years ago

Darn. segfault on Linux while testing financial. Looks like it's in the io package. Maybe due to io being installed, loaded, uninstalled, reinstalled, and reloaded in the same session.


testify.ForgePkgTester: Doing Forge package financial 0.5.3
testify.ForgePkgTool: Downloading financial from https://packages.octave.org/download/financial-0.5.3.tar.gz
testify.ForgePkgTester: Installing dependencies for financial: io statistics io
testify.ForgePkgTool: pkg install -forge io statistics io
Gtk-Message: 20:58:01.428: Failed to load module "unity-gtk-module"
Gtk-Message: 20:58:01.476: Failed to load module "canberra-gtk-module"
Gtk-Message: 20:58:01.476: Failed to load module "canberra-gtk-module"
warning: isdir is obsolete; use isfolder or dir_in_loadpath instead
warning: library /home/janke/octave/io-2.4.12/x86_64-pc-linux-gnu-api-v53/cell2csv.oct not reloaded due to existing references
fatal: caught signal Segmentation fault -- stopping myself...
janke@ubix:~ $

If that's what's going on, #17 could probably help clean this up.

apjanke commented 5 years ago

Tested on Windows and Linux now, and it's working well enough on both of them. Closing as fixed.

apjanke / octave-testify

forge tester: Test on Linux and Windows #23