Issues with YagiUdaAntennas example

[CCherqui]

I have compiled scuff-em and tried running the Runscript.SZParms script in the YagiUdaAntennas example folder and got the following segmentation fault.s

./RunScript.SZParms: line 17: 24688 Segmentation fault: 11  ${CODE} ${ARGS} --geometry ${GEODIR}/${GEOM}_Medium.scuffgeo

[HomerReid]

I couldn't reproduce this---the code ran to completion with no errors for me. Could you post the .log file and any other output files produced by your run? Do the other command-line codes have similar issues? Does the test suite (make check) run successfully?

[CCherqui]

So I ran the make check and got these errors (attached). Should I post the .log? testsuite_output.txt

[CCherqui]

test-suite.log

Ok, I think I see the issue but I don't get it. the scuff compiled executables are all in my main search path.

[HomerReid]

Thanks for doing this. Looking at the log files you posted, it seems the MacOS version of the time command does not support the -v option (which I had assumed was a POSIX standard...it's hard to keep track of all the little ways MacOS deviates from standard behavior). Could you please try again after editing the file tests/Mie/TestMie.sh to remove the -v option from time on line 77? You will want to make the same modification in tests/Fresnel/TestFresnel.sh.

[CCherqui]

Ok, I removed the -v from the TestMie.sh and TestFresnel.sh (like it is below) and got the same error. test-suite.log

Please excuse all the editing and poor formatting, I'm a github noob and still learning.

RUNTIME=${GEOM}.Runtime
if [ -x `which time` ]
then
TIMECMD="`which time` -o ${RUNTIME}"
fi

[CCherqui]

I also tried the new spectrum example for the nanorod dimer and got a segmentation fault error as well. scuff-spectrum.log

[HomerReid]

OK, so the time command on MacOS doesn't support the -o option either. So just change line 77 to say simply

   TIMECMD="`which time` 

As for the error you reported for SCUFF-SPECTRUM, note from the logfile that the core dump is not happening in SCUFF-EM, but rather in the linear-algebra library you are using, presumably the MKL if you are compiling with Intel compilers. This makes it hard to debug. Maybe you could compile with debugging support enabled and run in the Intel debugger? Or maybe the Intel compiler suite has a tool like valgrind for diagnosing core dumps?

[CCherqui]

I removed the -o option and I still get the same error. I also checked the which time command in my terminal and got /usr/bin/time as the response.

I'll compile with a debugger and open a new issue with it. Thanks for all your help.

[CCherqui]

Ok, So I uninstalled the intel version and just used gcc. This solved the segmentation fault error. I'll go back to intel later since that's the ultimate goal but I have now isolated the issue with the make check. I did as you suggested to the TestMie file and got the following errors (attached).

test-suite.log

[HomerReid]

The issue here, which was not arising in your previous build, is that SCUFF-EM seems to have been installed to a non-standard installation path, so the test harness can't find the files it needs. Is it possible you specified a different --prefix when you recompiled with GCC? Or that you didn't do a make install? These were not problems in your previous build.

As the error message in test-suite.log says, try setting the environment variable PKG_CONFIG_PATH to wherever the pkg-config file for SCUFF-EM (the filename is scuff-em.pc) is installed. For example, if you installed to a directory called /prefix/for/scuff then you would want to set PKG_CONFIG_PATH=/prefix/for/scuff/lib/pkgconfig. When this environment variable is set correctly, the command pkg-config scuff-em --variable=datadir should produce nontrivial output stating where the tests and example files are installed.

[CCherqui]

I did put scuff-em in a non-standard location and I did add that path variable but had a spelling error. I've recompiled and placed scuff-em in the more standard $~/scuff-em-installation. I also added export PKG_CONFIG_PATH=${HOME}/scuff-em-installation/lib/pkgconfig to my .bashrc. now, when I input the command kg-config scuff-em --variable=datadir I get ${HOME}/scuff-em-installation/share and the following errors when running make check. The file it can't find should be but is not in /scuff-em/tests/Mie/reference only the files LossySphere_327.DSIPFT LossySphere_327.EMTPFT LossySphere_327.EPFile.total LossySphere_327.Moments are. I'm sure I'm missing something obvious, so again, thanks for your patience.

=============================================
   scuff-em 0.96: tests/Mie/test-suite.log
=============================================

# TOTAL: 1
# PASS:  0
# SKIP:  0
# XFAIL: 0
# FAIL:  1
# XPASS: 0
# ERROR: 0

.. contents:: :depth: 2

FAIL: TestMie.sh
================

CHECKSCUFFDATA=
Using 8 CPU cores (0-7)
LossySphere_327.Runtime: No such file or directory
        0.00 real         0.00 user         0.00 sys
./TestMie.sh: line 130: --data: command not found
EPFile.total test: failure(127)
./TestMie.sh: line 130: --data: command not found
EMTPFT test: failure(127)
./TestMie.sh: line 130: --data: command not found
DSIPFT test: failure(127)
./TestMie.sh: line 130: --data: command not found
Moments test: failure(127)
FAIL TestMie.sh (exit status: 127)

[HomerReid]

The absence of the RunTime file isn't the main culprit here---the test script just skips over that step if the file is missing. The error here is that it can't find the CheckSCUFFData program. The TestMie.sh script uses pkg-config to get the path to this code, so if your installation is in a non-standard location and you haven't set PKG_CONFIG_PATH then it will fail.

I would try:

(a) Make sure PKG_CONFIG_PATH is set correctly (you mentioned you had been successful at this previously), then try make check (following make install).

(b) Failing that, try manually running the test script:

% export PKG_CONFIG_PATH={correct setting on your system}
% export PATH=${PATH}:/path/to/scuff/bin  
% cd /path/to/scuff/share/scuff-em/tests/Mie
% TestMie.sh 

[CCherqui]

So I've done all of that (I've also carefully read your documentation on using the test suite). http://homerreid.github.io/scuff-em-documentation/tests/Overview/

This is already in my .baschrc

export PATH=${PATH}:${HOME}/scuff-em-installation/bin
export PKG_CONFIG_PATH=${HOME}/scuff-em-installation/lib/pkgconfig

These are the correct folders (I checked). When I run pkg-config scuff-em --variable=datadir it returns ${HOME}/scuff-em-installation/share (which is correct)

And yet, when I run either make test or TestMie.sh, I get the following error

CHECKSCUFFDATA=
Using 8 CPU cores (0-7)
LossySphere_327.Runtime: No such file or directory
        0.00 real         0.00 user         0.00 sys
./TestMie.sh: line 130: --data: command not found
EPFile.total test: failure(127)
./TestMie.sh: line 130: --data: command not found
EMTPFT test: failure(127)
./TestMie.sh: line 130: --data: command not found
DSIPFT test: failure(127)
./TestMie.sh: line 130: --data: command not found
Moments test: failure(127)
FAIL TestMie.sh (exit status: 127)

• i.e. it can't find CheckSCUFFData. If add the path to it directly via export PATH=${PATH}:${HOME}/scuff-em-installation/share/scuff-em/tests, I get the same error but with the first line changed to CHECKSCUFFDATA=${HOME}/scuff-em-installation/share/scuff-em/tests/CheckSCUFFData

Maybe this is isn't worth wasting your time over abymore, you've been very helpful and patient. If this is useful to you than I'm happy to continue trying to resolve the issue. Otherwise, things seem to be working fine when I do the examples, but for whatever reason I can't get the test files to find CheckSCUFFData.

off topic - can scuff-spectrum be used on perdiodic systems?

[HomerReid]

No, I appreciate the opportunity to try to figure out what's going wrong.

Can you doublecheck that the CheckSCUFFData executable was correctly built by the installation procedure and installed in ${HOME}/scuff-em-installation/share/scuff-em/tests?

If that works, try setting the environment variable CHECKSCUFFDATA to the full path to the code, then running TestMie.sh:

% export CHECKSCUFFDATA=${HOME}/scuff-em-installation/share/scuff-em/tests/CheckSCUFFData
% cd ${HOME}/scuff-em-installation/share/scuff-em/tests/Mie
% TestMie.sh

[HomerReid]

off topic - can scuff-spectrum be used on perdiodic systems?

Yes, but I haven't tested this yet---the code is relatively new and I haven't had a chance to investigate all cases. It would be great to have some help with validation.

[CCherqui]

It is certainly installed in ${HOME}/scuff-em-installation/share/tests, but how do I check if it's correctly built?

[HomerReid]

If it's there then it was built and installed correctly. So the issue seems to be just that the attempt in the TestMie.sh script to use pkg-config to discern the whereabouts of SCUFF doesn't work on MacOS, for whatever reason. For the time being, just override that by explicitly setting CHECKSCUFFDATA to the full path of the CheckSCUFFData executable, as in the snippet above.

[CCherqui]

Ok, so I did the following

export CHECKSCUFFDATA=${HOME}/scuff-em-installation/share/scuff-em/tests/CheckSCUFFData
cd ${HOME}/scuff-em-installation/share/scuff-em/tests/Mie
sh TestMie.sh

and got the following output

CHECKSCUFFDATA=/Users/cherqui/scuff-em-installation/share/scuff-em/tests/CheckSCUFFData
Using 8 CPU cores (0-7)
LossySphere_327.Runtime: No such file or directory
        0.00 real         0.00 user         0.00 sys
error: could not open LossySphere_327.EPFile.total (aborting)
EPFile.total test: failure(1)
error: could not open LossySphere_327.EMTPFT (aborting)
EMTPFT test: failure(1)
error: could not open LossySphere_327.DSIPFT (aborting)
DSIPFT test: failure(1)
error: could not open LossySphere_327.Moments (aborting)
Moments test: failure(1)

[HomerReid]

It looks like scuff-scatter failed to run, and thus failed to generate the output files that the checker was looking to compare. I can't tell from the output here what went wrong. Did it produce a .log file?

The line of the script that executes the code is line 98, which looks like this:

${TIMECMD} ${CODE} ${ARGS}

Maybe something here is too fancy for MacOS to process, or maybe bash scripts execute differently on MacOS. Try replacing that line with simply

 scuff-scatter ${ARGS}

or maybe

 ${HOME}/scuff-em-installation/bin/scuff-scatter ${ARGS}

[CCherqui]

Ok, so a combination of explicitly setting CHECKSCUFFDATA and changing line 98 of TestMie.sh to scuff-scatter ${ARGS} produced the following output. (I also decided to just go ahead and put scuff-em in the normal install location)

CHECKSCUFFDATA=/usr/local/share/scuff-em/tests/CheckSCUFFData
Using 8 CPU cores (0-7)
Thank you for your support.
Data set LossySphere_327.EPFile.total: 360 total data items (0 wildcards)
Data set reference/LossySphere_327.EPFile.total: 360 total data items (0 wildcards)
360/360 matches.
EPFile.total test: success
Data set LossySphere_327.EMTPFT: 60 total data items (0 wildcards)
Data set reference/LossySphere_327.EMTPFT: 60 total data items (0 wildcards)
60/60 matches.
EMTPFT test: success
Data set LossySphere_327.DSIPFT: 60 total data items (0 wildcards)
Data set reference/LossySphere_327.DSIPFT: 60 total data items (0 wildcards)
60/60 matches.
DSIPFT test: success
Data set LossySphere_327.Moments: 60 total data items (0 wildcards)
Data set reference/LossySphere_327.Moments: 60 total data items (0 wildcards)
60/60 matches.
Moments test: success

Still not sure why I needed to set CHECKSCUFFDATA but at least it works now.

Also, in a related matter - line 63 of ComputeNanorodModes.sh leads to an error as nproc is not supported on Mac OSX and needs to be replaced with sysctl -n hw.physicalcpu to make the script work.

I'm working on using your scuff-spectrum to find modes of a finite lattice, if I am successful should I just email you the script or make a post here?

Thanks for all your help!

[CCherqui]

I tried doing something similar to the TestFesnel.sh script by changing line 98 to scuff-transmission ${ARGS} but didn't have the same success. I got a segmentation fault instead. I'm also attaching the relevant .log and .error files. (E10HalfSpace_40.errors.txt, CheckSCUFFData.log)

CHECKSCUFFDATA=/usr/local/share/scuff-em/tests/CheckSCUFFData
Using 8 CPU cores (0-7)
TestFresnel.sh: line 89: 25573 Segmentation fault: 11  scuff-transmission ${ARGS}
Data set E10HalfSpace_40.transmission: 0 total data items (0 wildcards)
Data set E10HalfSpace_40.transmission.reference: 912 total data items (0 wildcards)
0/912 matches.

[HomerReid]

Good progress that the Mie test runs correctly.

For the Fresnel test, is there a scuff-transmission.log file?

[HomerReid]

There should also be a file named E10HalfSpace_40.log?

[CCherqui]

Here they are,

scuff-transmission.log

E10HalfSpace_40.log

[CCherqui]

Oddly enough. I went back and ran make check (and changed line 98 in TestFresnel.sh to scuff-transmission ${ARGS} and got the following.

Making check in tests
Making check in Mie
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check-TESTS
PASS: TestMie.sh
============================================================================
Testsuite summary for scuff-em 0.96
============================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
Making check in Fresnel
/Applications/Xcode.app/Contents/Developer/usr/bin/make  check-TESTS
PASS: TestFresnel.sh
============================================================================
Testsuite summary for scuff-em 0.96
============================================================================
# TOTAL: 1
# PASS:  1
# SKIP:  0
# XFAIL: 0
# FAIL:  0
# XPASS: 0
# ERROR: 0
============================================================================
make[2]: Nothing to be done for `check-am'.
Making check in m4
make[1]: Nothing to be done for `check'.
make[1]: Nothing to be done for `check-am'.

But when I run TestFresnel.sh on its own i get the errors I posted above.

[HomerReid]

Thanks for the updates and the helpful feedback. In 7f4d783 I have

• fixed the issue that was causing the seg fault in the Fresnel test
• revised the bash scripts driving the Mie and Fresnel tests to hopefully be more portable
• updated NanorodResonantModes to replace nproc with a portable solution.

I am still working on the headline bug for this issue, and will have that fixed soon.

[CCherqui]

Do I recompiled and tried the new TestMie.sh script (making sure to set export CHECKSCUFFDATA=/usr/local/share/scuff-em/tests/CheckSCUFFData) and got the following.

Using 8 CPU cores (0-7)
/usr/bin/time: illegal option -- o
usage: time [-lp] command.
error: could not open LossySphere_327.EPFile.total (aborting)
EPFile.total test: failure(1)
error: could not open LossySphere_327.EMTPFT (aborting)
EMTPFT test: failure(1)
error: could not open LossySphere_327.DSIPFT (aborting)
DSIPFT test: failure(1)
error: could not open LossySphere_327.Moments (aborting)
Moments test: failure(1)

[CCherqui]

Changing the appropriate parts of the sh files so that they read

TIMECMD=`which time`
if [ $? -eq 0 ]
then
  ${TIMECMD} -lp ${RUNTIME} ${CODE} ${ARGS}
else
  echo "No time command found." > ${RUNTIME}
fi
  ${CODE} ${ARGS}

seems to have fixed it.

[HomerReid]

Thanks, yes, I thought I had read that the MacOS time command supports the -o option, but apparently not. I will modify the script.

[CCherqui]

Small point, but I also had to take ${CODE} ${ARGS} out of the if-then statement.

[HomerReid]

Can you run the Yagi-Uda antennas example without core dumping now?

[CCherqui]

yes, I'll close this topic. Thanks.