CogTool-Modern / cogtool

Predictive human performance modeling for UI/UX design
https://www.cogtool.org
GNU Lesser General Public License v2.1
17 stars 1 forks source link

Prediction: XX #62

Closed justingeeslin closed 4 years ago

justingeeslin commented 4 years ago

Sometimes when calculating a time the prediction is reported as xx. One expects a time for a skilled user to complete the task.

Screen Shot 2020-09-18 at 9 47 34 PM

justingeeslin commented 4 years ago

A quick search through the code for "XX" reveals the following comment.

//  ""  means no task application or no computable script for algorithm
//  --  means the demo is valid and current, but not computed
 // (><) means result is being computed
//  NN  means computed and the demonstration is valid and current
//  XX  means the computation failed and the demo is valid and current
 //  ??  means the demonstration is invalid or script needs regeneration

In our case, the XX case , "the computation failed and the demo is valid and current".

justingeeslin commented 4 years ago

Attached is the CogTool project that reveals the XX. For this same project, all is well on Catalina 10.15, but for 10.12 the XX is the result.

SECOM Beta.cgt.zip

justingeeslin commented 4 years ago

Adding the APredictionResult error lines to the tooltip of the cell reveals that CLISP not being installed may be the culprit.

Screen Shot 2020-09-18 at 10 38 06 PM

Is CLISP a dependency of CogTool? This seems odd since there's plenty of CLISP bundled inside of the app including a lisp.run.

justingeeslin commented 4 years ago

Installing CLISP solved this issue for me.

Install CLISP * `ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)" < /dev/null 2> /dev/null` * `brew install clisp`

Will reopen if anyone else has this issue for a different reason.

justingeeslin commented 4 years ago

To be more specific, On the Mac, CogTool 1.4 and above looks for the MacPorts version of CLISP. I've attached a package on the 1.4 release that will install CLISP.

PatrickonGH commented 4 years ago

I installed CLISP as described, but have still the XX issue. Tested on Mac 10.15.7:

2020-09-30_14-00-44
justingeeslin commented 4 years ago

@PatrickonGH Earlier in the issue, I cited the Homebrew CLISP as the solution to this issue only find out later that the MacPorts version is actually the solution.

Have you installed CLISP via MacPorts either by the package attached to the 1.4 release or via MacPorts CLI (I think 'port install clisp')?

justingeeslin commented 4 years ago

Additionally, we created an alpha release for debugging this very issue.

https://github.com/CogTool-Modern/cogtool/releases/tag/1.5-alpha

In the cell Tooltips you'll find error messages as to why the computation failed. Please Try this alpha release and report back the contents of the cell tooltip with your setup if installing MacPorts CLISP doesn't solve the issue.

PatrickonGH commented 4 years ago

@justingeeslin Thanks! I installed the package and it works now.

This is nevertheless somehow strange because I have a working CLISP installation on my computer. I would have thought that a symlink in the right place would be enough.

It also seems that loading actr6.mem expects a specific version.

justingeeslin commented 4 years ago

Glad that fixed the issue for you!

Yes, you are correct; ACT-R in the form of a saved memory image does require the same CLISP that generated it. If we relied on the CLISP already installed it seems like we'd have to generate a act6.mem for that CLISP installation for that OS / Arch combo.

I think the motivation for pre-generating a memory image in the first place was for performance reasons, but this decision was made back in the WinXP days, when an entire OS fit on a CD, less than 700 MB!

That said, actr6.lisp (not .mem) is included in CogTool and I wonder if it would be to our advantage to load actr6.lisp in memory at runtime, for portability reasons. Then maybe we could rely on a system with CLISP already installed.

PatrickonGH commented 4 years ago

That would be worth a try. It may also be possible to get the last stable release of ACT-R when ANT is running.

justingeeslin commented 4 years ago

Ah, Good idea!