[Script Editor] Handle unexpected input more robustly

[ctrueden]

Based on this report on SO, the Script Editor may be fragile when it comes to unusual characters or syntax.

The error in question is:

(Fiji Is Just) ImageJ 2.0.0-rc-36/1.50a; Java 1.6.0_24 [64-bit]; Windows NT (unknown) 6.2; 941MB of 6023MB (15%)

java.lang.Error: Error: could not match input
    at net.imagej.ui.swing.script.highliters.ImageJMacroTokenMaker.zzScanError(ImageJMacroTokenMaker.java:2619)
    at net.imagej.ui.swing.script.highliters.ImageJMacroTokenMaker.yylex(ImageJMacroTokenMaker.java:2882)
    at net.imagej.ui.swing.script.highliters.ImageJMacroTokenMaker.getTokenList(ImageJMacroTokenMaker.java:2469)
    at org.fife.ui.rsyntaxtextarea.TokenMakerBase.getLastTokenTypeOnLine(TokenMakerBase.java:219)
    at org.fife.ui.rsyntaxtextarea.RSyntaxDocument.updateSyntaxHighlightingInformation(RSyntaxDocument.java:646)
    at org.fife.ui.rsyntaxtextarea.RSyntaxDocument.setSyntaxStyle(RSyntaxDocument.java:530)
    at org.fife.ui.rsyntaxtextarea.RSyntaxTextArea.setSyntaxEditingStyle(RSyntaxTextArea.java:2764)
    at net.imagej.ui.swing.script.EditorPane.setLanguage(EditorPane.java:490)
    at net.imagej.ui.swing.script.EditorPane.setLanguage(EditorPane.java:435)
    at net.imagej.ui.swing.script.EditorPane.setLanguageByFileName(EditorPane.java:423)
    at net.imagej.ui.swing.script.EditorPane.setFileName(EditorPane.java:369)
    at net.imagej.ui.swing.script.TextEditor.setEditorPaneFileName(TextEditor.java:1683)
    at net.imagej.ui.swing.script.TextEditor.open(TextEditor.java:1388)
    at net.imagej.legacy.plugin.DefaultLegacyEditor.open(DefaultLegacyEditor.java:93)
    at net.imagej.legacy.DefaultLegacyHooks.openInEditor(DefaultLegacyHooks.java:299)
    at ij.io.Opener.open(Opener.java)
    at ij.io.Opener.openAndAddToRecent(Opener.java:288)
    at ij.plugin.DragAndDrop.openFile(DragAndDrop.java:176)
    at ij.plugin.DragAndDrop.run(DragAndDrop.java:152)
    at java.lang.Thread.run(Thread.java:662)

And the script which caused it:

        Variable initiation

    **********************************************

*/
//  Arrays
var list    = newArray();

//  Strings and numbers
var dir = "";                       //  directory
var outDir = "";                    //  output directory
var index = 0;                      //  Index of folder (well)

var minSize = 80, maxSize = 20000;          //  min and max size of nuclei for segmentation (in pixels)
var minCirc = 0;            

var valueStop = 0;
var valueStart = 0;
var startSlice=1;                   //  starts at slice 1, if some slices are done aldready, this value can be increased
var newrun = 1;//has to be 1!!!!
var n=0;
/*
    **********************************************

        Signal Measurements

    **********************************************
*/

macro "[M] Measure Nuclear Foldings"
{
    setup();
    setBatchMode(false);
    print("Analysis");
    print("************************************************************");
    index = 0;
    for(j=0; j<list.length; j++)
    {

        path = dir + list[j];

        if(endsWith(path,'1.tif'))
        {

            open(path);
            print(list[j],"list");
            print("list");
            id = getImageID;
            title = getTitle();
            slices=nSlices; 

            print(n);

            prefixCh2 = replace(title,"1.tif","2.tif");
            path2 = dir + prefixCh2;
            open(path2);
            idch2 = getImageID;

            for(ii=startSlice; ii<slices+1; ii++)
            {   

                //run("Collect Garbage");
                selectImage(id);
                if(newrun==1)
                {
                newrun=2;
                    run("Out [-]");
                    run("Out [-]");

                    //get a duplicate to present the rois and to measure the true values
                    //this copy has to have multiple slices so the roimanager can add the
                    //slice number to the results table 
                    selectImage(id);
                    setSlice(ii);
                    run("Duplicate...", "title=DuplicateChannel1.tif duplicate range="+1+"-"+slices);
                    id2 = getImageID;
                    run("Out [-]");     
                    run("Out [-]");

                    selectImage(idch2); 
                    run("Out [-]");     
                    run("Out [-]");

                }

                selectImage(idch2);
                setSlice(ii);

                //get a duplicate of a single image on which you will do the cell detection
                selectImage(id);
                setSlice(ii);
                run("Duplicate...", "title=BLUEchannel.tif");
                BLUEid = getImageID;
                selectImage(BLUEid);                
                run("Out [-]");
                roiManager("reset");
                run("Gaussian Blur...", "sigma=2");
                setAutoThreshold("Triangle");
                run("Convert to Mask");
                run("Gaussian Blur...", "sigma=2");
                setAutoThreshold("Li"+" dark");
                run("Analyze Particles...", "size="+60+"-"+6000+" circularity="+0+"-1.00 show=Nothing add stack");
                n=roiManager("count");
                print(n);

                selectImage(BLUEid);
                close;

selectImage(idch2); 
                roiManager("deselect");
                roiManager("Show All");
                roiManager("Set Color", "red");             

            if(n>0)
                {

                selectImage(id2);   
                setSlice(ii);       
                roiManager("deselect");
                roiManager("measure");
                selectWindow("Results");

                //start removing Rois that are incorrect
                if (nImages==0) exit("There is no image open");
                if (!isOpen("ROI Manager")) exit("There is no ROI manager open");
                if (!isOpen("Results")) exit("There is no results table. Create via Multi Measure option in ROI Manager");
                if (roiManager("count")==0) exit("There are no ROIs loaded in the ROI manager");
                roiManager("Associate", "true");
                roiManager("Centered", "false");
                roiManager("UseNames", "true");
                setTool("rectangle");
                roiManager("Show All with labels");
                roiManager("Show All");
                roiManager("Deselect");
                leftButton=16;

                x2=-1; y2=-1; z2=-1; flags2=-1;
                logOpened = false;

                print("Close this window when finished");
                while (!logOpened || isOpen("Log")) {
        §           getCursorLoc(x, y, z, flags);
                    if (x!=x2 || y!=y2 || z!=z2 || flags!=flags2) {     // Only when mouse moves new locatation is logged
                    wait(20);                       // Might have to be increased with large number of ROIs
                                if (flags&leftButton!=0) {

                                    // Check that a ROI is selected
                            if (roiManager("index")!=-1){
                                RoiName= Roi.getName();
                                print(RoiName);
                                del = false;            // Becomes "true" when a result is deleted from the table                               
                                selectWindow("Results");
                                valueDiff = (roiManager("count"));                                                          
                                valueStop=valueStart+valueDiff;
                                for (i=valueStart;i<valueStop;i++){
                                    if (del==false){
                                        // Check label ROI against labels in Results table                                      
                                        S = getResultLabel(i);                                      
                                        if (matches(S,".*"+RoiName+".*")){
                                            selectWindow("Results");                                            
                                            IJ.deleteRows(i, i);
                                            del = true;
                                        }
                                    }
                                }
                                roiManager("Delete");
                            }
                                    logOpened = true;
                        }
                            x2=x; y2=y; z2=z; flags2=flags;         // Only when mouse moves new location is logged
                            wait(10);
                                // Takes care that one mouse click is recorded as one mouse click
                        }
                }
        }           
                //keep track of the index of the results tabel, this increases now

                valueRois = (roiManager("count"));
                valueStartOld=valueStart;
                valueStart = valueStartOld+valueRois*2;

                //add something to change to channel two and measure the values of these cells.

if(n>0)
        {

                selectImage(idch2);         
                setSlice(ii);   
                roiManager("deselect");
                roiManager("measure");
                selectWindow("Results");
                saveAs("Measurements",dir+"ResultsFinalDanaiCh1.txt");
        }
                /*
                selectImage(id2);
                setSlice(ii);
                run("Duplicate...", "title=duplicateID2.tif");
                id3 = getImageID;
                selectImage(id3);
                run("Out [-]");
                roiManager("Show All");
                    roiManager("Set Color", "red");               
                    saveAs("tif",dir+ii+"Channel1_image.tif");                  
                selectImage(id3);
                close;
*/

            }

        }
    }
    print("************************************************************");

}

/*  
    **********************************************

        Functions

    **********************************************
*/

function getMoment()
{
     MonthNames = newArray("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec");
     DayNames = newArray("Sun", "Mon","Tue","Wed","Thu","Fri","Sat");
     getDateAndTime(year, month, dayOfWeek, dayOfMonth, hour, minute, second, msec);
     TimeString ="Date: "+DayNames[dayOfWeek]+" ";
     if (dayOfMonth<10) {TimeString = TimeString+"0";}
     TimeString = TimeString+dayOfMonth+"-"+MonthNames[month]+"-"+year+"\nTime: ";
     if (hour<10) {TimeString = TimeString+"0";}
     TimeString = TimeString+hour+":";
     if (minute<10) {TimeString = TimeString+"0";}
     TimeString = TimeString+minute+":";
     if (second<10) {TimeString = TimeString+"0";}
     TimeString = TimeString+second;
     return TimeString;
}

function setup()
{
    print("\\Clear");
    run("Close All");
    run("Clear Results");
    roiManager("reset");
    //run("Collect Garbage");
    run("Colors...", "foreground=white background=black selection=yellow"); 
    setOption("BlackBackground", false);
    run("Set Measurements...", "area mean shape stack slices ii center of mass display redirect=None decimal=4");
    dir = getDirectory("");
    list = getFileList(dir);
    isWin=indexOf(getInfo("os.name"),"Windows")>=0;
    //if(isWin)outDir=dir+"Output\\";
    //else outDir=dir+"Output/";
    //if(!File.exists(outDir))File.makeDirectory(outDir);
    TimeString = getMoment();
    print(TimeString);
    print("************************************************************");
}

[ctrueden]

@imagejan pointed out that there is a § character midway through the script, which might be the culprit. But regardless, I think it is a bug to dump a big stack trace regardless of the script's contents. Would you agree, @Squareys?

[ctrueden]

OK, I take it back: this has nothing to do with javascript syntax highlighting. Sorry for the wrong classification, @Squareys. From the stack trace, it is a bug in the ImageJ1 macro support, written awhile ago by Johannes.

I am investigating!

[ctrueden]

I cannot reproduce with the posted script. Maybe the § character is actually something else, or there is another hidden character.

In any case, I am closing this issue for now. If it comes up again, and someone can provide an MCVE to reproduce, then we will certainly fix it.

[stelfrich]

@ctrueden This issue caught my interest, because a user has experienced the same issue with another script. I have, however, for testing purposes used the script from SO.

The error does not appear every time I open the script, but it seems to happen more frequently after the script was executed and canceled. Opening it via File > Open Recent seems to increase the frequency of such errors occurring. I did about 30 runs in debug mode and caught the error several times. Every time, the issue was due to a comment. The end of a token (when the error occurred) during the tokenizing were somewhere way off, sometimes several lines below the actual end line of a comment.

What still baffles me is that the behavior is not deterministic..