Standardize list handling (Front-Loading of valid data)

[TheHarvard]

It is probably beneficial to standardize the way lists should be handled.

one question of this concern whether to require all lists to be sorted or not. by sorted I mean that the list is front-loaded with the in-use values. as RAPID does not allow for variable-length lists, there is usually going to be data in the list that is not in use. looping over this is unnecessary performance loss in larger lists. by front-loading the lists with the in-use data, you can stop looping through when you hit the first unused data, BUT you get some overhead in shifting the data over once you remove an entry, and added complexity in using the lists.

which method do we believe is best?

@RobotSigmund @AGus-RN adding this to the agenda for the next meeting

[RobotSigmund]

Just as a comment. I and Nicolai discussed writing a list-handling module that would simply create a new modfile and automaticly load it. It would contain an array and methods for get and push or shift values. Performance would be an issue, but I think it would work. We never tried it.

Regarding arrays as lists. I prefer to add an Id field to the array-record. A zero or non-positive value indicates an unoccupied index. Shifting the array may cause issues when shared between tasks. The Kolonial project had com bugs where this were suspected but never verified.

[TheHarvard]

Heavy Loading and editing of modules at runtime is, as you are saying, performance hungry. Loading and unloading modules also have limitations.

Interesting experience from kolonial. We should probably test and verify if such problems/issues do come from shifting arrays.

[TheHarvard]

Another alternative to writing and loading an entire module is to use rawbytes. Though this is also performance heavy

[AGus-RN]

As I mentioned to Sigmund briefly yesterday. Perhaps it could be considered to use a lock, similiar to multitasking in linux. For instance use a bool flag which is set by the task modifying the array. All other tasks trying to read/write from/to the list have to wait until the flag is not set.

[TheHarvard]

Yes, for inter-task activity (not just for lists), a Mutex/Semaphore/WriteLock system is desired. I have a prototype specifically for inter-task WriteLocks, but have not finalized the code. Should we start a separate issue on this?

[AGus-RN]

It sounds like a good idea to create a separate issue if you already have started on something which is more generic than this specific issue.

[TheHarvard]

@AGus-RN see #42

[RobotSigmund]

Majority wants front filled arrays for manual readability. Action; create mod for standard methods for arrays.

[TheHarvard]

Side comment:

Standard words for lists:

• Append -> add item to end of list
• pop -> get last item in list, and remove it from list.
• lenght -> the current number of populated spots
• dim -> the number of available spots

for lists that are actively front loaded, append should search for first availalbe index, and pop should remove the last index

[TheHarvard]

should we reserve the word "LIST" as a prefix to all methods that handle lists in standardized ways (like pop and append)?

Since it isn't possible to build 1 method that works with all datatypes, then having a standard LIST_ prefix for the methods that comply with some standardized specification might be the easiest to use?

example Code:

    PROC LIST_dataPointer_append(INOUT dataPointer list{*}, dataPointer item)

        VAR dataPointer NULL;
        NULL := dataPointer_NULL;

        FOR i FROM 1 TO Dim(list,1) DO
            IF list{i} = NULL THEN
                list{i} := item;
                RETURN;
            ENDIF
        ENDFOR

    ENDPROC

    FUNC dataPointer LIST_dataPointer_pop(INOUT dataPointer list{*})

        VAR dataPointer item;
        VAR dataPointer NULL;
        NULL := dataPointer_NULL;

        FOR i FROM Dim(list,1) TO 1 DO
            IF list{i} <> NULL THEN
                item := list{i};
                list{i} := NULL;
                RETURN item;
            ENDIF
        ENDFOR 

    ENDFUNC

    FUNC num LIST_dataPointer_lenght(INOUT dataPointer list{*})

        VAR num lenght;
        VAR dataPointer NULL;
        lenght:=0;
        NULL:=dataPointer_NULL;

        FOR i FROM 1 TO Dim(list,1) DO
            IF list{i}<>NULL THEN
                lenght:=lenght+1;
            ENDIF
        ENDFOR

        RETURN lenght;

    ENDFUNC

[TheHarvard]

see also #57