OPAL-2958: Exception in type coercion for numeric repeatable variable (search entities UI to view)

[ymarcon]

Jira issue originally created by user abhishek:

Hi Yannick,

I have come across the following java exception when I try to add a numeric repeatable variable to a view from the search entities UI.

Error in script: com.google.common.collect.RegularImmutableList cannot be cast to java.lang.Number

Basically, when I impose a condition (say less than 100) on a repeatable variable 'SBP' I get a list of count of all entities where at least one SBP is less than 100 ( magma: $('HV.SBP').le(100) )

When I add this variable to a view and I get a type coercion exception.

Another question for you now when we have filter methods for value sequences can we make the view to have the following magma by default?

('HV.SBP').filter(function(value) { return value.le(100); })

As I explained for repeatable variables it is helpful to get the actual rows meeting the criteria rather all.

Thank you

[ymarcon]

Comment created by abhishek:

The problem is the magma within the entity filter bit (i.e. $('HV.SBP').le(100)).

$(HV.SBP') is a repeatable variable where each value is numeric so we can't call a numeric method on it. Had $('HV.SBP') been a non-repeatable numeric variable it would have worked.

I have just tested and the following works,

$('HV.SBP').filter(function(value) { return value.le(100); }).isNull().not();

[ymarcon]

Comment created by @ymarcon:

I think the following fixes should be done:

• le() method (and similar ones) should work with value sequences: true when true for all the values of the sequence.
• the view entities filter generated should be as you suggested (combining filter() and isNull().not()) because in this case we are looking for the entities which have at least one value matching criteria.

[ymarcon]

Comment created by abhishek:

Hi Yannick,

The idea is that the search entities UI should take into account the variable 'type' (i.e. repeatable /non-repeatable) when filtering and this should be reflected in the magma.

How about the following scenario?

We have a table with 5 variables where first 3 are non-repeatable and others repeatable. I add a filter (>100) on var4 so the magma becomes,

$('var4').filter( function(value) { return value.gt(100); });

I then add one more filter but this time to var5 (< 50). Then the resulting 'Script' attribute for variables var4 and var5 should be,

var4 => $('var4').filter( function(value, index) {
return value.gt(100).and( $('var5').valueAt(index).lt(50).value() ); });

var5 => $('var5').filter(function(value, index) { return value.lt(50).and( $('var4').valueAt(index).gt(100).value() ); });

If at this stage I add both these variables to a view the 'Entity Filter' (Dictionary tab) script should be:

$('var4').filter( function(value, index) {
return value.gt(100).and( $('var5').valueAt(index).lt(50).value() ); }).isNull().not();

This way the user would get the relevant rows and the right count of entities (no empty rows/entities) that match their search criteria.

It is really cool that your filter method allow us to do filtering and now it is just the matter of implementing this in search entities UI.

By the way for some reason the 'Entity Filter' script editor counts blank lines and also doesn't like edits from the first line.

Many thanks

[ymarcon]

Comment created by @ymarcon:

The entities filter corresponding to the entities search is: having at least one value such as var4>100 AND at least one value var5<50; which is expressed by:

$('var4').filter( function(value, index) { return value.gt(100); }).isNull().not()
  .and($('var5').filter( function(value, index) { return value.lt(50); }).isNull().not())

In your example you express an even more restrictive filter because you expect criteria to apply at value occurrence level (i.e. measure). What about having tables about measures instead of participants? this would tackle the complexity of handling value sequences: each row is a measure and you could easily find measures that match criteria. One of the variable of the measures table would be the participant ID (you can define the referencedEntityType property of this variable) and use the $join() method.

[ymarcon]

Comment created by abhishek:

Sure but as per your code the resulting set will contain rows where var4 and var5 don't meet the criteria together.

Restrictive filter is precisely the idea because we are working with longitudinal or time-series data that have loads of rows. Any downstream analyses (e.g. univariate, multivariate) will use them as dependent or independent variables and it is therefore important that they together satisfy a criteria.

I have tried the approach of referencedEntityType and $(join). The problem is the counts from queries will no longer be entities. Also it would be difficult to join tables that have a common identifier. And of course it wouldn't be possible to split occurrences of repeated variables because there will be no repeated variables.

[ymarcon]

Comment created by @ymarcon:

• comparison js methods handle value sequences (all values must satisfy the comparison)
• view's entity filter matches the search entities criteria of repeatable variables

[ymarcon]

Issue was closed with resolution "Fixed"