StoXProject / RstoxData

R library for reading various biotic and acoustic data formats
https://stoxproject.github.io/RstoxData/
GNU Lesser General Public License v3.0
1 stars 1 forks source link

FilterBiotic agedetermination #241

Closed edvinf closed 1 year ago

edvinf commented 2 years ago

FilterBiotic does not propagate filters lower than individual level. That is agedetermination rows are kept even if the corresponding individual is removed.

partial example: filterSample <- list() filterSample$samples2021.xml$catchsample <- c( paste("catchsampleid == '", stationdata$samples2021.xml$catchsample$catchsampleid[i], "'", sep="") ) sampledata <- RstoxData::FilterBiotic(stationdata, filterSample)

nrow(sampledata$samples2021.xml$individual) 10 nrow(sampledata$samples2021.xml$agedetermination) 96689

No agedeterminations are expected for this example:

sampledata$samples2021.xml$catchsample$agingstructure NA

nrow(merge(sampledata$samples2021.xml$individual, sampledata$samples2021.xml$agedetermination)) 0

arnejohannesholmin commented 1 year ago

This should be tested in latest master on GitHub.

edvinf commented 1 year ago

Tested on master. Works now.

edvinf commented 1 year ago

There are still cases where this error occurs. Specifically it seems to happen when there are prey-samples in the data.

The following example and data file reproduces the issue, and removing the prey-element from the data file makes it go away.

library(RstoxData) bio <- RstoxData::ReadBiotic(c("biotic3.1_w_ageandprey.xml"))

filterSample <- list() filterSample$biotic3.1_w_ageandprey.xml$fishstation <- c( paste("serialnumber == '", 2, "'", sep="") ) sampledata <- RstoxData::FilterBiotic(bio, filterSample)

There is one individual under serialnumber 2

print(nrow(sampledata$biotic3.1_w_ageandprey.xml$individual))

There is one age reading under serialnumber 2

print(nrow(sampledata$biotic3.1_w_ageandprey.xml$agedetermination))

biotic3.1_w_ageandprey.xml.zip

edvinf commented 1 year ago

A possible soultion has been drafted in https://github.com/StoXProject/RstoxData/commits/propogateFilterBug

edvinf commented 1 year ago

filtering on raw landings (LandingData) should also be a test case, as it also has parallell levels in the hierarchy, and we have some indication that the current filtering does not work (does not apply to aggreagted landings (StoxLandingData))