search

DistanceDevelopment / readdst

Convert Distance for Windows projects into R code/data
GNU General Public License v3.0
1 stars 2 forks source link

2018 non-functionality #34

Open erex opened 6 years ago

erex commented 6 years ago

Error arises when doing conversion of (allegedly) simple project (songbird, filtering for song sparrow, w=68)

Tracked error to function unit_tab()

Error in `$<-.data.frame`(`*tmp*`, Conversion, value = numeric(0)) : 
  replacement has 0 rows, data has 74
In addition: Warning message:
Unknown or uninitialised column: 'Conversion'.

The beast created by read.delim() trick is messy. Solution appears to be this:

Browse[2]> str(unit_tab)
Classes ‘tbl_df’, ‘tbl’ and 'data.frame':   74 obs. of  2 variables:
 $                  Unit     : chr  "                     Degree  " "                        Gon  " "                       Grad  " "                Microradian  " ...
 $      Conversion           : chr  " 1.745329251994329547437e-02" " 1.570796326794896696777e-02" " 1.570796326794896696777e-02" " 9.999999999999999547481e-07" ...
 - attr(*, "spec")=List of 2
  ..$ cols   :List of 2
  .. ..$                  Unit     : list()
  .. .. ..- attr(*, "class")= chr  "collector_character" "collector"
  .. ..$      Conversion           : list()
  .. .. ..- attr(*, "class")= chr  "collector_character" "collector"
  ..$ default: list()
  .. ..- attr(*, "class")= chr  "collector_guess" "collector"
  ..- attr(*, "class")= chr "col_spec"
Browse[2]> names(unit_tab)
[1] "                 Unit     " "     Conversion"           
Browse[2]> names(unit_tab) <- c("Unit", "Conversion")
Browse[2]> n
debug: return(unit_tab)

I'll made that change and commit

erex commented 6 years ago

yes, there are no strata areas in songbird project, so abundance cannot be computed, but many other things break when trying to duplicate analysis in which size-biased regression by stratum is requested:

> fred <- test_stats(tmp[[1]], statuses=2)
Error in create.varstructure(model, region.table, sample.table, obs.table) : 
  Invalid or missing Area values for regions
In addition: Warning messages:
1: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
2: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
3: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
4: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
5: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
6: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
7: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
8: In create.varstructure(model, region.table, sample.table, obs.table) :
  Some samples not included in the analysis

no object is created

quite an exotic P_a is computed from the fitted detection function:

                    Estimate       SE        CV
Average p            4.53580  2.66565 0.5876912
erex commented 6 years ago

Still working with songbird project, but trying second analysis with cluster size as detection function covariate: somewhere (don't know what function does this), the column "Cluster size" does not have the space replaced by a full stop. No problems arise from convert_project()

But run_analysis() fails with:

> two <- run_analysis(tmp[[2]])
Error in parse(text = analysis$call) : <text>:1:52: unexpected symbol
1: mrds::ddf(dsmodel=~mcds(key="hn", formula=~Cluster size
dill commented 6 years ago

Contrary to my confidence this afternoon, I can't reproduce step one here where the Conversion column appears to be read in as character rather than numeric. Can you give me the output of sessionInfo()?

erex commented 6 years ago

still the same result:

> library(readdst)
> tmp <- convert_project("C:\\Users\\erexstad\\Desktop\\readdst-test-projects\\songbird")
Loading required package: RODBC
> one <- run_analysis(tmp[[1]])
> fred <- test_stats(tmp[[1]], statuses=2)
Error in create.varstructure(model, region.table, sample.table, obs.table) : 
  Invalid or missing Area values for regions
In addition: Warning messages:
1: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
2: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
3: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
4: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
5: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
6: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
7: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
8: In create.varstructure(model, region.table, sample.table, obs.table) :
  Some samples not included in the analysis
> sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: i386-w64-mingw32/i386 (32-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252   
[3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RODBC_1.3-15  readdst_0.0.4

loaded via a namespace (and not attached):
 [1] RColorBrewer_1.1-2  checkmate_1.8.5     cluster_2.0.6       rstudioapi_0.7     
 [5] magrittr_1.5        acepack_1.4.1       gtable_0.2.0        minqa_1.2.4        
 [9] data.table_1.10.4-3 base64enc_0.1-3     pillar_1.2.1        htmltools_0.3.6    
[13] stringr_1.3.0       truncnorm_1.0-8     splines_3.4.3       Formula_1.2-2      
[17] lattice_0.20-35     survival_2.41-3     Rvmmin_2017-7.18    setRNG_2013.9-1    
[21] htmlwidgets_1.0     testthat_2.0.0      plyr_1.8.4          knitr_1.20         
[25] gridExtra_2.3       Matrix_1.2-12       R6_2.2.2            optimx_2013.8.7    
[29] numDeriv_2016.8-1   digest_0.6.15       colorspace_1.3-2    Rsolnp_1.16        
[33] stringi_1.1.6       yaml_2.1.18         lazyeval_0.2.1      Hmisc_4.1-1        
[37] dfoptim_2017.12-1   tibble_1.4.2        mgcv_1.8-23         compiler_3.4.3     
[41] Rcgmin_2013-2.21    rpart_4.1-11        backports_1.1.2     htmlTable_1.11.2   
[45] munsell_0.4.3       Rcpp_0.12.15        optextras_2016-8.8  BB_2014.10-1       
[49] svUnit_0.7-12       parallel_3.4.3      ggplot2_2.2.1       ucminf_1.1-4       
[53] mrds_2.1.18         latticeExtra_0.6-28 tools_3.4.3         foreign_0.8-69     
[57] nnet_7.3-12         scales_0.5.0        quadprog_1.5-5      rlang_0.2.0        
[61] nlme_3.1-131.1      grid_3.4.3
dill commented 6 years ago

Odd. I wonder if this is a Windows issue? I'll try to source some way of testing that. In the meantime can you upload the project here so I can double check?

On 10/03/2018 10:52, erex wrote:

still the same result:

| > library(readdst) > tmp <- convert_project("C:\Users\erexstad\Desktop\readdst-test-projects\songbird") Loading required package: RODBC > one <- run_analysis(tmp[[1]]) > fred <- test_stats(tmp[[1]], statuses=2) Error in create.varstructure(model, region.table, sample.table, obs.table) : Invalid or missing Area values for regions In addition: Warning messages: 1: In predict.lm(size_lm, pred_data) : prediction from a rank-deficient fit may be misleading 2: In predict.lm(size_lm, pred_data) : prediction from a rank-deficient fit may be misleading 3: In predict.lm(size_lm, pred_data) : prediction from a rank-deficient fit may be misleading 4: In predict.lm(size_lm, pred_data) : prediction from a rank-deficient fit may be misleading 5: In predict.lm(size_lm, pred_data) : prediction from a rank-deficient fit may be misleading 6: In predict.lm(size_lm, pred_data) : prediction from a rank-deficient fit may be misleading 7: In predict.lm(size_lm, pred_data) : prediction from a rank-deficient fit may be misleading 8: In create.varstructure(model, region.table, sample.table, obs.table) : Some samples not included in the analysis > sessionInfo() R version 3.4.3 (2017-11-30) Platform: i386-w64-mingw32/i386 (32-bit) Running under: Windows >= 8 x64 (build 9200) Matrix products: default locale: [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C [5] LC_TIME=English_United Kingdom.1252 attached base packages: [1] stats graphics grDevices utils datasets methods base other attached packages: [1] RODBC_1.3-15 readdst_0.0.4 loaded via a namespace (and not attached): [1] RColorBrewer_1.1-2 checkmate_1.8.5 cluster_2.0.6 rstudioapi_0.7 [5] magrittr_1.5 acepack_1.4.1 gtable_0.2.0 minqa_1.2.4 [9] data.table_1.10.4-3 base64enc_0.1-3 pillar_1.2.1 htmltools_0.3.6 [13] stringr_1.3.0 truncnorm_1.0-8 splines_3.4.3 Formula_1.2-2 [17] lattice_0.20-35 survival_2.41-3 Rvmmin_2017-7.18 setRNG_2013.9-1 [21] htmlwidgets_1.0 testthat_2.0.0 plyr_1.8.4 knitr_1.20 [25] gridExtra_2.3 Matrix_1.2-12 R6_2.2.2 optimx_2013.8.7 [29] numDeriv_2016.8-1 digest_0.6.15 colorspace_1.3-2 Rsolnp_1.16 [33] stringi_1.1.6 yaml_2.1.18 lazyeval_0.2.1 Hmisc_4.1-1 [37] dfoptim_2017.12-1 tibble_1.4.2 mgcv_1.8-23 compiler_3.4.3 [41] Rcgmin_2013-2.21 rpart_4.1-11 backports_1.1.2 htmlTable_1.11.2 [45] munsell_0.4.3 Rcpp_0.12.15 optextras_2016-8.8 BB_2014.10-1 [49] svUnit_0.7-12 parallel_3.4.3 ggplot2_2.2.1 ucminf_1.1-4 [53] mrds_2.1.18 latticeExtra_0.6-28 tools_3.4.3 foreign_0.8-69 [57] nnet_7.3-12 scales_0.5.0 quadprog_1.5-5 rlang_0.2.0 [61] nlme_3.1-131.1 grid_3.4.3 |

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/DistanceDevelopment/readdst/issues/34#issuecomment-372020972, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAHoRJCrCKM2cSvI_IZTxBJQVCTruueks5tc7BlgaJpZM4Si1kg.

-- PS. I am currently engaged in industrial action (university pensions are under threat in the UK, see https://www.ucu.org.uk/why-we-are-taking-action-over-USS) so I may be very slow to reply during the coming weeks.

erex commented 6 years ago

as requested Songbird.zip

dill commented 6 years ago

hmm, okay, can you ensure you have the latest readdst from github?

I am now getting the warnings that you get, now that I am using this data set:

Warning messages:
1: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
2: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
3: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
4: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
5: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
6: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
7: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
8: In dht(model, obs.table = analysis$env$obs.table, sample.table = analysis$env$sample.table,  :
  Point transect encounter rate variance can only use estimator P3, switching to this estimator.
9: In ticks[as.numeric(res_text) >= tolerance] <- "" :
  NAs introduced by coercion
10: In data.frame(Statistic = stats$Name, Distance_value = as.numeric(stats$Value),  :
  NAs introduced by coercion

but not the error. When we looked at this in your office I don't recall seeing the as.numeric at line 90 of units_table() but I'm guessing that's my memory. I'll look further into the dht() call just now...

dill commented 6 years ago

Okay, one possible issue here:

print(analysis$env$region.table)
  Region.Label Area
1       1980_1    0
2       1980_2    0
3       1980_3    0
4       1981_0    0
5       1981_1    0
6       1981_2    0
7       1981_3    0

wonder why this doesn't cause an error for me but does for you... mrds version perhaps? According to the above you have mrds_2.1.18, where as I am running the development version from github (displaying as 2.2.0).

In this case (where areas are 0) I guess we should not return comparisons of Nhat? What does Distance do in this situation?

FYI, the output of running the above for me:

        Statistic Distance_value    mrds_value   Difference Pass
1               n   3.110000e+02  3.110000e+02 0.000000e+00    ✓
2      parameters   2.000000e+00  2.000000e+00 0.000000e+00    ✓
3             AIC   2.462557e+03  2.462518e+03 1.564564e-05    ✓
4         Chi^2 p   7.224567e-01  7.997105e-01 1.069321e-01
5             P_a   1.409893e-01  1.409634e-01 1.833682e-04    ✓
6         CV(P_a)   9.722771e-02  9.722315e-02 4.685437e-05    ✓
7  log-likelihood  -1.229278e+03 -1.229259e+03 0.000000e+00    ✓
8           K-S p   9.291519e-01            NA           NA    ✓
9          C-vM p   1.000000e+00  9.000000e-01 1.000000e-01
10        density   2.115865e+00  5.972563e+04 2.822652e+04
11    CV(density)   1.125527e-01  1.005158e-01 1.069441e-01
erex commented 6 years ago

Guess I'll grab a 2.2.0 version of mrds and see what happens. Appears to have gotten the version bump in October of last year when you were dealing with the K-S test revamp; with estimated values of the distribution parameters

Without stratum areas provided, Distance 7 simply omits any attempt to produce abundance estimates and only reports density estimates:


                         Estimate      %CV     df     95% Confidence Interval
                        ------------------------------------------------------
 Stratum: 1. 1980_1                                         
 Half-normal/Cosine     
                 DS      1.9530       17.05   239.93  1.3993       2.7259    
                 D       1.9530       17.05   239.93  1.3993       2.7259    
 Stratum: 2. 1980_2                                         
 Half-normal/Cosine     
                 DS      1.9921       17.89   233.29  1.4042       2.8260    
                 D       2.1464       18.06   241.70  1.5082       3.0545    
 Stratum: 3. 1980_3                                         
 Half-normal/Cosine     
                 DS      2.0245       17.64   234.17  1.4341       2.8580    
                 D       2.2124       17.74   239.75  1.5640       3.1296    
 Stratum: 4. 1981_0                                         
 Half-normal/Cosine     
                 DS      2.0995       17.46   195.32  1.4917       2.9550    
                 D       2.2740       17.73   206.50  1.6077       3.2166    
 Stratum: 5. 1981_1                                         
 Half-normal/Cosine     
                 DS      2.3436       16.82   206.64  1.6861       3.2575    
                 D       2.7021       16.94   212.66  1.9394       3.7648    
 Stratum: 6. 1981_2                                         
 Half-normal/Cosine     
                 DS      1.9530       17.22   199.34  1.3941       2.7360    
                 D       2.0436       17.32   203.87  1.4560       2.8682    
 Stratum: 7. 1981_3                                         
 Half-normal/Cosine     
                 DS      1.4796       21.01   155.03 0.98134       2.2307    
                 D       1.4796       21.01   155.03 0.98134       2.2307
erex commented 6 years ago

results using mrds 2.2.0

btw: the 'as.numeric()' statement did exist in earlier versions of readdst

> library(readdst)
> tmp <- convert_project("C:\\Users\\erexstad\\Desktop\\readdst-test-projects\\songbird")
Loading required package: RODBC
> one <- run_analysis(tmp[[1]])
> fred <- test_stats(tmp[[1]], statuses=2)
Error in create.varstructure(model, region.table, sample.table, obs.table) : 
  Invalid or missing Area values for regions
In addition: Warning messages:
1: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
2: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
3: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
4: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
5: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
6: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
7: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
8: In create.varstructure(model, region.table, sample.table, obs.table) :
  Some samples not included in the analysis
> sessionInfo()
R version 3.4.3 (2017-11-30)
Platform: i386-w64-mingw32/i386 (32-bit)
Running under: Windows >= 8 x64 (build 9200)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.1252  LC_CTYPE=English_United Kingdom.1252    LC_MONETARY=English_United Kingdom.1252
[4] LC_NUMERIC=C                            LC_TIME=English_United Kingdom.1252    

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] RODBC_1.3-15  readdst_0.0.4

loaded via a namespace (and not attached):
 [1] RColorBrewer_1.1-2  checkmate_1.8.5     cluster_2.0.6       rstudioapi_0.7      magrittr_1.5        acepack_1.4.1      
 [7] gtable_0.2.0        memoise_1.1.0       minqa_1.2.4         data.table_1.10.4-3 base64enc_0.1-3     pillar_1.2.1       
[13] htmltools_0.3.6     stringr_1.3.0       curl_3.1            truncnorm_1.0-8     splines_3.4.3       Formula_1.2-2      
[19] lattice_0.20-35     survival_2.41-3     Rvmmin_2017-7.18    setRNG_2013.9-1     htmlwidgets_1.0     testthat_2.0.0     
[25] plyr_1.8.4          knitr_1.20          git2r_0.21.0        gridExtra_2.3       Matrix_1.2-12       R6_2.2.2           
[31] optimx_2013.8.7     numDeriv_2016.8-1   digest_0.6.15       colorspace_1.3-2    devtools_1.13.5     Rsolnp_1.16        
[37] stringi_1.1.6       dfoptim_2017.12-1   yaml_2.1.18         lazyeval_0.2.1      Hmisc_4.1-1         tibble_1.4.2       
[43] mgcv_1.8-23         httr_1.3.1          compiler_3.4.3      Rcgmin_2013-2.21    withr_2.1.1         rpart_4.1-11       
[49] backports_1.1.2     htmlTable_1.11.2    munsell_0.4.3       Rcpp_0.12.15        optextras_2016-8.8  BB_2014.10-1       
[55] svUnit_0.7-12       parallel_3.4.3      ggplot2_2.2.1       ucminf_1.1-4        mrds_2.2.0          latticeExtra_0.6-28
[61] tools_3.4.3         foreign_0.8-69      nnet_7.3-12         scales_0.5.0        quadprog_1.5-5      rlang_0.2.0        
[67] nlme_3.1-131.1      grid_3.4.3
dill commented 6 years ago

Okay, well that's annoying that didn't fix things. On the other hand, I do think your version of mrds is doing the right thing: attempting to calculate abundance from zero areas should throw an error BUT readdst should catch that... Will write some code to not ask dht impossible questions in readdst::test_stats.

dill commented 6 years ago

26b0688 gets around this error, but do need to implement a density estimator (for groups and individuals and their respective s.e.s) when dht isn't called.

erex commented 6 years ago

Indeed the most recent commit does sneak around the error trying to produce the abundance estimate in test_stats() However, stepping back to simply looking at the results produced by run_analysis() is unusual:

> summary(out1)

Summary for ds object 
Number of observations :  311 
Distance range         :  0  -  68 
AIC                    :  2466.627 

Detection function:
 Half-normal key function with cosine adjustment terms of order 2,3,4,5 

Detection function parameters 
Scale coefficient(s):  
            estimate         se
(Intercept) 3.362437 0.06409272

Adjustment term coefficient(s):  
               estimate         se
cos, order 2 -0.3275389 0.10075289
cos, order 3 -0.1880255 0.12151259
cos, order 4 -0.1667302 0.10897364
cos, order 5 -0.2034401 0.08989036

                    Estimate       SE        CV
Average p            4.53580  2.66565 0.5876912
N in covered region 68.56563 39.62665 0.5779376

The D7-GUI fits a single adjustment term, whereas mrds fits 4 adjustment terms, (having a reasonably similar log-likelihood) with the end result being absurd estimate of detection probability:

> out2 <- test_stats(tmp[[1]], statuses=2)
Warning messages:
1: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
2: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
3: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
4: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
5: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
6: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
7: In predict.lm(size_lm, pred_data) :
  prediction from a rank-deficient fit may be misleading
8: In ticks[as.numeric(res_text) >= tolerance] <- "" :
  NAs introduced by coercion
9: In data.frame(Statistic = stats$Name, Distance_value = as.numeric(stats$Value),  :
  NAs introduced by coercion
> print(out2)
       Statistic Distance_value    mrds_value     Difference     Pass
1              n            311           311              0 <U+2713>
2     parameters              2             5            1.5         
3            AIC 2462.556884766 2466.62744576    0.001652935 <U+2713>
4        Chi^2 p      0.7224567      0.424512      0.4124049         
5            P_a      0.1409893     4.5358003       31.17124         
6        CV(P_a)     0.09722771    0.58769121       5.044483         
7 log-likelihood      -1229.278     -1228.314              0 <U+2713>
8          K-S p      0.9291519            NA             NA <U+2713>
9         C-vM p              1             1              0 <U+2713>
erex commented 6 years ago

20Mar efforts

House wren sample project: point transects, multiple visits, with observer as potential covariate

tmp <- convert_project("C:\\Users\\erexstad\\Desktop\\readdst-test-projects\\house wren")
one <- run_analysis(tmp[[1]])
two <- run_analysis(tmp[[2]])
fred <- test_stats(tmp[[1]], statuses=2)
> summary(one)

Summary for ds object 
Number of observations :  212 
Distance range         :  0  -  50 
AIC                    :  1641.048 

Detection function:
 Half-normal key function with cosine adjustment term of order 2 

Detection function parameters 
Scale coefficient(s):  
            estimate        se
(Intercept) 3.729443 0.1957697

Adjustment term coefficient(s):  
               estimate        se
cos, order 2 -0.3535751 0.1037593

                      Estimate       SE        CV
Average p             1.254026  0.23332 0.1860567
N in covered region 169.055460 30.90474 0.1828083
> two <- run_analysis(tmp[[2]])
Error in `contrasts<-`(`*tmp*`, value = contr.funs[1 + isOF[nn]]) : 
  contrasts can be applied only to factors with 2 or more levels
> fred <- test_stats(tmp[[1]], statuses=2)
Error in data.frame(Region.Label = names(Effort.by.region), CoveredArea = CoveredArea,  : 
  arguments imply differing number of rows: 10, 0
erex commented 6 years ago

Revert to Stratify solutions project

This was used for readdst testing two years ago when it was being created

tmp <- convert_project("C:\\Users\\eric\\Documents\\My Distance Projects\\d71testing\\D70Stratify solutions")
out1 <- run_analysis(analysis = tmp[[1]])
out2 <- test_stats(tmp[[1]], statuses=1)

resulted in 

Error in merge_results(model, analysis) : 
  object 'this_region.table' not found

somehow merge_results() has lost its way

erex commented 6 years ago

Visit Tiago's amakihi project

library(readdst)
tmp <- convert_project("C:\\Users\\eric\\Documents\\My Distance Projects\\d71testing\\D70fTAMAUK07")
out1 <- run_analysis(analysis = tmp[[1]])
out2 <- test_stats(tmp[[1]], statuses=2)

first oddity is that out is of class ddf_analyses but it is a list of length 1292, most of which are empty. That's not right, but does not throw an error.

Eventually however, 

> out2 <- test_stats(tmp[[1]], statuses=2)
Error in if (model$meta.data$point) { : argument is of length zero
erex commented 6 years ago

stratify solution project and amakihi Distance projects (in zip format) found at https://my.pcloud.com/publink/show?code=XZG1Y87Z9auz57yxph8306wfxxER84Jc6WpV

and

https://my.pcloud.com/publink/show?code=XZc1Y87ZWVLWUCcvDX8Ue7R1h7mqbQxGhCLy

respectively

dill commented 6 years ago

Stratify project error should be fixed in b2081c5.

erex commented 6 years ago

Fast work on stratify solution problem. Running the first analysis (full geographic stratification) generates this comparison agains the Distance7 result

> out2
        Statistic   Distance_value       mrds_value       Difference     Pass
1               n               88               88                0 <U+2713>
2      parameters                4                4                0 <U+2713>
3             AIC 315.789489746094 315.789836548511   0.000001098354 <U+2713>
4  log-likelihood        -153.8947        -153.8949                0 <U+2713>
5         density         0.020906       0.02755602        0.3180915         
6     CV(density)        0.2778745       0.25907508       0.06765434         
7     individuals            14954    19711.2633153        0.3181265         
8 CV(individuals)        0.2778745       0.25907508       0.06765434

If the likelihood is spot-on, I assume this means the detection function and hence P_a ought to be spot on. However, as there are stratum-specific P_a, they aren't presented in the summary table. I'll refresh my memory on how density estimate is being produced for this analysis

 Encounter rate by stratum           
 Detection probability modelled by stratum, and             
  estimated by stratum           
 Density by stratum           
 Pooled estimate of density is made from area weighted stratum estimates
dill commented 6 years ago

[master d1f501c] fix the 193 empty list elements issue

dill commented 6 years ago

As to the second amakihi error... it looks like there are some issues with how this analysis is put together. Is it the case that the detection function is being estimated per stratum? readdst seems to be trying to combine analyses here, so it's possible this is wrong? If you can give me the spec. in DISTANCE then I can check with what readdst thinks it should be doing.

erex commented 6 years ago

This, I believe, is what the first amakihi analysis (number 139, labelled "HN by strat f0 pooled w82.5") is trying to do

 Parameter Estimation Specification
 ----------------------------------
 Encounter rate by stratum           
 Detection probability for all data combined
 Density by stratum           
 Pooled estimate of density is made from area weighted stratum estimates