benjamin-allevius / scanstatistics

An R package for space-time anomaly detection using scan statistics.
GNU General Public License v3.0
49 stars 10 forks source link

scan_permutation and scan_eb_poisson #4

Open lhdatalab opened 6 years ago

lhdatalab commented 6 years ago

Hi Benjamin,

I was trying to us both of these functions in the scanstatistics package in a jupyter R notebook. It crashed the kernel for scan_eb_poisson (small sample based on simulated data and NMgeo), and was a long running operation for scan_permutations. Any ideas. 16GB RAM, 4core CPU, Windows 10. `` Warning message in seq_len(nrow(x)): "first element used of 'length.out' argument"

Error in seq_len(nrow(x)): argument must be coercible to non-negative integer Traceback:

  1. scan_permutation(counts = counts2, zones = zones, population = NULL, . n_mcsim = 1, max_only = TRUE)
  2. flipud(population)
  3. rev(seq_len(nrow(x)))

    ' @keywords internal

    flipud <- function(x) { x[rev(seq_len(nrow(x))), , drop = FALSE]

counts2.zip

ghost commented 6 years ago

Would you mind sharing the full code; particularly how you define the variable zones?

lhdatalab commented 6 years ago

Hi Benjamin,

The csv file I sent is a sample of the data?

knn_mat <- coords_to_knn(ferg_latlon_n[c("min_lon", "min_lat")], 3)

zones <- knn_zones(knn_mat)

counts <- ferg_latlon_n[c("time", "location", "count1")]

for (i in 1:ncol(counts2)) {

counts2[ , i] <- ifelse(is.na(counts2[ , i]), 0, counts2[ , i])

print (i)

}

population = matrix(rnorm(504, 50, 2), nrow = 6)

scan_permutation(counts = counts2,

zones = zones,

population = NULL,

n_mcsim = 1,

max_only = TRUE)

On Wed, Nov 7, 2018 at 8:14 AM Benjamin Allévius notifications@github.com wrote:

Would you mind sharing the full code; particularly how you define the variable zones?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/BenjaK/scanstatistics/issues/4#issuecomment-436618403, or mute the thread https://github.com/notifications/unsubscribe-auth/AQuK9iBxeXCR7nS5-hlR_XJcVdMwEHpPks5ustyYgaJpZM4YRTMR .

lhdatalab commented 6 years ago

Hi Benjamin,

The data came from a shape file and I just sampled enough rows to test the scan_permutation and scan_eb_poisson functions.

ferg_latlon_n <- ferguson8m_latlon2 %>%

                dplyr::select(one_of(c("City_code", "lon", "lat",

"yday", "ID"))) %>%

                                   mutate(tday = case_when(

                                              yday  == 221 ~ 1,

                                              yday == 222 ~ 2,

                                              yday == 223 ~ 3,

                                              yday == 224 ~ 4,

                                              yday == 225 ~ 5,

                                              yday == 226 ~ 6)) %>%

                                    group_by(City_code, tday) %>%

                                             dplyr::summarise(n = n(),

                                                             min_lon =

min(lon),

                                                             min_lat =

min(lat)) %>%

                                            dplyr::rename(count1 = n,

time = tday, location = City_code) %>%

                                                as.data.frame()

knn_mat <- coords_to_knn(ferg_latlon_n[c("min_lon", "min_lat")], 3)

zones <- knn_zones(knn_mat)

counts <- ferg_latlon_n[c("time", "location", "count1")]

for (i in 1:ncol(counts2)) {

counts2[ , i] <- ifelse(is.na(counts2[ , i]), 0, counts2[ , i])

print (i)

}

population = matrix(rnorm(504, 50, 2), nrow = 6)

scan_permutation(counts = counts2,

zones = zones,

population = NULL,

n_mcsim = 1,

max_only = TRUE)

On Wed, Nov 7, 2018 at 8:14 AM Benjamin Allévius notifications@github.com wrote:

Would you mind sharing the full code; particularly how you define the variable zones?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/BenjaK/scanstatistics/issues/4#issuecomment-436618403, or mute the thread https://github.com/notifications/unsubscribe-auth/AQuK9iBxeXCR7nS5-hlR_XJcVdMwEHpPks5ustyYgaJpZM4YRTMR .