HOG failing for 3D arrays

[dslate1]

I am trying to apply the HOG function from OpenImageR_1.1.7 to a 3D array, but it fails with an error message. Here is the log of a small sample program demonstrating the problem:

Script started on Mon 14 Sep 2020 10:19:09 PM CDT $ ./hog-test.R [1] "#!/usr/bin/env Rscript"
[2] "system( \"cat hog-test.R\", intern = T)"
[3] "library( OpenImageR)"
[4] "sessionInfo()"
[5] "Mag <- array( runif( 30), c( 5, 3, 2))"
[6] "cat( \"\nclass( Mag):\n\")"
[7] "class( Mag)"
[8] "cat( \"\ndim( Mag):\n\")"
[9] "dim( Mag)"
[10] "cat( \"\nMag:\n\")"
[11] "print( Mag)"
[12] "cat( \"\n\")"
[13] "Hog <- try( OpenImageR::HOG( Mag, cells = 3, orientations = 6))"
[14] "if( class( Hog)[ 1] == \"try-error\") {\t\t# if HOG failed"
[15] " cat( sprintf( \"OpenImageR::HOG error %s\n\", as.character( Hog)))" [16] "} else {"
[17] " cat( \"Hog: \", Hog, \"\n\")"
[18] "}"
R version 4.0.1 (2020-06-06) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 16.04.6 LTS

Matrix products: default BLAS: /usr/local/lib/R/lib/libRblas.so LAPACK: /usr/local/lib/R/lib/libRlapack.so

locale: [1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] OpenImageR_1.1.7

loaded via a namespace (and not attached): [1] shiny_1.2.0 compiler_4.0.1 magrittr_1.5 R6_2.4.1
[5] promises_1.0.1 later_0.7.5 htmltools_0.3.6 Rcpp_1.0.4.6
[9] jpeg_0.1-8.1 grid_4.0.1 tiff_0.1-5 digest_0.6.25
[13] xtable_1.8-2 httpuv_1.4.5.1 mime_0.9 png_0.1-7

class( Mag): [1] "array"

dim( Mag): [1] 5 3 2

Mag: , , 1

      [,1]      [,2]      [,3]

[1,] 0.4173635 0.7906940 0.4889265 [2,] 0.8768264 0.9576416 0.6127565 [3,] 0.7443837 0.2411659 0.1802162 [4,] 0.3481922 0.5618764 0.2384123 [5,] 0.6923521 0.1958070 0.5041096

, , 2

      [,1]      [,2]      [,3]

[1,] 0.6272674 0.3110115 0.3748221 [2,] 0.2367036 0.5387666 0.8246165 [3,] 0.2820331 0.9368423 0.9716071 [4,] 0.5958321 0.2095327 0.1982946 [5,] 0.7164110 0.5976341 0.2446890

Error in OpenImageR::HOG(Mag, cells = 3, orientations = 6) : valid types of input are matrix, data frame and 3-dimensional array OpenImageR::HOG error Error in OpenImageR::HOG(Mag, cells = 3, orientations = 6) : valid types of input are matrix, data frame and 3-dimensional array

$ exit exit

Script done on Mon 14 Sep 2020 10:19:16 PM CDT

Any ideas on what is going wrong? Thanks.

[issue-label-bot[bot]]

Issue-Label Bot is automatically applying the label kind/bug to this issue, with a confidence of 0.88. Please mark this comment with :thumbsup: or :thumbsdown: to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[mlampros]

Hi @dslate1, (see my edited answer)

there are 2 valid input data types for the OpenImageR::HOG function,

• a matrix or data.frame (2-dimensional)
• a 3-dimensional array, but the 3rd dimension must consist of 3 image bands

In your array the 3rd dimension consists of 2 image bands (assuming your input is an image). You can find example cases in the documentation of the package and in the first vignette:

path = system.file("tmp_images", "1.png", package = "OpenImageR")
image = readImage(path)
res = HOG(image, cells = 3, orientations = 6)

[dslate1]

Ah, I see. Thanks @mlampros.

[mlampros]

@dslate1,

Edited:

I took a look to the code once again because I ported it from python using Rcpp a while back. The main purpose of the HOG function is to create features from 2-dimensional data as is the case with the MNIST images (explained in the first vignette). Therefore, in case that a user has an RGB image (3-dims) then the image is first converted to a matrix (2-dims) and then the HOG function is applied.

In your case where you don't have an RGB image (3-dims) you can still use the HOG_apply function assuming each image band in your array has the same number of rows and columns (the HOG_apply function is also explained in first vignette),

Mag <- array( runif( 30), c( 5, 3, 2))

res_mag = OpenImageR::HOG_apply(object = Mag, cells = 3,
                                orientations = 6, 
                                rows = nrow(Mag), 
                                columns = ncol(Mag), 
                                threads = 1)

str(res_mag)
# num [1:2, 1:54] 0 0 0 0 0 ...

The HOG_apply function just applies the HOG function to each image band. Thus, each row of the output matrix (res_mag) represents the created features for each input image band.

[dslate1]

Thanks again @mlampros. I had considered doing something like that, but I thought there might be some kind of HOG variant for 3D images, as opposed to the 2D kind that I have used before. I will try your suggestion and find a way to aggregate the features across the bands, of which in my case there are around 48.