search

verbling / assetflow

Asset deployment pipeline
MIT License
44 stars 10 forks source link

AssetsS3 Operation failed: TypeError: Cannot call method 'indexOf' of undefined #1

Closed paulbjensen closed 11 years ago

paulbjensen commented 11 years ago

Hi, we're encountering an error when attempting to upload the assets to S3 (assetsS3:all):

Operation failed: TypeError: Cannot call method 'indexOf' of undefined

For debug info, our Gruntfile.js file:

module.exports = function(grunt) {

      // Project configuration.
      grunt.initConfig({
        pkg: grunt.file.readJSON('package.json'),
        requirejs: {
          compile: {
            options: {
              baseUrl: "public/js",

              mainConfigFile: "public/js/config.js",

              modules: [
                // TODO: The account module needs reworking before we can optimise it.
                //       The index module makes use of account and therefore can't yet be
                //       optimised. This is caused by the require('/templates.js) call.
                //       The optimiser cannot resolve the path as it's actually an app route

                // Common modules
                //{name: 'account'}

                // Sections
                //{name: 'index'},
                {name: 'eventView'},
                {name: 'eventViewMobile'},
                {name: 'eventChannelMobile'},
                {name: 'eventSessions'},
                {name: 'eventSessionsMobile'},
                {name: 'eventSession'},
                {name: 'eventSessionMobile'},
                {name: 'eventPage'},
                {name: 'eventPageMobile'}
              ],

              findNestedDependencies: true,
              removeCombined: true,
              preserveLicenseComments: false,

              dir: "public/dist/js"
            }
          }
        },

        assets: {
          options: {
            rel: 'public/dist/',
            truncateHash: 8,
            manifest: 'temp/manifest.json'
          },
          all: {
            src: 'public/dist/**',
            dest: 'temp/'
          }
        },

        assetsS3: {
          options: {
            // no debug info
            debug: true,
            // enable checking md5 hashes by performing S3 HEAD requests.
            checkS3Head: true,
            // the manifest file
            manifest: 'temp/manifest.json',

            // aws credentials, marked as XXX 
            key: 'XXX',
            secret: 'XXX',
            bucket: 'XXX',

            // Enable public access
            access: 'public-read',

            // don't show the fancy progress indicator.
            progress: false
          },
          all: {
            // These options override the defaults
            options: {
              // limit concurent uploads to 100
              maxOperations: 100
            },
            // the 'upload' option key is required
            upload: {
              // all files from temp/
              src: 'temp/**',
              // a prefix folder on S3
              dest:  'assets/',
              // upload assets with a looong expire Cache-Control header.
              headers: {'Cache-Control': 'max-age=31536000, public'}
            }
          }
        }
      });

      grunt.loadNpmTasks('grunt-contrib-requirejs');
      grunt.loadNpmTasks('assetflow');

      // Default task(s).
      grunt.registerTask('optimise', ['requirejs']);

    };

Are we doing something dumb in our config?

thanpolas commented 11 years ago

try adding the prepend: '/' options in the assets task.

Please send the whole error stack that you saw so i can pinpoint file / lineno that the error originated...

paulbjensen commented 11 years ago

Hi, thanks for your suggestion. I tried it, but the error still occurs.

Here is the stack trace that I get:

Running "assetsS3:all" (assetsS3) task

Check of S3 Head requested. 43 assets to check. Starting...
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/account.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/accounts/accounts.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/accounts/models/user.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/accounts/views/login.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/accounts/views/signup.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/admin/questions/index.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/build.txt
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/channels/templates/channel-item-mobile.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/channels/templates/channel-tab.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/channels/templates/channel.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/channels/templates/session-current.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/channels/templates/session-next.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/channels/templates/session.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/channels/views/session-mobile.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/chassis.io.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/config.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/engine.io.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/experience/templates/modal.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/experience/templates/toolbar.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/helpers/slughead.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/highlights/templates/highlight.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/index.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/json2.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/day-filter.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/day.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/days-filter-mobile.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/days-filter.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/empty.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/session.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/timeslot.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/track-filter.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/tracks-filter-mobile.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/sessions/templates/tracks-filter.ejs
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventPageMobile.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventSessionMobile.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/vendor/jwplayer/jwplayer.html5.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventChannelMobile.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventPage.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventSession.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventSessions.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventSessionsMobile.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventViewMobile.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/eventView.js
DEBUG ::  s3stat.Task._s3Head() :: Req s3.HEAD ops,max:0,100 for: /js/vendor/jwplayer/jwplayer.flash.swf
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 1, assets/js/accounts/models/user-0edeab8e.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 2, assets/js/accounts/accounts-f1c7db1a.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 3, assets/js/accounts/views/login-5d11efa8.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 4, assets/js/account-a47c1c6b.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 5, assets/js/accounts/views/signup-68c7a3b1.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 6, assets/js/admin/questions/index-742790df.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 7, assets/js/channels/templates/channel-tab-05a33088.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 8, assets/js/build-8269617c.txt
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 9, assets/js/channels/templates/session-current-c95b7719.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 10, assets/js/channels/templates/channel-item-mobile-9e4cb79d.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 11, assets/js/channels/templates/channel-ac8f77aa.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 12, assets/js/channels/templates/session-next-abe466dd.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 13, assets/js/channels/templates/session-0c253a0e.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 14, assets/js/channels/views/session-mobile-b22c09ae.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 15, assets/js/chassis.io-476719d4.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 16, assets/js/config-d65ec2a0.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 17, assets/js/helpers/slughead-3a9bb47f.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 18, assets/js/experience/templates/toolbar-7aee27b8.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 19, assets/js/experience/templates/modal-96ed4a7c.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 20, assets/js/engine.io-c2373c32.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 21, assets/js/highlights/templates/highlight-0b3900cb.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 22, assets/js/index-5599a15a.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 23, assets/js/json2-dbe85c84.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 24, assets/js/sessions/templates/day-f8f1c668.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 25, assets/js/sessions/templates/day-filter-8bfc3822.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 26, assets/js/sessions/templates/days-filter-mobile-7d16e032.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 27, assets/js/sessions/templates/empty-e112bbae.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 28, assets/js/sessions/templates/days-filter-7d16e032.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 29, assets/js/sessions/templates/session-a66413c4.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 30, assets/js/sessions/templates/timeslot-617c7e2f.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 31, assets/js/sessions/templates/track-filter-7b0f816d.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 32, assets/js/eventPageMobile-d6af7775.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 33, assets/js/sessions/templates/tracks-filter-mobile-16fc98ed.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 34, assets/js/eventSessionMobile-ef991a87.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 35, assets/js/sessions/templates/tracks-filter-9daef588.ejs
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 36, assets/js/vendor/jwplayer/jwplayer.html5-36b106b1.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 37, assets/js/eventChannelMobile-815cc08d.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 38, assets/js/eventSessions-ea7123bf.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 39, assets/js/eventSession-5d4f728d.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 40, assets/js/eventPage-f1ee83d9.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 41, assets/js/eventSessionsMobile-66420e03.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 42, assets/js/eventViewMobile-26ac9e3e.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 43, assets/js/eventView-eb94606c.js
DEBUG ::  s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 44, assets/js/vendor/jwplayer/jwplayer.flash-7dda174a.swf

S3 Check complete. Total checks done: 44 New files to upload: 44 elapsed time: 0m 1s 96ms
>> Operation failed: TypeError: Cannot call method 'indexOf' of undefined
Warning: Task "assetsS3:all" failed. Use --force to continue.

Aborted due to warnings.
thanpolas commented 11 years ago

so 2 remarks so far:

First, The S3 Head operation fails:

DEBUG :: s3stat.Task._s3Head() :: AWS Response: err,http,count,asset: null, 404, 27, assets/js/sessions/templates/empty-e112bbae.ejs

which suggests that you need the prepend option, experiment a bit with the options till you get that right...

Second, please try to run Grunt with the --trace option on the command line so it generates a proper trace =)

thanpolas commented 11 years ago

@paulbjensen i guess this project isn't on GH so i can clone and try to replicate the issue ha?

thanpolas commented 11 years ago

@paulbjensen any updates?

paulbjensen commented 11 years ago

Hi,

sorry for not replying earlier. I tried your suggestion, but Grunt is not providing any more detail beyond the stack trace posted in one of the previous comments.

I think I'm going to have to experiment a bit more, do some console.logging.

The repo is a private one, but I'd be happy to create a duplicate repo without the private code, so I can figure out why this is happening. I will do that now.

thanpolas commented 11 years ago

that's odd... --stack should definitely show more information...

Waiting for the test repo so i can see what's going on...

paulbjensen commented 11 years ago

ahh, I ran --trace. Trying --stack now.

thanpolas commented 11 years ago

ops, my bad... sorry

paulbjensen commented 11 years ago 
>> Operation failed: TypeError: Cannot call method 'indexOf' of undefined
Warning: Task "assetsS3:all" failed. Use --force to continue.
Error: Task "assetsS3:all" failed.
    at Task.<anonymous> (/Users/paulbjensen/Work/axisto/axisto-events/node_modules/grunt/lib/util/task.js:197:15)
    at Object.Task.runTaskFn.context.async [as _onTimeout] (/Users/paulbjensen/Work/axisto/axisto-events/node_modules/grunt/lib/util/task.js:225:33)
    at Timer.list.ontimeout (timers.js:101:19)

I'll setup the public repo now. Sorry for the wait.

thanpolas commented 11 years ago

That stack doesn't really say much to me unfortunately :( I understand there's asynchronicity involved so a complete stack is not possible... Any ideas how we can better understand what's wrong using --stack @cowboy, @tkellen ?

@paulbjensen i'll wait for it and dig into when ready

paulbjensen commented 11 years ago

Hi, sorry for the wait. I've replicated the issue in this app: https://github.com/axisto-live/assetflow-issue-repo

thanpolas commented 11 years ago

@paulbjensen thank you very much for your thorough sample, it really helped find that bug... v0.1.4 has been released and is now on NPM :)

paulbjensen commented 11 years ago

You're welcome, thanks for your help. Am I ok to remove the repo I created for the issue?

thanpolas commented 11 years ago

Yep, i'm done with it, it's up to you :)

On Friday, May 10, 2013, Paul Jensen wrote:

You're welcome, thanks for your help. Am I ok to remove the repo I created for the issue?

— Reply to this email directly or view it on GitHubhttps://github.com/verbling/assetflow/issues/1#issuecomment-17709995 .

Thanasis Polychronakis

http://thanpol.as/http://thanpol.as/?utm_source=email&utm_medium=email&utm_campaign=email