1.0.0 busting multiple times

[olimortimer]

Since upgrading from 0.4.13 to 1.0.0, I'm finding that that busted files are being busted multiple times, so I end up with lots of the same file:

projects.c1979eef7285467a.js
projects.c1979eef7285467a.c1979eef7285467a.js
projects.c1979eef7285467a.c1979eef7285467a.c1979eef7285467a.js
projects.c1979eef7285467a.c1979eef7285467a.c1979eef7285467a.c1979eef7285467a.js

How do I stop this from happening?

Maybe I haven't quite got my Gruntfile.js setup correctly:

// 0.4.13
cacheBust: {
    options: {
        baseDir: '.',
        rename: false,
        enableUrlFragmentHint: true
    },
    assets: {
        files: [{
            src: [
                'skin/*/view/**/*.php'
            ]
        }]
    }
},

// 1.0.0
cacheBust: {
    assets: {
        options: {
            assets: ['skin/**/css/*.css', 'skin/**/css/*.scss', 'skin/**/css/*.min.css', 'skin/**/scripts/*.js', 'skin/**/scripts/*.min.js'],
            baseDir: '.',
        },
        files: [{
            src: [
                'skin/*/view/**/*.php'
            ]
        }]
    }
},

Any help would be appreciated, and sorry if I'm missing something obvious to solve all these issues.

[benhoIIand]

Hi @olimortimer.

Looks like you were using the rename: false option with the 0.4.x version, meaning that the file would have a query string appended instead of the hash appended to the filename. The rename option was removed in the new 1.x.x version. If you go over to #147 you can read about this change and similar ones.

The problem you're seeing is because the task is processing the same files over and over. It's recommended that the process for running these tasks is:

copy files to a new folder -> process files in there -> deploy that version of the code

Following this mentality means that only a copy of the source files are altered. In your case, it looks like the raw source file is being altered every time the task runs.

In previous versions, there was an attempt to detect when a reference to a file was already "busted" but it proved very difficult to do accurately and also meant that the above process was probably not being taken into consideration.

If you provide more details of the project and setup then I might be able to help out

[johny-gog]

I had that issue too (in a single run, it was very long), but don't know why. I see that 'skin/**/css/*.css' rule is catching the same files as 'skin/**/css/*.min.css' so maybe that's the reason here? Same for .js.

[benhoIIand]

Are either of you able to create a simple test to show this happening? Could you also post the JSON output by setting jsonOutput to true in your options.

The list of files to be hashed should be set in reverse alphabetically order so we shouldn't be seeing this issue.

[johny-gog]

I was able to reproduce mine.

I have multiple directories (Symfony bundles) that contains assets. I've set a config object for each bundle having base dir of all bundles (so every bundle goes through that same assets)

cacheBust = { options: { baseDir: 'someRootDir' } }
cacheBust['bundle1'] = { src: 'someRootDir/bundle1/public/**/*.css' }
cacheBust['bundle2'] = { src: 'someRootDir/bundle2/public/**/*.css' }

Then it would go hashing all the assets twice. That is part of JSON output:

    "Website/CommonBundle/Resources/assets/icons/plus.svg": "Website/CommonBundle/Resources/assets/icons/plus.a2f25fe6.svg",
    "Website/CommonBundle/Resources/assets/icons/plus.a2f25fe6.svg": "Website/CommonBundle/Resources/assets/icons/plus.a2f25fe6.a2f25fe6.svg",
    "Website/CommonBundle/Resources/assets/icons/plus.a2f25fe6.a2f25fe6.svg": "Website/CommonBundle/Resources/assets/icons/plus.a2f25fe6.a2f25fe6.a2f25fe6.svg",

I've fixed it by having a separate baseDir for each bundle (it had to point to /public/ dir because otherwise it wouldn't catch those files in .css).

---

Of course this is not the issue with @olimortimer - although I would change this line:

assets: ['skin/**/css/*.css', 'skin/**/css/*.scss', 'skin/**/css/*.min.css', 'skin/**/scripts/*.js', 'skin/**/scripts/*.min.js'],

to

assets: ['skin/**/css/*.css', 'skin/**/scripts/*.js'],

Because why hash .scss? Plus .min.css, .css, .min.js and .js are matched by above. I guess issue here is running cacheBust on the same files again and again. Maybe the files are watched by grunt and watch is running cacheBust task? If so, maybe not hashing .scss would help just a little?

[jenzer]

Hi, I ran into the same problem. sorry, don't understand the comment https://github.com/hollandben/grunt-cache-bust/issues/180#issuecomment-180350232 what do you mean by your mentality?

[benhoIIand]

What I mean by mentality is a change in the way your application builds. So many web applications have a build process that does some form of pre and post processing of these files. To make sure the build steps will produce the same results no matter how many times you run them, it's best to follow this mentality:

copy files to a new folder -> pre/post process files in there -> deploy that version of the code

This will keep your source files the same. I'm assuming the problem both are having is that you running this cachebust task against the source files for the first time (it works) then running it again against the now changed source files, causing it to hash the file again.

[j4m3s]

@hollandben I understand the thinking on the workflow for the src files (javascript and css etc, the files that are being renamed because their hash has changed), but are you suggesting a separate copy of the html must be maintained too?

[djjohnjosephuk]

I'm also getting this issue, and can't think of a workaround to get my configuration working.

I have the following set up:

File structure:

 src/
     js/
     css/
     ...
 public/
     js/
     css/

 views/
     template.html
     ...

So src/ contains some css and javascript source files. I have a gruntfile that watches for file changes in those directories, and runs a few tasks to concatenate and minify the files before putting them in the public/ folder. Finally, the files within views/ are 'busted' to replace the links to the hashed files.

The gruntfile:

 module.exports = function(grunt) {
grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    clean: {
        assets: ['site/js/**/*.js', 'site/css/**/*.css'],
        options: {
            force: true
        }
    },
    concat: {
        options: {
            separator: ';'
        },
        dist: {
            src: ['src/js/*.js'],
            dest: 'site/js/<%= pkg.name %>.js'
        }
    },
    cssmin: {
        target: {
            files: [{
                expand: true,
                cwd: 'src/css',
                src: ['*.css', '!*.min.css'],
                dest: 'site/css',
                ext: '.min.css'
            }]
        }
    },
    uglify: {
        options: {
            banner: '/*! <%= pkg.name %> <%= grunt.template.today("dd-mm-yyyy") %> */\n'
        },
        dist: {
            files: {
                'site/js/<%= pkg.name %>.min.js': ['<%= concat.dist.dest %>']
            }
        }
    },
    jshint: {
        files: ['Gruntfile.js', 'src/js/*.js'],
        options: {
            reporterOutput: ""
        }
    },
    watch: {
        grunt: { files: ['Gruntfile.js'] },
        css: {
            files: ['src/css/*.css'],
            tasks: ['default']
        },
        js: {
            files: ['src/js/*.js'],
            tasks: ['default']
        }
    },
    cacheBust: {
        options: {
            assets: ['js/*.js', 'css/*.css'],
            baseDir: './site/',
            deleteOriginals: true
        },
        taskName: {
            files: [{
                expand: true,
                cwd: 'views/',
                src: ['layout/*.html.php', 'fragment/*.html.php', '**/*.html.php']
            }]
        }
    }
});

grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-contrib-uglify');
grunt.loadNpmTasks('grunt-contrib-jshint');
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-cssmin');
grunt.loadNpmTasks('grunt-cache-bust');

grunt.registerTask('default', ['jshint', 'clean', 'concat', 'cssmin', 'uglify', 'cacheBust']);
 };

I have added the contrib-clean task to clean the public/ folders as prior to that, each file was being hashed each iteration (remove the clean task to see this happening using that gruntfile). I had hoped cleaning the folders before rebuilding and minifying them, then finally hashing them would solve the issue but the problem I have now is the cache bust cannot replace versions of the files in the templates as presumably the current/existing hashed file has been deleted by the clean task so it cannot find that reference in the file.

So i'm stuck in a catch 22, either the files are hashed multiple times or the references in the template files can't be changed.

Is there a workaround, or am I missing something really simple/can't see the wood for the trees? I'm guessing the main problem is the matching rules for the assets, they are always going to match with existing hashed files and just rehash them again, as well as creating another copy of the file. So 5 files become 10, 10 becomes 15, etc.....

[djjohnjosephuk]

After a few hours tinkering with this, I've managed to hack around this issue by using grunt-text-replace prior to the cache bust running. What this does is replace the references back to the same name that the concat and minify tasks use, so when the cache bust runs it has the correct reference to be able to replace. It does mean replacing the references twice with an extra task running, but it at least now supports that workflow. I'm putting this here incase anyone else comes across a similar issue.

The gruntfile:

 module.exports = function(grunt) {
grunt.initConfig({
    pkg: grunt.file.readJSON('package.json'),
    clean: {
        assets: ['site/js/**/*.js', 'site/css/**/*.css'],
        options: {
            force: true
        }
    },
    concat: {
        js: {
            src: ['src/js/*.js'],
            dest: 'site/js/<%= pkg.name %>.js'
        },
        css: {
            src: ['src/css/*.css'],
            dest: 'site/css/<%= pkg.name %>.css'
        }
    },
    cssmin: {
        target: {
            files: [{
                expand: true,
                cwd: 'site/css',
                src: ['*.css'],
                dest: 'site/css',
                ext: '.min.css'
            }]
        }
    },
    uglify: {
        options: {
            banner: '/*! <%= pkg.name %> <%= grunt.template.today("dd-mm-yyyy") %> */\n'
        },
        dist: {
            files: {
                'site/js/<%= pkg.name %>.min.js': ['<%= concat.js.dest %>']
            }
        }
    },
    jshint: {
        files: ['Gruntfile.js', 'src/js/*.js'],
        options: {
            reporterOutput: ""
        }
    },
    watch: {
        grunt: { files: ['Gruntfile.js'] },
        css: {
            files: ['src/css/*.css'],
            tasks: ['default']
        },
        js: {
            files: ['src/js/*.js'],
            tasks: ['default']
        }
    },
    replace: {
        replaceExistingCachedAssets: {
            src: ['layout/*.html.php', 'fragment/*.html.php', '**/*.html.php'],
            overwrite: true,
            replacements: [{
                from: /<%= pkg.name %>\.[a-z0-9.]+\.js/g, // NOTE: <%=%> Doesn't work here, enter manual value eg 'MYPackagename'
                to: "<%= pkg.name %>.min.js"
            },
            {
                from: /<%= pkg.name %>\.[a-z0-9.]+\.css/g, // NOTE: <%=%> Doesn't work here, enter manual value eg 'MYPackagename'
                to: "<%= pkg.name %>.min.css"
            }]
        }
    },
    cacheBust: {
        options: {
            assets: ['js/<%= pkg.name %>.min.js', 'css/<%= pkg.name %>.min.css'],
            baseDir: './site/',
            deleteOriginals: true,
            createCopies: true
        },
        taskName: {
            files: [{
                expand: true,
                cwd: 'views/',
                src: ['layout/*.html.php', 'fragment/*.html.php', '**/*.html.php']
            }]
        }
    }
});

grunt.loadNpmTasks('grunt-contrib-clean');
grunt.loadNpmTasks('grunt-contrib-uglify');
grunt.loadNpmTasks('grunt-contrib-jshint');
grunt.loadNpmTasks('grunt-contrib-watch');
grunt.loadNpmTasks('grunt-contrib-concat');
grunt.loadNpmTasks('grunt-contrib-cssmin');
grunt.loadNpmTasks('grunt-text-replace');
grunt.loadNpmTasks('grunt-cache-bust');

grunt.registerTask('default', ['jshint', 'clean', 'replace', 'concat', 'cssmin', 'uglify', 'cacheBust']);
};

The above allows a workflow whereby you have your asset source files outside of the public folder for editing, and have them watched by grunt for changes. Upon change, they are concatenated, minified and published to a public folder for inclusion in view templates. The view templates have their references replaced by grunt-text-replace back to the file name that the concat and minify produces (For example package.json.name.min.js), so that when cache bust runs, it cache busts' that file name, and the reference in the view templates can be replaced as it's the same file name that cache bust created the hashed file from.

So to sum up the above will work as follows:

Application structure:

src/css/file.css // Source file, outside of public root
public/css/file.min.HASHED.css // publically available css file for referencing
views/template.html // view template which contains link to the hashed css file.

1. A change is made to a CSS source file
2. grunt watch runs the defaulttask
3. The public/css folder is cleared
4. grunt-text-replace replaces all old references to the current hashed/busted files - eg replaces public/css/package.json.name.min.HASH.css back to public/css/package.json.name.min.css in the view templates
5. grunt-contrib-concat joins all src/*.css files together into one file, and places it in public/css
6. grunt-contrib-cssmin minifies the content of the new single css file, calling it public/css/package.json.name.min.css
7. grunt-cache-bust then hashes the above file - new name public/css/package.json.name.min.NEW_HASH.css
8. Finally grunt-cache-bust replaces references of the file it hashed in the view templates - (public/css/package.json.name.min.css) to the new hashed file name (public/css/package.json.name.min.NEW_HASH.css)

This allows continuous development of asset files, ensuring the application has bang up to date changes with the files cached until they are changed again. Version control is also unaffected as the source files are can be in the repository and the public files ignored, created after a checkout followed by executing grunt as part of a post-build step (for example from Jenkins or some other build/deployment tool)

Great plugin @hollandben , coupled with a few other grunt tasks this has turned out to be a great way of automating the caching process with no impact on source files and no manual renaming and moving files to do.

[revalgovender]

@djjohnjosephuk I am running in to the exact catch 22 scenario. I can't believe this has not being sorted, it seems to me as if this package does not do what it is intended to do.

I appreciate the hard work on the project, but I have spent hours trying to figure this out. Having to "hack/workaround" this issue is a big problem to me. Are we missing something? 1000's of people seem to download this plugin every week. So I can only assume I am at fault?

[benhoIIand]

@revalgovender @djjohnjosephuk this plugin's purpose is to hash files and change the references in the specified files. It does some extra minor things as requested by some users over time.

Prior to version 1.x.x, the plugin would do some very crude find and replace based on a regular expression to swap old hashes for new ones. This caused quite a few problems for both others and myself so it was not kept when pushing out version 1.x.x

As you have probably read in a few issues, there was a change to follow an immutable approach as well, i.e. you can run the task 1000 times and you'll get the same output. This meant forcing users to figure out their own approach to "cleaning up" there workspace. I have given some advice in other issues about doing this but clearly it doesn't fit with some users build systems and methodologies.

I can't believe this has not being sorted...

Firstly, I haven't used grunt-cache-bust for quite some time as I use alternative task runners now so the level of support has understandable dropped; this is an open source project so you have the ability to contribute or create your own fork of the code. I have modelled the latest version of the plugin to be similar to gulp-rev so go and take a look at that ecosystem and how they recommend you set a build system up.

...it seems to me as if this package does not do what it is intended to do.

In terms of the functionality, it does do what is intended. Maybe some of the language in the readme should be changed to point out that it will not re-hash your already processed files - I'm open to suggestions and pull requests

[revalgovender]

Thanks for your response @hollandben . Again, appreciate the work on the plugin so we can use it for free.

Unfortunately, it does not work as I need it to work. I guess I was confused by the description. It does add a hash to the file and it does update the reference. But, if you run it again, then you run into issues as described above. I just assumed I could do it over and over again.

Maybe make that clear/clearer in the ReadMe?

Have a good day!

[krobing]

I've found the way for update the reference to the file that already was hashed. the code is this and it must to put it in /tasks/cachebust.js file, replacing something of the function "replaceInFile".

The code is the follows:

function escapeStr(s) {
   return String(s).replace(/[\\^$*+?.()|[\]{}]/g, '\\$&');
} 

function replaceInFile(filepath) {
            var markup = grunt.file.read(filepath);

            _.each(assetMap, function(hashed, original) {

                // get file extension and the before file path
                var extOriginal = original.split('.').slice(-1),
                    trunkOriginal = original.split('.').slice(0, -1).join('.');

                var pattOrigFile = new RegExp(escapeStr(original) +"(\\?[a-fA-F0-9]{"+ opts.length +"})?|("+ escapeStr(trunkOriginal+opts.separator) +"[a-fA-F0-9]{"+ opts.length +"}\\."+ extOriginal +")", "gm");

                markup = markup.replace(pattOrigFile, hashed);
            });

            grunt.file.write(filepath, markup);
        }

This resolves the issue #183.

[shikharseth]

@krobing great work..i think this should be considered