check age of files

[svx]

have a script which will check the age of a file, and give a warning if the modification date is older than a certain time this is handy for quality testing

script:

#!/bin/sh
git ls-tree -r --name-only HEAD | while read filename; do
  echo "$(git log -1 --format="%ai" -- $filename) $filename"
done

and then just do: sort -M -k 2 on the output which we redirected to a file

with this jenkins is able to give us a pretty warning aka: 'please check the file: develop/addons/components/genericsetup.rst is not modified for more than a $certain time' please check if the file is still valid.

[svx]

maybe include that for now into mr.publisher ?

[svx]

@gforcada would it make sense to run this with jenkins ?

I am not sure about that, the idea is to set a date, for example one year, and if the file is not updated for a year you will get a warning like 'please check $FILE', that does not mean the file is wrong, maybe there are no updates at all, but maybe you just forget to update it :)

I am just not sure if jenkins would be the right place for doing that, I could also just run it in cron once a week. :)

[gforcada]

@svx Jenkins is the perfect place for that!

Not only because it allows you to not have that cron job, but at the same time it gives visibility to it, you can send it by email to a public mailing list and so on...

I would say that instead of just 1 year I would just sort all files by age, no need to filter by a certain time...

Where can I find the script to run this on jenkins?

[svx]

@gforcada hm I have to look it up again, it should be more or less like what I wrote in the top post but I will check :)

[gforcada]

@svx I will give it a try during the week (hopefully), now the question is what steps need to be done before that git fu? I mean which repository needs to be cloned and after cloning what needs to be done before running?

So something like this would be enough (pseudo code ahead):

git clone github.com:plone/documentation
git-fu > timestamps.log

Or some sort of bootstrap needs to happen before that?

[svx]

Ok, don't run away, this is now really bad bash, but enough as an example :)

script: check.bash

#!/bin/bash
git ls-tree -r --name-only HEAD | while read filename; do
  echo "$(git log -1 --format="%ai" -- $filename) $filename"
done | sort -M -k 2

If we run this in a repo/branch we will get output like:

... 2015-05-06 20:48:17 +0200 develop/addons/components/genericsetup.rst 2015-05-11 12:52:55 +0200 develop/plone/misc/annotations.rst 2015-05-14 15:25:00 +0200 develop/addons/index.rst 2015-05-14 15:25:00 +0200 develop/addons/paste.rst ....

which is a good start !!!

Later on, when we have all docs and tests more up to date, I want to add a check, to give me a sign, if a file is older than a certain age, my intention with that is, that you get a hint to check, maybe the docs in this file are still valid and nothing changed at all or maybe it changed something but we forgot to update the docs.

That is why I had the idea, with for example the age to check is one year or so :)

[gforcada]

So I get that it needs to be run on documentation repo? Cloning is enough or some bootstraping (fetch external sources, etc) needs to be done?

Or is something to run on each docs folder for each plone repository? (i.e. CMFPlone, plone.app.testing, mockup, etc etc) ?

[svx]

No we do need anything else, we only need to do a checkout of the documentation repo and run it there.

Later if we want to run it on the branches we have to switch to them, but for the begin I say we start with master and first get it working and later we will do it for the branches :)

[gforcada]

There you go: http://jenkins.plone.org/view/All/job/docs-age/lastSuccessfulBuild/artifact/report.txt/*view*/

Should we filter to only search for rst files?

[svx]

Awesome ! Thanks !

As for only rst, I would say no, because we also keep images up to date, at least till developers start to include them into robot-framework. Hint :)

[gforcada]

Then do you want another job that counts how many static images do we have? So as to create a count down of images that need to be robotized?

[svx]

Would that be possible ? If so yes ! And I own you a lot of non-alcoholic cocktails !!

[gforcada]

Well, is possible as long as there is a way to know, given an image if it's static or from robot, how do you know that? :)

And for non-alcoholic cocktails, sure, bring them ;)