[TW-125] A more usable timesheet

[taskwarrior]

Hessian Nerd on 2013-07-29T16:37:09Z says:

Move timesheet notations from annotations into a separate space. Allow reporting of time worked in a given week to include not completed tasks, better yet allow custom timesheet reports including totaling and averaging by project and or tag. Allow custom date range reports.

[taskwarrior]

Migrated metadata:

Created: 2013-07-29T16:37:09Z
Modified: 2016-02-11T14:24:28Z

[taskwarrior]

Paul Beckingham on 2013-09-29T17:05:35Z says:

Thank you for the idea. I think the best approach to this would be a hook script that, on task modification, writes out sufficient details to a data file that is especially suited for timesheet reporting. Then an external script can use that data file to generate ideal timesheet reports.

Reporting is a tricky subject, because the formatting is so specific and personal. Everyone will want something different. That makes it an ideal external script, which can be used as a reference report, for further customization.

[taskwarrior]

David Patrick on 2013-09-29T17:52:30Z says:

The timelog format I would recommend would be http://hledger.org/MANUAL.html#timelog-files, which would then allow for things like invoicing, accounting, reporting, etc.

[taskwarrior]

Paul Beckingham on 2013-09-29T17:57:58Z says:

That's app-specific, so it would have to be a "no". Unless of course there were no standard formats out there, which seems unlikely.

[taskwarrior]

David Patrick on 2013-09-29T19:08:14Z says:

The Timelog format that hledger and ledger-cli use is a subset of emacs/orgmode timeclock.el format, so they are "app-specific" only in as much as these three apps share the format. There may indeed be a world-wide timeclock text-file format standard, but the Timelog format linked above looks like this;

i 2009/03/27 09:00:00 projects:a
o 2009/03/27 17:00:34
i 2009/03/31 22:21:45 personal:reading:online
o 2009/04/01 02:00:34
i 2009/04/02 09:00:00 projects:b
o 2009/04/02 17:00:34

which couldn't get much simpler, and seems perfectly congruent with our start-stop times and projects.

ledger (hledger, ledger-cli and add-ons) are the only really credible candidates for accounting companion-apps for taskwarrior. Whether tracking and billing for time, or any other accounting functions, if you want to work comprehensively from the console, and have confidence in the results, ledger is the only game in town. If the above text-file format is as good as any other, it comes with immediate compatibility, so it's still worth considering.

[taskwarrior]

Hessian Nerd on 2013-10-02T22:29:26Z says:

Paul Beckingham wrote:

Thank you for the idea. I think the best approach to this would be a hook script that, on task modification, writes out sufficient details to a data file that is especially suited for timesheet reporting. Then an external script can use that data file to generate ideal timesheet reports.

Reporting is a tricky subject, because the formatting is so specific and personal. Everyone will want something different. That makes it an ideal external script, which can be used as a reference report, for further customization.

Thanks for the reply. I agree reporting is very individualized which is why I suggested this. Currently there is a narrow use case of how to interact with active time, I had hoped to expand it to a general case. If you are arguing to remove start stop & timesheet entirely from taskwarrior I don't know how to respond other than to say time reporting is pretty useful and closely linked to task completion. With the assumption that we will keep them, I think the following will help improve them.

Right now the only function that seems to work with data from start and stop logging is timesheet and it seems severely stunted (only works on completed tasks etc). Furthermore when it does this logging it pollutes the annotations with "Started task" and "Stopped task", making annotations unusable for anything else (any report that shows annotations is useless when you start and stop tasks regularly).

The only way an external script could solve this problem is if it was running locally and the annotations were turned off and it looked for the active tag. It seems to me that this breaks down when you rely on file syncing (something pretty handy). I havn't looked at the task server model but it seems like this would be an issue here as well.

Maybe the solution is a more powerful annotation filtering and search system? Just being able to search for "Started task" and "Stopped task" will go a long way, as will being able to filter these very specialized annotations out of the remaining notes. I think such a filtering and searching system (returning task uuid, annotations that meet filter criteria) would be a very powerful tool that could be used for many other things that I cant think of right now.

Sorry to ramble, thanks for your time.

[taskwarrior]

Frédéric Meynadier on 2015-01-28T11:37:19Z says:

Hello,

I hope you won't mind if I re-ignite this comment thread... Unless I missed it, I didn't see the following point discussed in more recent threads.

I think the previous comment makes an important point : storing start/stop times in the annotations makes them almost useless for anything else when tasks are started/stopped often (granted, this may be a symptom that these tasks are defined too broadly)... It has been a very good idea to do this in the first place, because it allowed to validate the principle while minimising the amount of code needed to implement it, but now may be time to move to the next step...

To make things clear : I agree with the general principle that time tracking should be handled by external scripts / hooks, but then Taskwarrior has to store the necessary information in a convenient way (that is, start and stop timestamps series for each task : everything else can be calculated from that).

I would therefore advocate storing start/stop timestamps in a different, dedicated field (e.g. "Timetracking:" ?), keeping the same format as the current one. This would leave the "Annotation:" field for legitimate annotations, and also allow external scripts to make some assertions on the content (e.g. basically "Timetracking:" fields can only contain a succession of "Started task" / "Stopped task" sequences, or there is something wrong and a warning should be issued when the script parses it).

My use case : I use an external script (derived from https://github.com/svenhertle/tasktime) to generate reports, that gives the proportion of time spent on each of my projects over a certain period of time (which means I can't just take the total active time of each task : I must calculate how much of this active time falls within the given time period). So, as far as I am concerned, Taskwarrior already has all the time-tracking information I need for my reports (many, many thanks to the authors for that !), but I can't use the annotations properly because of the crowding by time-tracking data, which is a bit frustrating.

Should the proposed change be implemented, it would be fairly easy to adapt existing scripts (parse "Timetracking" entries instead of "Annotation" entries, or even parse both for backwards compatibility), and also easy to convert existing databases to the new format (move all annotations containing "Started task" and "Stopped task" to "Timetracking" entries). Failing to do the conversion in the DB would not break anything, except active time accounting by external scripts would not take into account the new "Timetracking" entries until they are themselves upgraded.

[taskwarrior]

Tomas Babej on 2015-01-28T12:11:22Z says:

I guess you can write a on-modify hook to keep your annotations clean and at the same time move the timetracking data to your own UDA for now.

[taskwarrior]

Frédéric Meynadier on 2015-03-12T16:17:38Z says:

I am trying to follow this path but I don't understand how to mimic the annotations in an UDA : what type should I assign to a "timetracking" UDA to accept lists of {'description': foo, 'entry': bar} just like "annotations" does ?

Of course another solution would be to dump all the time-tracking data into one string and parse it later, but ideally I would like to just move the annotations to another multi-line UDA if possible : the on-modify script would then be completely straight forward, and adaptation of existing scripts would be very easy too.

Could you please give me some guidance on this ?

[taskwarrior]

Paul Beckingham on 2015-03-12T16:20:59Z says:

Your fears are true - there is no UDA that can store multiple values. You would have to encode/decode like the tags field.

[taskwarrior]

Frédéric Meynadier on 2015-03-13T10:58:02Z says:

Thanks for your answer...

I begin to think it would be easier (for my case) to put my "real" annotations in an UDA and keep the annotations for timetracking only... !

I fully understand (and respect) the decision of keeping timetracking issues separated from the core of Taskwarrior. But the use cases are so close that it is very, very tempting to find a practical way to deal with that...

If I understand well your comment [here| https://bug.tasktools.org/browse/TW-1562?focusedCommentId=15521&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15521] you consider that the timestamps packed with "Started task/Stopped tasks" annotations are just that - timestamps of when the command has been issued, so in a sense they just "accidentally" convey the information needed for time tracking, and tampering with them a posteriori is a kind of history-rewriting hack... Fair enough !

In order to replace that, would it be complicated to add "on-start" and "on-stop" hooks to the hooks API ? That would allow to externalize completely the problem (data storage included...), while keeping Taskwarrior's usage of start/stop commands unchanged.

[taskwarrior]

Paul Beckingham on 2015-03-13T11:04:55Z says:

You are exactly right - start/stop records time, and accidentally (but faithfully) record time spent in the active state.

While we have no on-start/on-stop hooks, we do have on-modify, and it is that hook that is used to track time here: https://github.com/kostajh/taskwarrior-time-tracking-hook https://gist.github.com/wbsch/d977b0ac29aa1dfa4437

[taskwarrior]

Frédéric Meynadier on 2015-03-16T12:30:54Z says:

Hi again,

Sorry, but I can't find how to write something in an UDA :

I have written a hook much to the tune of https://github.com/kostajh/taskwarrior-time-tracking-hook, but storing the timestamps like this (for example):

20150316T110100Z-20150316T120106Z;

I have declared a "string" type UDA in my .taskrc :

{code} uda.timetracking.type=string uda.timetracking.label=TimeTracking {code} (I also tried adding uda.timetracking.values='' as suggested in https://github.com/kostajh/taskwarrior-time-tracking-hook/pull/12/files)

and input/output seems correct to me according to debug output :

{code} Hook: Calling /home/fmeynadier/.task/hooks/on-modify.timetracking Hook: input {"description":"Write hook script for timetracking","entry":"20150312T154057Z","modified":"20150312T154057Z","project":"tw","status":"pending","uuid":"f7acb75b-f3c9-453f-9034-49f1f257753c"} {"description":"Write hook script for timetracking","entry":"20150312T154057Z","modified":"20150312T154057Z","project":"tw","start":"20150316T120106Z","status":"pending","uuid":"f7acb75b-f3c9-453f-9034-49f1f257753c"} Timer Hooks::execute (/home/fmeynadier/.task/hooks/on-modify.timetracking) 0.041314 sec Hook: output {"timetracking": "20150316T120106Z-", "project": "tw", "description": "Write hook script for timetracking", "start": "20150316T120106Z", "status": "pending", "uuid": "f7acb75b-f3c9-453f-9034-49f1f257753c", "entry": "20150312T154057Z", "modified": "20150312T154057Z"} Hook: Completed with status 0 {code}

but nothing stays in the "timetracking" UDA : if I use the "edit" command, I see

{code}

1. User Defined Attributes UDA timetracking:
   {code}

If my hook output is not considered "valid", I would be expecting an error message (at least in the debugging output). If it is valid, then I fail to see what additional steps are needed.

Below is a minimal version of my hook :

{code}

1. !/usr/bin/env python import sys import json

old = json.loads(sys.stdin.readline()) new = json.loads(sys.stdin.readline())

if 'timetracking' not in new: new['timetracking'] = ";"

if (not 'start' in old and 'start' in new): new['timetracking'] += new['start'] + "-" if ('start' in old and not 'start' in new): new['timetracking'] += new['modified'] + ";" print(json.dumps(new)) sys.exit(return_code) {code}

Surely I am missing something very simple, but I can't find what...

(Besides, I first tried to put "rc.debug.hooks=2" in my .taskrc file but it seemed to be ignored, I had to write it directly in the command line and it worked, but I got a "Configuration override rc.debug.hooks:2 " message. Also strange... Other options are read correctly. All this has been tried with the latest master branch).

[taskwarrior]

Paul Beckingham on 2015-03-16T14:05:48Z says:

Hi!

First problem - values written via on-modify hooks are not stored. That is the big bug we addressed by releasing 2.4.2 yesterday. You'll need to upgrade to make your script work.

Second problem - you put "debug.hooks=2" in the .taskrc. The "rc." prefix is for the command line, to give the parser something it can easily spot in the first pass.

[taskwarrior]

Frédéric Meynadier on 2015-03-16T14:18:51Z says:

Great, it works ! Thanks...

[taskwarrior]

Frédéric Meynadier on 2016-02-11T14:24:28Z says:

Hello,

It took me nearly one year to reach this conclusion, but finally I switched to "hamster" for the time-tracking part. I wrote a small hook that interfaces Taskwarrior with it, it can be found here :

https://github.com/fmeynadier/taskwarrior-hamster-hook

Basically it calls "hamster-cli start" with relevant arguments when I do "task start", and "hamster-cli stop" when I do "task stop". Hardly rocket science, but I did not find it in the current list of user-contributed hooks, so I thought it may be worth sharing.

Previously I had been trying to develop what was discussed in this thread :

• a hook that stores timetracking data in a separate UDA
• a modified version of the user-contributed "tasktime" script that takes this UDA into account (+ some features, like looping over projects)

My attempts may be found here :

https://github.com/fmeynadier/yatwtt https://github.com/fmeynadier/tasktime (forked from https://github.com/svenhertle/tasktime)

... so people interested in the outcome may still get some pieces of code there : the UDA part works, as well as the modified "tasktime", but I never finished the script that would have allowed to manipulate the data correctly (e.g. add some time, tweak dates, etc...), whereas hamster already has most of those features.