`timew import` command

[sruffell]

I would like to actively track my time on two systems and have a way to sync them together. One way that would be relatively easy is if timewarrior had an import command that could parse the output of an export from another instance, and add any intervals that do not conflict with existing intervals or that are already being tracked in the destination database.

This way I could track my time on one computer, and then sync that time to my "master" timewarrior instance by running on a remote machine over ssh like:

timew export | ssh <server> -c 'timew import'

This is an idea that I first expressed in a comment on #340.

Does anyone have any thoughts about why this wouldn't be a good idea? It seems to me that a command symmetrical with the existing export is not a bad idea.

[lauft]

I think it is not a bad idea, but the merge could pose some hurdles. You already pointed out a policy how to deal with merge conflicts (to ignore them) but this is maybe not the only way Timewarrior should deal with it.

A first draft for the procedure:

1. Determine overlaps between import and existing database
2. Report to user

   X intervals can be imported without any conflicts. Proceed? (yes/no)

   Of X intervals in the import, there are Y conflicts with Z existing intervals:
     - @A [<from> - <to>] "TAG1", "TAG2"
         overlaps with
       @N ...
     - @B ...
         overlaps with
       @M ...
       @O ...
     - @C ...
       @D ...
         overlap with
       @P ...

3. Request user feedback on how to process overlapping intervals
   • keep: Imported intervals are truncated if there is an overlap
   • adjust: Existing intervals are truncated if there is an overlap
   • ignore: Conflicting intervals are ignored
   • abort: import is aborted (These strategies are the same as with the interval overlap resolution when adding intervals to the database)
4. Perform the import as instructed.

As a start, we could implement the all or nothing approach, where we only allow the import in case of no conflicts. If there are conflicts Timewarrior reports them and quits. (I.e. steps 1. and 2.)

In general I think the import command should not proceed if there is active time tracking, i.e. an open interval.

User feedback should also be possible via hints, e.g.

$ timew import <file> :yes :adjust
$ timew export | ssh <server> -c 'timew import :yes :abort'

One level up would be to offer something where the user can decide on each conflict on how to resolve it, but let's start slow... 😉

[sruffell]

One problem with an interactive interface on the import command is that it would complicate reading the imported data from stdin, which makes the ssh user case very useful.

I like your idea of having different strategies to handle any conflicts.

I have started a branch based on the sorted datafiles branch (https://github.com/sruffell/timewarrior/commits/wip-cmd-import) that implements the basic use case with validation, as that will be useful for me in the short term, I'll look at adding the different strategies as hints there.

[sruffell]

The import command might also be a good way to preload large databases for test without touching the datafiles themselves.

Creating ~20000 entries in the database takes ~12min running start in a loop. It takes less than a second to process the export via the import command:

$ rm -fr ~/.timewarrior/data; time ( for x in {20000..1}; do src/timew rc.journal.size=0 :quiet start ${x}min ago proj-${x}; done )

real    12m28.520s
user    11m55.149s
sys     0m36.136s
$ src/timew stop
Recorded "proj-1"
  Started 2020-08-12T03:09:42
  Ended                 10:48
  Total               0:01:06
$ src/timew export > export.txt
$ wc -l < export.txt
20002
$ rm -fr ~/.timewarrior/data; time src/timew import :quiet rc.journal.size=0 < export.txt

real    0m0.932s
user    0m0.932s
sys     0m0.000s
$ src/timew export | wc -l
20002

[lauft]

One problem with an interactive interface on the import command is that it would complicate reading the imported data from stdin, which makes the ssh user case very useful.

This is what the :yes hint was intended for.

The import command might also be a good way to preload large databases for test without touching the datafiles themselves.

It is always good to hear about positive side effects... 😉

I have started a branch based on the sorted datafiles branch (https://github.com/sruffell/timewarrior/commits/wip-cmd-import) that implements the basic use case with validation

I try to take a look at it as soon as possible.

[simmel]

Any progress here?

[sruffell]

Any progress here?

I've not thought about this for awhile (as I've mostly settled on using a dropbox / google drive to keep my timewarrior database in), but I can pick this up again if it would solve a real problem for you?

(although, given the reactions on the open issues....more flexible time parsing is probably what I should look at)

[sskras]

I've not thought about this for awhile (as I've mostly settled on using a dropbox / google drive to keep my timewarrior database in), but I can pick this up again if it would solve a real problem for you?

(although, given the reactions on the open issues....more flexible time parsing is probably what I should look at)

Yesterday it would have solved a problem for me, where:

• I have been using my custom track converter/impot-script (using hand-written lines from Notepad as an input).
• I have made a backup of TIMEWARRIORDB before that.
• The converter had finished job at some time.
• Then sudden things of life happened.
• I just switched to handle them instead of verifying the conversion mistakes / quality.
• Later I found one defect in my script which made me to fix and rerun it on the backed up data.
• Before this I diffed two JSON exports (from the backed up and the current databases).
• Here I have found that I tracked a 8 more time intervals since the scripted import.

I knew already that task already supports import command. And since both tools support export command, I wrongly assumed that timew would support import too, to my dismay.

So I just parsed that interesting part of JSON by my eyes and went to track my intervals by hand, manually.

It would have been very nice for TimeWarrior to be able to import that part automatically. Provided that it would assign new, different IDs for every track it import. At least in cases where there is no timing overlaps.

Thanks! : )

[simmel]

My usecase is mostly to merge databases from multiple computers. I did some digging and found people syncing the database with unison and diff3 so I'm looking into that right now.

On Thu, 2022-03-31 at 08:06:41 -0700, Shaun Ruffell wrote:

Any progress here?

I've not thought about this for awhile (as I've mostly settled on using a dropbox / google drive to keep my timewarrior database in), but I can pick this up again if it would solve a real problem for you?

-- Reply to this email directly or view it on GitHub: https://github.com/GothenburgBitFactory/timewarrior/issues/359#issuecomment-1084722689 You are receiving this because you commented.

Message ID: @.***>