Store Journals in folder (with multiple files)

[matthiasbeyer]

Hi,

I think it would be great if the program would use several files instead of just one file. One file per day would be feasable, I think!

[matthiasbeyer]

Maybe as additional comment: I really like the idea of jrnl, but my current 'diary' setup writes stuff to several files (one file per day), which is much better if you write a lot. I would use jrnl heavily, and not only write one line or so to it, but maybe 50 lines per day. Keeping this in one file makes it really long (1500 lines a month, 18k lines per year).

As proposal: My current setup writes a directory-structure as follows:

/
/pages/
/pages/2014/
/pages/2014/01/
/pages/2014/01/01.txt
/pages/2014/01/02.txt
/pages/2014/01/04.txt

I would like if jrnl would do something similar.

The /pages directory is nice because if jrnl grows you can store the configuration or something withing the 'root' directory.

[maebert]

Hm, I see the value in this. In the current setup, it would actually be fairly easy to implement that, however hard to implement it well. Ie, jrnl would still open all the files all the time; I'll have to think about the code organisation a bit more to implement this proposal. But I'll keep it in the backlog for 2.0, which should hopefully have a release candidate within a month.

[matthiasbeyer]

Nice!

For the first step I would be fine with opening all files on each run. But maybe others are not fine with it.

The changes from "we read all files on each run" to "we read the required files on each run" would not change the "API" (the file system operations and the cli), so this would be a improvement for later. The changes in the first place change the file system behaviour of the program, so this has to result in a major-version bump. The rest is just runtime-fixing.

I would implement it myself, but I'm not that familiar with python, and I don't want to mess up... thank you for taking this serious!

[grplyler]

I second this enhancement, but also agree it would be more than trivial to implement well. It was however like the second question I asked myself about jrnl. "Does it give you the option to store entires in multiple files as well as one?" (To give you a new users perspective.) Perhaps a caching mechanism of some sort might be appropriate for a 2.0.0 release, so it have to parse all the files every single time to look for tags and such.

[matthiasbeyer]

Will this be implemented? It's open for over half an year now... and it seems nothing has happened...

[notbalanced]

Hey, I've got something working on my fork of jrnl, in the "folder" branch. There are some bugs with deleting/editing entries, still, so not really ready for prime time, but if you'd like to try it out...feel free.

[flight16]

This issue is important to me, too. I just found jrnl and it looks like everything I wanted, except the directory/file structure. Is this still being considered by @maebert ? Is there anything I can do to help?

[matthiasbeyer]

@flight16 :+1:

[flight16]

@notbalanced I had a look at your change, and it appears you added the changes to the base Journal class. Just curious, what issues did you run into when editing and deleting entries? Were these implementation bugs or design issues?

I am not the author of jrnl, but I think we could cleanly solve this issue by enhancing cli.py and the config to support arbitrary journal formats. There is already DayOneJournal that is hard-coded. All that needs to be done is to create a mechanism to optionally tag a journal as a specific format, and then instantiate the associated class. So I see this issue as "Support custom journal formats".

@maebert Would you accept a pull request for the above enhancement?

[notbalanced]

@flight16 Hi, my implementation is actually a new Folder class derived from the Journal class (similarly to what is done with the DayOne class.) In the cli.py, I do instantiate the Folder class if the journal in the .jrnl_config is a folder that doesn't have ".dayone" in the folder name.

It works just fine for adding entries to the journal. But when you use the --edit flag, and then delete an entry, it doesn't get deleted. (You'd have to go into the txt file and delete it manually.) I run into this problem because I can have multiple entries in one file (unlike the dayone journal that only has 1 entry per file). I haven't had time to figure out a way to identify which entry gets deleted, so that it doesn't get written when you update the file.

There is also the issue that if you only have 1 entry in a file and delete that entry, that the code doesn't delete the file.

Finally, I haven't had the time to run the tests on the update. I'm pretty sure I've broken something there, so I need to update those tests and add additional tests to cover this new journal type.

[matthiasbeyer]

Please note that I'm still interested in this.

---

Maybe another note why I want this: My current journal has 179 entries, which result in ~4,7 K lines. The files are ~800 kbyte overall, but nevertheless having multiple files seems much nicer IMHO.

Also, I'm writing imag and I'm thinking about integrating jrnl as my diary module. But for this I must be able to identify one entry, and a date is not enough for me. A UUID would be best, a filepath is feasible IMHO.

If you have questions, don't hesitate to ask! :smile:

[wren]

@notbalanced @flight16 @matthiasbeyer Thanks for your interest in the project! I just wanted you to know that this is one of my favorite features, and I plan to prioritize it.

We're just getting the project back up running right now, though, and need some time to get everything organized. Thanks for waiting, and look out for this feature!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[matthiasbeyer]

Let me note that I have implemented a diary (and a "log", as a "oneline-diary") module in imag already and it is working and I am actively using it.

I'm leaving this open anyways because others might be interested in such a functionality/feature.

[matthiasbeyer]

Can this be closed?

[wren]

Not yet! We're automating the changelog, and need this to stay open for now until we get that done.

[paanvaannd]

Hey, folks! Sorry for the thread necromancy and thank you all for the work on this.

I just started using jrnl and want to enable this feature but I can't seem to find documentation on how to do this. Is there something I need to add to jrnl.yaml for this?

After enabling this feature, will jrnl retroactively split my monolithic file into different entries in the above subdirectory structure or will I have to manually port those over?

[micahellison]

Hi there @paanvaannd, thanks for posting about this. We're working on a docs change to address this problem (pr #1331, preview here), though your comment makes me realize we should have instructions on how to migrate a single file journal to a folder journal.

To start with your last question: no, jrnl won't retroactively work on your journal. You'll want to create a new journal, then move your entries to it.

To create a folder journal:

• First create the directory where your folder journal will be
• Then add a new journal to your jrnl.yaml with a journal key pointing to that path, like so:

  newfolderjournal:
  journal: C:\Users\micah\Documents\new-folder-journal

To move your old journal's entries to your folder journal: first, make sure you are running jrnl v2.8.3 or later (jrnl --version) because it has a bug fix regarding this. Then, run the following, replacing the names of your journals as appropriate:

jrnl oldjournal --format text | jrnl newfolderjournal --import