Item Metadata

[stranger-danger-zamu]

I found this the other day on lib.rs and I've been using for a bit now. It's great and the alias trick in the readme really makes the ergonomics sing. So thanks for uploading it to cargo and open-sourcing it. You probably will have a few other issues I'll open up to keep things clean.

---

I looked over how sigi works internally and noticed some really nice things are left on the table. You track creation, "success", and "failure" time, but none of that's is exposed via the current interface.

You could just have these values be printed in parenthesis after the item (eg, Now: foobar ($start_time, $elapsed_time, $end_time)). Or you could also go for a detailed view which allows the user to index to any item in a stack and see those values (ie. just print the entire item).

[booniepepper]

Thanks for the interest and for opening issues!

Yes, the statuses are definitely a thing I've been meaning to improve.

Sorry for not responding much earlier. Two things have slowed me down a bit here:

1. I have some competing ideas in this project. It's partially a CLI that I use (and share) for organizing. It's also partially a stack-based database implementation where (roughly) a stack name is a key, and an "item" is an arbitrary text document.

   • The "always read and write a JSON document" was intentionally naive so I could build out something that works and think about a really good solution over time. It works well for small usecases, but for a stack of millions of items it would be incredibly slow and memory/IO intensive. I've been writing a few proof-of-concept databases with more robust filesystem-based implementations (instead of JSON documents) over on branches of this project: https://github.com/hiljusti/kamajii

     Anyway, at some point this may be able to take over the backend of Sigi, or if not, at least let me focus Sigi down to being just an organization tool.

2. Life things - halfway through a move, sleep training daughter, presenting at some conferences, etc.

   • This should calm down for me in a few weeks.

[booniepepper]

As far as getting to a better status, it's already possible to get some of the details by using --verbose

e.g.

$ sigi --verbose -qt $STACK list
Now: Do a thing 3 | Created: 2021-06-20 21:58:55.659902507 -07:00 | Completed: N/A | Deleted: N/A
  1: Do a thing 2 | Created: 2021-05-30 23:20:47.058666259 -07:00 | Completed: N/A | Deleted: N/A
  2: Do a thing 1 | Created: 2021-05-28 23:20:34.700290551 -07:00 | Completed: N/A | Deleted: N/A

I'm very open to suggestions. I like the start/elapsed/end idea you have there.

I think Sigi probably also needs:

1. A more human-friendly time format (possibly customizable? Best if it defaults to something locale-dependent)
2. A more computer-friendly time format (epoch?)
3. Some global or stack configuration (file? command?) where you can choose a default logging level (quiet, normal, verbose)

[stranger-danger-zamu]

Thanks for the interest and for opening issues!

No problem! I've been also working on a similar, but different tool for ad hoc entity relation graphs (think a namespace'd tag system for entities and their relationships, but with a CLI and without setting up a proper database each time). So, when I found that Sigi solved a complimentary problem and had a much better UI/UX than the hot mess I was doing, I had to share some observations.

1. I have some competing ideas in this project. [...] , at least let me focus Sigi down to being just an organization tool.

Ah, very interesting. Does Sigi support pluggable backends at the moment? It'd be pretty simple to define the interface for a persistence layer/data repository/database and allow for having the simple JSON backend remain while not blocking different implementations.

Regarding backends, have you played around with using Redis as a backend? I know that Redis can be configured to have strong persistence guarantees that are at least as strong as a relational database system. On the other hand, Redis does provide easy SSL for secure remote server capabilities.

That said, if a Redis backend never happens it's totally cool since your focus is on the stack-based DB aspect of it.

2. Life things [...]

Don't even mention it. Life is life and this is open source, there is no expectation with respect to turn around time (at least for me).

--verbose

Ah, I think I saw verbose and just assumed that it was related to the CLI and its operation and not the data.

item representation

You could have the output representation be pluggable or templateable. I can imagine having several output representations:

• a verbose human output (similar to or exactly the verbose output right now)
• short human output (current non-verbose output)
• JSON output (for piping into tools)
• CSV/TSV output (for piping into other tools)

These representations would give the user a broad useful coverage. You can further expand the representations by doing something similar to gallery-dl where the configuration file supports a string or a list of strings which represent attributes from the item (ie. provide users the list of valid attributes that an item exposes and read the config to print what they want).

elapsed time

This one is a bit dangerous since a thing can be within the stack for a while before floating to the top and then it might not be worked on. So it's pretty simple to have something balloon in elapsed time pretty quickly.

• One option is have elapsed time be a number of seconds as the head of the stack. So Sigi would track when an item become the head of some stack and then save the elapsed time when it exits being the head of that stack. This would probably be the most accurate tracking, but with some added complexity.
• You can also just compute elapsed time as the difference between creation and completion/deletion/finished time, but then the numbers might be huge.
• Another option is utilizing an activity log which allows for querying an individual item's events and calculating elapsed time between any two events becomes pretty trivial. Determining which events should be stored does become a larger factor though (eg. only transitions or transitions and something becoming the head).

Configuration

Please use the XDG Base Directory Specification! Having consistent paths and locations is immensely helpful.

You could add a system from setting the configuration via the CLI (eg. sigi config set attr1=value1 attr2=value2), but for the most part people are fine with just editing a file or sticking to defaults.

[stranger-danger-zamu]

😅 Oh boy that's a wall of text.

[booniepepper]

Well, this was instructive!

Ah, I think I saw verbose and just assumed that it was related to the CLI and its operation and not the data.

That's really good feedback.

I think the takeaway for me is I can better document (or make more obvious) the difference between what is now quiet/normal/verbose modes.

Pluggable backends

Yeah I agree. The code right now is highly coupled to the (kinda lackluster) json-files implementation, but I was wanting to make some kind of sane interface (Trait?) for multiple backends. At the very least, I don't want to break anyone using this now, so any idea of a remote/shared/distributed something-or-other should be an addition, not a substitution.

Side note: I know some people who really love SQLite and other similar familiar backends

Redis

Hmm I had been looking at both memcached and redis, but I guess I didn't dig into redis in parrticular nearly enough

https://redis.io/topics/data-types-intro#lists https://redis.io/commands/lpush https://redis.io/commands/lpop https://redis.io/commands/rpush https://redis.io/commands/rpop

Redis has O(1) for single item push/pop regardless of the "side" used... so this can already be a stack database.

XDG

Yes absolutely for mac/linux/bsd/etc. Probably the "appdata" something-or-other folder for Windows. Windows support isn't high on my list yet, but I think I haven't precluded it yet?

[booniepepper]

As far as getting to a better status, it's now possible (2.0.1) to get some details by using --verbose

e.g.

$ sigi --verbose -t $STACK list
Now: Do a thing 3 (Fri, 26 Nov 2021 04:26:09 -0800)
  1: Do a thing 2 (Fri, 26 Nov 2021 04:26:08 -0800)
  2: Do a thing 1 (Fri, 26 Nov 2021 04:26:07 -0800)

$ sigi --format csv -t $STACK list
position,item,created
0,Do a thing 3,Fri, 26 Nov 2021 04:26:09 -0800
1,Do a thing 2,Fri, 26 Nov 2021 04:26:08 -0800
2,Do a thing 1,Fri, 26 Nov 2021 04:26:07 -0800

$ sigi --format tsv -t $STACK list
position    item    created
0   Do a thing 3    Fri, 26 Nov 2021 04:26:09 -0800
1   Do a thing 2    Fri, 26 Nov 2021 04:26:08 -0800
2   Do a thing 1    Fri, 26 Nov 2021 04:26:07 -0800

So... A couple new formats introduced. JSON pending some time to focus on it.

I'm very open to suggestions on expanding this. I like the start/elapsed/end idea you have there. I moved the idea over to https://github.com/hiljusti/sigi/issues/15

I think to close this issue out, I probably should also do these things:

1. Have an option for time formats (maybe make a new issue for this)
2. Accept the --format argument and --verbose and --quiet flags whether they're at the beginning or end of the input

[booniepepper]

JSON added: https://github.com/hiljusti/sigi/commit/8d950e2479aa7bb3b98f0631d5768949708556d2

(Will be in the next version after 2.0.1)

[booniepepper]

The --verbose/--quiet/--silent flags and the --format FORMAT param can be before/after subcommands since: https://github.com/hiljusti/sigi/commit/035be30c3821761484dade1c77024927b1e5674b

(Will be in the next version after 2.0.1)

[booniepepper]

I think to close this issue out, I probably should also do these things:

• Have an option for time formats (maybe make a new issue for this)
• Accept the --format argument and --verbose and --quiet flags whether they're at the beginning or end of the input

Well, I did the latter, and decided to not do the former... I guess I'll call this done?