Filter: Hashtags

[xeor]

I'm the author of another project (https://github.com/xeor/taggo), which aims to solve a problem where that ends up needing many of the same ideas as this one. In the middle of a little rewrite now, but you get the idea by looking at it.

The basics idea we are both solving:

• Scan for files
• Collect data about file
• Only if a filter matches
• Do something

organize are:

• Using config-files
• Being very generic what files you support
• Having pluggable actions

taggo are:

• Able to define everything via cli
• Not being file-generic, but focusing on files/folders that are "tagged" (having the # in it)
• Creating symlinks, not using actions

Would it be meaningful to try to join these two projects under the same umbrella? Maybe create a github org and make something that solves both problems..?

Feel free to say no, there are still a lot of ideas we can share..

A joint efford could:

• Use configs or cli
• Have python functions to act as an api for other tools

What do you think? Feel free to contact me at: lars {dot} solberg at gmail

[tfeldmann]

Hi, really cool project you have there!

The django-style filter declarations are awesome and something I want to include into organize sometime. The exif support is really cool, too.

If I see this correctly, you could have the taggo functionality in organize with the following config:

# imaginary config. TaggoFile, Exif, MD5, FileSize, Symlink not yet implemented
rules:
  - folders: ~/Desktop
    filters:
      - TaggoFile
      - Exif:
          camera__contains: 'Nikon'
      - Extension
      - MD5
      - FileSize:
          bigger: 5 MB
    actions:
      # symlink into folders by tags and add md5 sum to filename
      - Symlink: '~/tags/{taggofile.tags.as_path}/{path.stem}.{md5}.{extension}'

Needed / enhancements:

• The TaggoFile filter that reads the tags
• An Exif filter, MD5 filter, FileSize filter and Symlink action
• for the cleanup something like a "EmptySymlink" filter would be needed?
• taggo's filtering syntax for the Exif filter is niiice

Am I missing something else in terms of functionality? What do you think of this?

To be honest I'm still skeptical about adding hashtags to your filenames - I know of some cases where windows has problems with this. This is a very opinionated way of organizing files - as a filter for organize that's fine, but I do not want to make this a central assumption in the tool.

[xeor]

Yea, a config like that would probably do it. I'm actually thinking about just using jmespath on the metadata I collect for now. Confusing to have both filter-types, at least in the beginning. Will probably add it back later tho :)

Not sure what you saw, but some things that are important, at least for my use of this is:

• Being able to support weird characters in filenames
• Use subfolders if tags is in the format "#people-name", I want it to be in a folder under ".../people/name/{file}".
• Support for parameters in tags, example "#tag(param)", but "#tag" still being the tagname
• Files and folders
• If there are more than 1 tag in the name, each tag must get it's own symlink..

I was sceptical at first as well, but I have had no problem having # in filenames, not on windows, linux or osx. I'm tagging a lot of files this way. Even for ignoring files in .gitignore.

After twitter and the "hashtag" term got here, it also feels natural to add them, not only for me, but for non-tech people as well. The only problems is in terminals, where you haveto put the filename in ticks, or escape the character.

What I ment wasnt it to be a "central assumption in the tool", but rather something to support. If this is worth looking into, I think it should:

• Be based on the organize base
• Add "#" idea from taggo
• Add a way to run using cli, without a config
• A way to do it all from python, so people can import this and use it as a base.
• Make a container for easy usage if you got docker..

I think it would be easier to start the project with the right assumptions tho. So if you are keen, we can create a new org on github and create this as a joint effort. Then, when it is ready, we can deprecate both projects and refer to the new one that supports it all 🥇

[tfeldmann]

I thought about this and I think it is possible to integrate this functionality into organize with only some small modifications to the core.

The only problem with the current core is the support for multiple hashtags in a name.

So my syntax proposal is:

rules:
  - folders: ~/Desktop
    filters:
      - Hashtags: 'people-john,stars>4,USA'
    actions:
      - Symlink: '~/tags/{item.as_path}/'
    with_items:
      - hashtags

• The Hashtags filter accepts a query string to filter by hashtags and hashtags with parameters
• The with_items syntax is inspired by Ansible standard loops and could be used for other filters as well.

I think this would be a really nice solution and I added it to the project's wishlist.

To your other points:

• organize is a very young project and I'm very happy with its structure. I'd much rather enhance this project than to create a new project from scratch. It does not need a new tool just to add a single feature. You can always submit a pull request if you want to contribute!
• Could you explain the use case for a command line interface? How many one-off rules do you really have? I certainly like the clean CLI and the ability to track (and backup) the config file with git.

[xeor]

Cool.. Glad to see you saw this as something useful. :)

Not thinking about creating from scratch, most of whats needed is already in place. And I think your project is ahead of mine when it comes to structure. So organize should be the template.

I didn't reach out just to add hashtag support, I reached out to ask if you wanted to create an github org where the project can live in, and we can work on a single codebase. Since we already share the basic idea.

Currently, the CLI is most useful for testing, and one-off commands renaming a tag. But it gives enormous flexibilities when using the tool. Maybe someone wants to use it in a shell-script. Maybe someone wants to create an alias that is a shortcut to symlinking everything from . to a special folder. Maybe someone want's it as an easy thing to do in a container..

I'm almost done with a rewrite to further separate the logic even more, making it much easier to use it as a library, so people can use it in native python-code.

Let's think for a week or so :) Maybe it doesn't make sense after all.. It would however be nice to have 1 tool to rule them all, that it might end up being..