Auto categories need better documentation, edit dialog.

[christofmuc]

"I haven't been able to get the auto categories I've added to do anything (in 1.14.0).

I currently have a couple of hundred Init Program patches, from importing with my Minilogue XD adaptation. I thought I'd try separating them from the others, by auto tagging them, so I tried making a category for Init, then re-running auto categorisation, while filtering for "Init", or "Untagged", or both. Nothing happens.

I also tried making an auto category for "Prog" - same result.

I thought maybe programs called "Init Program" are excluded, and I also have a couple of dozen factory patches starting with "TPL...", which are as yet untagged, so I thought I'd try making a TPL category - that doesn't work either.

Am I just not understanding what to do?"

Originally posted by @Andy2No in https://github.com/christofmuc/KnobKraft-orm/issues/124#issuecomment-1005054510

[Mostelin]

I am also rather lost with categories. There is no point going into the details of all I have tried, because I have become very confused in the process.

I now see that the categories in automatic_categories.jsonc cannot be changed. Furthermore, the categories in mapping_categories.jsonc must match these, so they too cannot change, and thus there is no real way to set up new categories. This makes the 'edit auto categories' function seem quite cosmetic.

I have 5762 programs already categorised hierarchically and would like to be able to make use of at least two levels (synth model (n=2) and sound family (n=21)). My json file gives these for every program, along with one further level, instrument name (n>100), but it seems that the only way to go is to identify these with the fixed preset list. This is too much work on the face of it.

And yet I am sure that at some stage I was able to import categories that didn't fit the presets - namely the two synth models. As I said, quite confused.

[christofmuc]

@Mostelin Let me try to help, and I probably need to get the naming straight in the software.

There is a list of categories that you can use to classify/tag your sounds. This list is prepopulated when installing KnobKraft, but changeable. Old versions had the list in a json file, but newer versions (can't remember which) have the edit dialog.

The menu item is called "Edit auto categories" for a historical reason, but it brings up the edit categories dialog. There you can change the name of an existing category (aka tag), and add new ones to the end of the list.

All categories setup there are available for manual tagging or for import via the Patch Interchange Format files produced by another KnobKraft exporting a database.

In addition, we have two more mappings that can be partially edited:

1. The "auto categories" of the older software, which is a list of naming conventions that will look at the name of a patch and try to match a rule, and then automatically assign the category based on the name.
2. The "automatic mapping" categories json file, which is a mapping when importing sounds from a synth that supports categories/tags on its own. I think currently the only synth that has this fully implemented is the Access Virus family. Think of the access calling a pad "strings" and you want it to map into your category, this is where you would set this up. I don't think this is used during the PIF json import.

Coming back to your problem - I think you should be fine to turn the synth model into just two normal categories, and use the "all must match" feature when searching. The sound family is probably what my original idea for the tags was - remember the Or will create a button for each tag, so having too many of these might make things less easy to handle.

The instrument > 100 shouldn't go into the category IMHO, I would rather recommend some kind of naming convention for this.

When you import the json file it could be that the categories you are trying to import must already exist - so make sure to create them via the "edit (auto) categories" dialog in the database before trying the import. And look closely at the log output from the import, it will probably report what kind of import mappings are done.

I probably need to look at the code again myself over all of this, it has been a while!

[Mostelin]

@christofmuc Thanks for the response. Are you saying that providing I simply

1. set up my 23 (2 + 21) required categories using Menu>Categories>Edit auto-categories
2. run the PIF import
3. run Menu>Categories>Rerun auto-categorize...

then the categories will be correctly assigned (two per program)? If so, I will try this later today.

I am happy not to have the third category.

[christofmuc]

@Mostelin I believe you can skip #3. This is only required when the rules for naming conventions (like "BS:" -> assign bass) have changed, which with the current version is not exposed. Reason is I wanted to add this to the edit categories dialog, but then that was so buggy I concentrated on making it work for the simple case.

So yes, setup 23 categories, and when those are referenced by the PIF import, they should be automatically assigned during import. If that doesn't work, it's a bug and I'll fix it ;-)

[Mostelin]

@christofmuc I tried what you advise and here are my observations...

1. Adding new categories has no effect. Only the original 15 categories are applied on import. Any more seem to be ignored. Bug?
2. When a program fits two categories, it is coloured only according to the earlier one in the category list. Feature?
3. Existing categories may be renamed, but the logic of applying them based on program name analysis ignores this. Bug?

On point 3, I thought I could fix this by editing the file that specifies the regular expressions, and replacing or removing these terms, but that file seems to have disappeared, or else I forgot what it was called.

So, in summary, I think I am a little clearer about what I should be doing, but I am not sure it is possible to get the result I want.

[christofmuc]

@Mostelin Let me think:

1. Sounds like a bug. Can you give me a little piece of the json file you're trying to import. You're using version 1.15.x?
2. That is a feature, double assigns are somewhat random. With your "two tier" categories, we could think of giving the brighter colour precedence or something, so things wouldn't just be "ESQ-1" coloured, but the synth category could be a colour that is overridden by any other. I though about coloured stripes/multi colour patterns, but fear that this will lead to visual overload.
3. I have to look it up - I think the old json file with the regular expressions is ignored as the plan was to put this into the edit category dialog. I must look at it again and see what is the simplest solution to get this back in order.

[christofmuc]

@Mostelin Ah! I found a bug. Just to check what I tried back in December, I used the json file you attached here: https://github.com/christofmuc/KnobKraft-orm/issues/124#issuecomment-986743042

When I use Orm 1.15.0, select the ESQ-1, and then select the json file in the import from computer dialog.

I get 5 patches imported, and the log output: "11:17:30: Ignoring category Hybrid of patch SYMPY3 because it is not part of our standard categories!"

I then create the Hybrid category, and import the file again.

Same error.

I close the Orm, and start it again, this time the Hybrid category is already present at load time.

I import the json file and voila - categories are assigned properly.

So the bug is that a newly created category becomes available for import only after a restart of the software. Can you try?

[Mostelin]

@christofmuc

1. I am using 1.15.1. I've attached a sample of the json file I am using. In it you will see the three levels of category, looking something like this: "Categories": [ "ESQ-1", "Woodwind", "Clarinet" ], Ensoniq-programs-sample.zip]

2. I agree, one colour is enough. When things are working I shall deliberately order my categories so that the two top-level categories come last, giving the mid-level categories the opportunity to shine.

3. I did later find my copy of automatic_categories.jsonc on a removable SSD. I don't think the ORM would have found it, so I suspect that it is indeed being ignored now.

[Mostelin]

@christofmuc I just saw your latest message. Let me give that a try and report back to you.

[Mostelin]

@christofmuc A big step forward!

I created all 23 categories I need, retaining your presets if they exactly match my own, quit ORM and started again, noted all categories were present, deleted and reloaded all 5000+ programs and the result is that my categories now seem to be in effect wherever I look, and they are combined with the automatically assigned categories that I kept (e.g. a piano I examined has three categories now: ESQ-1, Piano, Keys). No programs are untagged. I think I am happy with this state of affairs.

[Mostelin]

@christofmuc BTW, I suspect that the automatic assignment of categories using regular expressions is still in effect. A couple of percussive sounds called STRPDX and RAPDRM, for example, are being classed as pads on the grounds that they have PD in the name. They are as unlike pads as can be. I wonder if there is some way to disable the automatic categorisation? I hardly need it, after all.

[christofmuc]

@Mostelin I created #167, we need to be able to edit the rules again and thus you could just delete all of them, effectively disabling name based categories.

[Mostelin]

@christofmuc I would do that, but I don't know where the file is stored. It is called automatic_categories.jsonc, isn't it?

[christofmuc]

@Mostelin Ok, I looked it up. Yes, the file is called automatic_categories.jsonc. In the "old days" the function to edit it would actually create it (the original rules are in the executable). That function I removed because I had no idea how to update those later, but it will still pull it in when the file ist there.

For me, I see that it is working in the log:

18:35:44: Overriding built-in automatic category rules with file C:\Users\Christof\AppData\Roaming\KnobKraft\automatic_categories.jsonc

There you can see the location of the file. To disable all rules, just cleanout the naming rules and just leave "{}" so the file still is valid json. That worked for me!

I will reinstall the function that creates this file in the first place, and try to fix some of the menu naming.

Overall, the naming rules and categories seem to need a bit of an overhaul.

[christofmuc]

There were quite a couple of bugs in the automatic categories in the 1.17.x versions and before, and I fixed a lot while doing the 2.0.0 release. I'll close this ticket as I think the content of this has been processed, and we can start new tickets should it become necessary.