Smart item search by abbreviation

[fsufitch]

Eve has a variety of abbreviations and acronyms for certain module types, and being able to search based on these would make the fit planning flow much smoother.

Examples:

• Heavy Assault Missile Launchers are kind of a pain to search for. You need a query like hea ass miss, when it would be far more natural and intuitive to use players' actual abbreviations for it: HAM and HAML.
• Ballistic Control Systems suffer from the same issue. Using BCS rather than bal con to find them would be fantastic.
• Energized Adaptive Nano Membranes are difficult to search for because each one of their four words is shared with other very different modules. EANM would be much simpler to use.
• Not all cases are strict acronyms based on the first letters of words. Damage Control modules are usually called DCUs by players.

Additionally, searching by these abbreviations should bring up all related items. For example, XLSB should bring up not only things that abbreviate to it, such as "X-Large Shield Booster II", but also things with obscure names, such as "X-Large C5-L Emergency Shield Overload I".

Taking a brief look at the code, this can probably be accomplished with a custom list of abbreviations manually managed in source code, rather than a complex query against the Eve DB. The results of abbreviation lookup can then be joined with the results of a regular item search.

[fsufitch]

If this is OK to implement but low priority, I could actually try to put my Python skills to the test and get a PR out there.

[blitzmann]

Hey there @fsufitch! Thanks for feature request!

This is a feature that has been brought up before, as outlined in #750. Your way would indeed be the most straight forward, however I'm never a big fan of hardcoding things, and maintaining a mapping such as that could be a nightmare. I much prefer the third option that I laid out in #750: taking the acronym (which we can define as a search term of 5 or less characters, optionally require them all to be capitalized), split them, and create a filter such as LIKE 'X%L%S%O%' or however it's done in SQLAlchemy (to use your example).

While you're correct in that this does not cover cases such as DCU, and things like MWD are a bit funky (would have to put in something like 5MWD because it doesn't start with a M), I think would cover the vast majority of the cases and is a more universal solution to the problem without it being overly complex (and maybe we can have a hybrid in place that uses a custom mapping for things like DCU = "damage control"). Please let me know what you think about it. And yes, I am absolutely open to PRs.

Please also note that the search has a minimum of 3 characters - this is unlikely to change without a really good reason (ei, lots of 2-character acronyms that can be determined).

As an aside, would you know of a resource that has a list of popular acronyms to use as a base of information? That would be handy to start out with to see what can be accomplished with something like this :)

[ghost]

@blitzmann https://wiki.eveuniversity.org/EVE_Lexicon seems to be regularly updated with acronyms, would just need to extract the ones related to modules.

[blitzmann]

Thanks @burnsypet! I should have known eve uni has something like this lol

[fsufitch]

It's still a brief enough list that manual maintenance is feasible, especially in an open source project. I am not referring to "hardcoding" as in literally baking it into the source, but instead including a abbreviations.yml file or something, with a syntax like:

- keywords: 
  - dc
  - dcu
  expansions: 
  - damage control

- keywords:
  - mwd
  expansions:
  - 5mn
  - 50mn
  - 500mn
  - 5000mn

- keywords:
  - ham
  expansions:
  - heavy assault missile

This approach would allow for very easy maintenance even by folks without coding background. Version 1 of the file could even be inspired by the E-UNI wiki and include a link to it as a comment. The code itself could transparently replace each found keyword with its expansions and return the union of the results. Examples:

• ham ii would be equivalent to searching for heavy assault missile ii and correctly return Heavy Assault Missile Launcher II and Polarized Heavy Assault Missile Launcher II.
• compact mwd would search for compact 5mn, compact 50mn, compact 500mn, compact 5000mn, and return the combined results of those searches, which would be all "compact" MWDs.

Better, this opens the possibility to introduce more "smart" searching by keyword in the future. The same logic can be used to search cpu to get Co-Processors and Overclock rigs, for example.

I think I might actually try my hand at a mini version of this idea and see how you like it.

[fsufitch]

@blitzmann to respond to the solution proposed in #750 , that would work too, but I can see a couple problems:

1) As you pointed out, some names would be hard to match that way. "Microwarpdrive" is one word, are "afterburner", "autocannon", "multifrequency" and many others are as well.

2) That sort of wildcard text search feels like it would be wildly inefficient and cause significant lag, due to its combinatorial complexity. Maybe my DB instincts are off, but I wouldn't want this sort of feature at the price of each search lagging Pyfa out by 1-2 seconds on slower computers.

[blitzmann]

You make some good points. Okay, if you're interested in building this feature out, go for it! :)

The one thing I would be weary of is multiple "searches" per keywords. The search itself (currently) doesn't support OR (so we can't do "5mn" OR "50mn", etc). The tokens are all AND'd together. Of course we can make it support that if needed, just wanted to throw that out there just so that it's known (searchItems() also takes a where parameter that might be useful, it's been a while since I touched on any of these though so I'm not sure how it would be implemented.

That sort of wildcard text search feels like it would be wildly inefficient and cause significant lag, due to its combinatorial complexity.

I don't believe so. There's only 17k records in the types table, and mine returns fairly quickly. Of course, this is SQLIte and not SQL Server, and I'm running off an m.2 drive and i7-8700K, so who knows ¯_(ツ)_/¯

---

edit: Additionally, it would be neat to be able to edit this configuration through the GUI somewhere, possibly from the preferences so that people can easily setup their own without having to edit the file directly. That can be on the backburner for now though :)

[fsufitch]

Here's a first pass at doing this: https://github.com/pyfa-org/Pyfa/pull/1513

Lets you do neat stuff like this:

Or like this:

Or this:

It is configured using the jargon.yaml file, which has some sane defaults as part of the code itself, but gets written locally in the user's save directory. This allows for manual customization, and possibly a future UI customization system.

[blitzmann]

👍 just approved the PR (sorry for the wait!). This is pretty cool, and I agree we can iterate further and introduce a UI that allows for easy customization :D