search

ppy / osu-web

the browser-facing portion of osu!
https://osu.ppy.sh
GNU Affero General Public License v3.0
969 stars 380 forks source link

Increase the variety of languages and genres available #5852

Open Raidencio opened 4 years ago

Raidencio commented 4 years ago

in light of the recent update that allows mappers to set their own genres and language, it is widely known that the number of both of those are fairly limited, and some songs have to be classified into "other" or "novelty", while there could be much more accurate genres to describe them

Some of the new genres could be:

(will expand this list with community input)

Some of the new languages could be the ones available in the page (https://i.imgur.com/oyJGQpn.png)

this has been apparently suggested alrdy: https://github.com/ppy/osu-web/issues/5153 Russian: https://github.com/ppy/osu-web/issues/3376

notRealazy commented 4 years ago

adding more broad genres would be nice, but i don't think we should try to delve into more precise ones, mostly looking at electronic genres. if we add dnb, dubstep, breakcore, people might wonder why we don't add drumstep, riddim, mashcore, or other genres like techno/house/etc.

more genres = good, too many precise genres = bad

Raidencio commented 4 years ago

makes sense, what broad genre should we add that encompasses multiple electronic subgenres?

currently "Electronic" is a little ?? because you can pretty much stuff anything in there that is not recorded in a studio

frukoyurdakul commented 4 years ago

At its current state I think the suggestions only contain "Vocaloid, Metal, Classical / Orchestral, Breakcore" as the rest can fall into more general categories, and not that common for now.

NoffyNoffeh commented 4 years ago

I think Metal, Classical, Folk would be good additions for musical genres a lot of folk and classical stuff currently get categorized weirdly due to lacking categories, and metal gets thrown into rock for lack of better tag. Some of electronic's weirdness is a result of lacking tags for piano pieces as well.

I agree it shouldn't be too specific unless the genre system was revamped entirely - allowing the user base to add the genre tags, easily accessible genre trees, allowing more than 1 genre to be assigned to a map, etc. Not really sure how beneficial that would actually be.

Vocaloid would be kind of weird since it encompasses all genres, and Vocaloid itself is presently dying down in favor of other singing synthesizer programs (CeVIO, piapro, synthesizer V, NEUTRINO, etc). We'd be better off recategorizing existing Vocaloid tracks to other genres appropriately.

frukoyurdakul commented 4 years ago

I'm not sure about how common "Classical" and "Folk" are but almost everybody seems to be agreeing that "Metal" should be separated, from both discussions internally and from this issue alone. If the thing about "Vocaloid" is also correct (I've heard that @dorothy3242 listens to vocaloid a lot) then it can be removed from the additions.

TheDefaultGuy commented 4 years ago

Metal, Classical and Folk seem like the best additions, all the others seem too specific or just a very minor sub-genre.

Ephemeralis commented 4 years ago

We've been intending to flesh out genre lists for a while, but have never conclusively sat down and hashed out what genres are worth an individual classification.

As Realazy mentioned, a list that is too specific is just as useless as one that isn't specific enough. Electronic is a good example of this given it covers dozens, if not hundreds of classifications of music.

We should probably sit down in #modding and have a big brainstorming session sometime and hash this all out.

peppy commented 4 years ago

Assigning this to @Ephemeralis for now.

yaspo commented 4 years ago

Heyo, we held recently held a meeting about this topic in osu dev server's modding channel, refer to the pins to find the discussion, here is the meeting document

From there we concluded a set of ideas that we think would work well and would like to push forwards.

Baseline Changes we consider necessary or highly useful in the current state of Genre and Language settings.

Improving current Genres and Languages

To cover categorizations that refer to the type of song or to the source material, we'd like to add an Extra Category field alongside Genre and Language.

A filter list for these items should be displayed on the beatmap search, just like for genres and languages. When unspecified, this field shouldn't be displayed on a beatmap's page to regular users, but should be displayed to anyone with the permissions to set it.

To match these changes, some automatic refactoring will be useful to save lots of manual work.

These changes should affect Beatmap Nominators in the following ways:

This way we can ensure that Genre and Language is always set before nomination. The same rules should affect the Nomination Assessment Team as addition their existing permissions.

Additional Changes These are changes we consider very useful, but either depend on the baseline or don't take priority over it.

Extra These are ideas that have been suggested but weren't fully considered.

nl-tatatat commented 4 years ago

so.. all my video game maps no longer have a genre? Or they’re all set to Electronic? Electronic is WAY too broad. It can cover anything from Kobaryo speedcore, to r3 music box, to video game music.

Very disappointing. Video Game music is a genre. There are certain things that are near universal to all video game songs, especially BGM. Nearly all video game background music is meant to loop infinitely, and so the music is designed with this in mind.

Shmiklak commented 4 years ago

You suggest to prevent nominating if language is set to "Other", thus I wonder if song would be in some unavailable language, would it mean it's unnominatable?

venix12 commented 4 years ago
  • Prevent Beatmap Nomination when both
    • Genre is set to Unspecified
    • Language is set to Other
peppy commented 4 years ago

What kind of name is "Extra Category"? Is this based on any existing service? I would hope there was some existing premise..

ikin5050 commented 4 years ago

In my opinion electronic is a horrible way to classify a songs genre as electronic refers to the way it is produced, as opposed to the musical techniques and conventions a song adheres to. More suitable would be to add more options like electronic hardcore/EDM/electro pop/house/trap/breakcore

venix12 commented 4 years ago

@peppy Extra Category thing is meant to categorize songs according to source or something I believe, like in fact Anime or Video Game isn't a song genre. That would require adding another row along with genre and language, wouldn't be so hard to implement I think.

peppy commented 4 years ago

Yes I understand the concept. I'm looking for premise. I don't believe we are making something new here so we shouldn't be making something new here.

cl8n commented 4 years ago

I was under the impression "extra category" was a placeholder name because in the discussion we weren't sure exactly what it would contain until the end

venix12 commented 4 years ago

the entire extra category thing is rather a proposal for now I believe, would make more sense to have it as a separate issue imo

yaspo commented 4 years ago

It's mainly like that because it also includes things that don't refer to the source (eg. Remix, Cover, Nightcore). We did have a hard time coming up with a better alternative, other suggestions were "Song Category" or "Suggested Search Terms" A better term could be helpful yeah

peppy commented 4 years ago

Great, so now you have a "cover" of a "vocaloid" song from a "video game". How does this fit?

TheDefaultGuy commented 4 years ago

Electronic, at l

so.. all my video game maps no longer have a genre? Or they’re all set to Electronic? Electronic is WAY too broad. It can cover anything from Kobaryo speedcore, to r3 music box, to video game music.

Very disappointing. Video Game music is a genre. There are certain things that are near universal to all video game songs, especially BGM. Nearly all video game background music is meant to loop infinitely, and so the music is designed with this in mind.

Big agree. The way Electronic will be used now is not as a genre, but as the recording medium. Plus as a musician, not putting Video Game Music as its own genre is ridiculous. While Video Game Music can vary ALOT, it still has a WAAAYYY different vibe than music that was made to be listened to separately and not in a game medium. I understand that sometimes you have to use umbrella terms to describe types of music that don't fall into specific genres, but electronic is a way too big term. Also, having Video Game music be labeled as electronic is even more stupid, Is this electronic?, what about this?, and this?. It doesn't make any sense. However, thank you for finally adding Metal, that I am very happy with.

yaspo commented 4 years ago

@nl-tatatat @TheDefaultGuy can we not have this discussion if the intent is to repurpose Video Game to a different field, it's not gone or lost.

@peppy If you meant "how does this fit into the same group remotely", then uh, good point. The main idea here is to have these filters because they are things that appear commonly within the game and within players' interests.

A "Song Type" field could alternatively work by changing references to source to mean a typing instead? You'd rename Video Game to Game OST or similar, and maybe Anime could become like "Anime Opening" "Anime Ending" (or even "TV Release")?

peppy commented 4 years ago

May I suggest those are just left in tags, and we add common tag filters as shortcuts if that's what people want..?

yaspo commented 4 years ago

That's definitely an improvement, though having it as a visible field on the beatmap page is preferable. This gives players the option to find songs of a similar type through simply clicking on the respective link on the beatmap page. Works the same as genre and language in that sense.

Tagging is not ideal for a bunch of reasons. Tags have a variety of uses, causing clutter to anyone wanting to use them to search efficiently. They also get cut with ... in case they're too long. Tags are also hardly consistent across all beatmaps of all times, and it seems there is no existing functionality to edit them on the new site. Should this be the approach we take then ^ will appear in separate issues.

venix12 commented 4 years ago

couldn't certain tags be somehow highlighted though (e. g. on beatmapset card as you proposed)? believe it's possible and would work nicely here

frukoyurdakul commented 4 years ago

Preventing the nomination is not a good solution when the content does not suit what it has already. A music might fall into a category that fits to none. Why would picking "other" would prevent the nomination then? What even is the point of having that for instance if not making the beatmap prepared for nomination rather than leaving it "unspecified"?

cl8n commented 4 years ago

Great, so now you have a "cover" of a "vocaloid" song from a "video game". How does this fit?

idk why this was skipped past so quickly in the meeting, some people brought up the concern and the only real response I can find is

[12:56 PM] yaspo: yeah we kinda need the option to have multiple at this rate

so then it would seem super pointless to put them in the same 'category' because they're unrelated and you can set multiple.

"common tag filters as shortcuts" is what I'd been getting at during the meeting, but if you use the plaintext tags field to do this you run into the problems yaspo mentioned above, and you also need to rely on mappers and modders being aware of these "common tags" because the filter wouldn't be very useful if it didn't catch all of the maps that it applies to.

instead of special genre and language and 'extra category' and common tag filters I think it would be a lot more straightforward to database as many tags as possible and add types to them so they show up in the right place and can be made optional/required as necessary (like booru). it's definitely not the same scope as the OP of this issue but that meeting highlighted a bunch of issues that could be solved this way

yaspo commented 4 years ago

@frukoyurdakul note the "both", it'd only be limiting in the case of Unspecified - Other, which with making Genre/Language as broad as possible ideally never happens (you'd have to really try)

Fully support @cl8n's idea as well, in general it seems an improvement to describe information rather than keeping a collection of non-descript tags around and hope the user will somehow make use of it. This especially goes for the beatmap page where they're visible. The addition of 'Extra Category' or 'Song Type' aims to improve that in a way that's a bit less far off.

yaspo commented 4 years ago

Alright, back on track, feels like this didn't entirely go as it should've. I took some time to reconsider some of the points made regarding "Extra Category". The main goals we want to reach are categorization, searchability and correctness + visibility of these categories. These are to support the evergrowing list of beatmaps by allowing for more defined filtered lists, efficient searching, and improving discoverability in amount and accuracy.

you have a "cover" of a "vocaloid" song from a "video game"

Taking this sentence as a base, the main issue in the original "Extra Categories" list is combining items that are semantically different into one list; "is (a)" versus "from (a)".

So, I'd suggest to split these up into "Type" for "is (a)" and "Source" for "from (a)". Source is evident, Type is fairly natural to use. For example "types of food" and "x is a fruit/vegetable", "y is meat". This results in the following categorizations:

This should resolve the naming issue.

As for common search terms, I think they're a nice idea but they don't answer to the need for categorization. It'd also be weird to put down static recommendations, since generally any storefront seems to generate recommendations dynamically based on common search behavior. So personally I'd recommend against using common searches to replace our suggested improvements; they should be their own thing. Same thing for clayton's proposal.

Does this address your concerns @peppy ?

peppy commented 4 years ago

We already have a source field. We can't add a second one it will just get confusing. Why not call it "medium" or something more appropriate?

And oh god this is just too complicated. Why can't we leave this stuff to tags? I mean what is the benefit of this

cl8n commented 4 years ago

@yaspo "Type" still isn't a category, those properties of songs aren't related to each other. and yeah "Source" is listing media

@peppy the benefit is the same that you get from genre and language already, you add the ability to browse the options themselves, and prompt mappers/modders to pick something out of the options available. it makes organization easier too because full-text tagging requires everyone to follow precise standards, words have spelling overlap with unrelated words, etc.... anyway I agree that building out this one-table-per-category thing for even more options would be complicated and messy, that's why I suggested the alternative in my last post.

for the time being can we do the following (from yaspo):

  • Add genres Metal, Classical, Folk, Jazz. These don't fall under any existing genre.
  • Add Languages: Russian, Polish, Conlang. Russian and Polish are popular languages for beatmaps, conlang covers made-up languages.

that would solve the issue reported in OP.

peppy commented 4 years ago

is "conlang" really an understood word? would that not just fit best under the existing "other" category?

cl8n commented 4 years ago

I don't think it'd be understood well, but I just read over the #modding discussion and it seems like ppl wanted Conlang to differentiate novelty languages from everything else. currently "other" is including a lot of songs with multiple main languages, and languages that aren't in the listing, so there's no good way to search for that if you wanted to

I'm not sure if it's prominent enough to include that. leaning towards no

peppy commented 4 years ago

Apart from that, I think it's fine to add the rest.

Note that this requires localisation entries to be added in osu-web code, and also updates to stable / lazer. If anyone wants to contribute to this to get it working, we can move forward with it.

Also should check if will still fit horizontally in the beatmap listing design.

lassem22 commented 4 years ago

I think having conlang in there would be nice to have a better differentiation between songs in fictional languages and such in languages that are just rarely mapped.

But then again, a big part of the songs ranked in osu that use fictional languages (that I know of) use a mix of those and Japanese and thus would get set to Japanese anyway, leaving the Conlang category pretty empty unless setting multiple languages would be possible as well.

tldr: Makes a lot of sense to have from a "help people find what they are actually looking for" standpoint, but would be quite empty.

yaspo commented 4 years ago

Conlang definitely is a "nice-to-have" compared to the other languages listed for addition, especially with how niche it'd be as Lasse mentions. In that sense, attention should probably go to real languages first and a way to handle songs with mixed languages; that way conlang will sort itself out (ideally).

So, yeah, leaving conlang out is probably the better call here.

cl8n commented 4 years ago

the filters are still a single line on normal width, if you make it smaller it can wrap to a second line but the design was intended to do that as well. see current mobile view

those should be it for lazer/web ^ where are genres and languages used in stable? that's news to me

cl8n commented 4 years ago

I guess I lied about it being one line because I wasn't planning to add Unspecified originally. still looks fine on web though

Ephemeralis commented 4 years ago

This cropped up again in discussion with the NAT last night.

The general consensus was that as is, the current genre/language classification system is not really adequate as it stands now for one core reason: granularity.

Type/source/genre/language classifications essentially offer only one best fit for a given song. Many works push across multiple genres, styles or even sources at times, leading towards there being a concerted and ultimately wholly arbitrary declaration of which particular set of data is the "best fit" for a given set.

Some tracks may benefit from multiple classifications based on the works in which they appear - a good example would be the track Falling Down by Oasis, which has appeared in two different sources, both Guitar Hero: World Tour and in the anime Eden of the East. Any attempt under the current system to correct denote this means that either bits of data (both of which are relevant to users trying to search for the track) risk being lost since only one of them can be displayed at a given time. Someone has to make the call about which of these sources is greater than the other, which destroys data in the process and potentially makes searching harder for users overall.

This is often addressed by other sources or pertinent into being offloaded into the beatmap's tags instead, which got me thinking... why don't we just switch to an entirely tags based system similar to how booru-style imageboards operate?

booru-style tagging moderated by the NAT+GMT to avoid orphaned tags encompasses the best of all worlds and allows us to classify beatmaps with any pertinent information. It also allows osu-web the means to generate popular categories for search filtering based on tag prominence, permitting the search listing to be organically representative without need for developer intervention or issues like this.

peppy commented 4 years ago

https://github.com/ppy/osu-web/issues/5852#issuecomment-625619084 ?

Ephemeralis commented 4 years ago

Right, but what I'm saying is we'd benefit from going even further and moving the whole thing across to a tags-based system at some point instead of leaving stuff as it is now.

peppy commented 4 years ago

as in all genres? that would definitely be the next step. a good start could be soft-enforcing the tag prefix genre: for non-existent or multiple genres. can be used immediately with no dev side changes.

Ephemeralis commented 4 years ago

We can do that (and probably will), but I really genuinely mean everything, turning beatmap classification metadata into giant collections of tags with category sorting a la sites like danbooru. Genre, source, the works. It wouldn't be too difficult to get the NAT to retroactively go through all existing ranked maps and assign appropriate tags to them.

cl8n commented 4 years ago

typed tags like that could be set up alongside existing metadata for now. it'd be nice to have a "test" system up to at least allow for creating and setting these tags, even if they don't do anything, to gauge how much effort it would take to plan and maintain

I'm all for it and a big fan of those types of databases as said in my earlier comments here 👍

gustarmartins commented 3 years ago

I think adding the Portuguese language could still be a nice implementation while the new proposal made by Ephemeral is not made.

8 months ago yaspo added new languages like Polish and Russian, so it doesn't seem to be much trouble adding one more (which already has enough beatmaps to support it) and would definitely help the Brazilian and Portuguese community while the new proposal is not applied.

You could also be adding languages based on the most spoken languages in the entire world, right now we got all the languages in there except for Portuguese, Hindi, Arabic, Bengali, and Indonesian; while these last four has probably 0 to 5 beatmaps in the Ranked/Loved section, making it probably not necessary at the moment, leaving only Portuguese.

I can't code, so I don't know how to add that through a pull request, but it shouldn't be hard, I suppose ^^

AJT127 commented 2 years ago

bumping since i made https://github.com/ppy/osu-web/issues/8345 then was Alerted to the fact that this exists

doggoboat commented 1 year ago

bumping this since I was just made aware of it, made #9601 beforehand

flashlight-fl commented 1 year ago

I was going to open a new issue regarding the Portuguese tag, but I discovered this exists.

Since MkGuh's comment in 2020, nine maps in Portuguese got ranked, and one is currently qualified. Including the maps mentioned in #6963, if I counted correctly, this will totalize 37 ranked + loved maps in Portuguese. For comparison, Polish got its own tag in 2020, but it only has 19 ranked + loved maps where it's applicable, and Swedish has only 15 ranked maps.

As recently as 2020, Polish and Russian were added as tags; considering the arguments provided, I don't see a reason to not add Portuguese, considering it's one of the biggest languages in the world and it is being actively represented in osu!, as MkGuh said in his comment.

"[...] yaspo added new languages like Polish and Russian, so it doesn't seem to be much trouble adding one more (which already has enough beatmaps to support it) and would definitely help the Brazilian and Portuguese community while the new proposal is not applied."