Lemmatization of t-adverbials

[AleksandrsBerdicevskis]

(See previous discussion here: https://github.com/UniversalDependencies/UD_Swedish-Talbanken/issues/8).

In Talbanken (both UD and SBX), adverbs that are homonymous with neuter adjectives (samtidigt) are lemmatized in the same way as adjectives. i.e. samtidig. The same is true for all machine-annotated Språkbanken's corpora, since this is the lemmatization provided by Sparv (which, in turn, must depend on how adverbs are treated in Saldo).

Meanwhile, in SUC all such adverbs are lemmatized using a t-form (samtidigt). This is also how Efselab (trained on SUC) lemmatizes adverbs. Swedish_Lines is not fully consistent wrt to adverb lemmatization. A related question is how comparatives and superlatives should be treated (a lot of inconsistency within UD, for instance).

My opinion: 1) There are two legitimate solutions: either treat t-forms as adjectives and lemmatize them as samtidig (in line with the SAG analysis) or treat them as adverbs and lemmatize them as samtidigt, but the current hybrid is inconsistent and not motivated. Both solutions have their advantages.

Lars Borin: "SAG analys är att orden i punkt (3) ("snabbt", etc.) faktiskt är att betrakta som adjektiv och att ord av ordklassen adjektiv kan uppträda som satsdelen adverbial och att man då använder deras t-form (analogt med att ord av ordklassen substantiv också kan uppträda som adverbial, typ /måndagar hämtar jag på dagis/)... SAG:s analys var något av en innovation när den kom, men åtminstone jag gillar den mer än att ha -t som både böjningsändelse och ordbildningsmorfem hos adjektiven."

Lars Ahrenberg: "My current preference is for t-forms as lemmas as there are some words where the meaning of the adverb is somewhat different from that of the corresponding adjective, for example 'riktig(t), väldig(t)'" (AB: I concur!)

Joakim Nivre about UD: "The lemmatization has been done automatically (using an early version of the Språkbanken pipeline) and has only been partially validated manually. So my guess is that these are all (?) cases where the disambiguation has gone wrong. Do you at Språkbanken generally assign the lemma "samtidigt" (rather than "samtidig") to the adverb "samtidigt"? If yes, then we should do the same in the treebank."

2) Lemmatization of comparatives and superlatives should follow the lemmatization of positives.

Can we choose one of the solutions for all the resources? Or shall we agree to disagree? In any case, I think, we should get rid of the current hybrid.

[gerlofbouma]

I agree with the opinion in 1, that if you treat them as adverbs they go with t, if you treat them as adjectives, they go without.

Koala follows SAG and treats these as adjectives, so the lemmatization – whenever that is going to happen – will be to the adjectival base form.

[gerlofbouma]

Lars Ahrenberg: "My current preference is for t-forms as lemmas as there are some words where the meaning of the adverb is somewhat different from that of the corresponding adjective, for example 'riktig(t), väldig(t)'" (AB: I concur!)

Koala was in on this track for a while too. The idea was that word semantics is not a direct argument for one POS over another, but it is a way of distinguishing lexical items, so that one in principle could say: attributive/predicative sjuk+infl and intensifier (=adverbial?) sjukt are semantically distinguishable, and can thus be seen as different items with different distributions, and if the latter only is adverbial, then we should consider treating the former as an adjective and the latter as an adverb.

The fly in the ointment is however the fact that, at least for these intensifiers, they can typically be used in front of massa with their intensifying meaning intact. And then they are placed and agree like adjectives. So: en sjuk/väldig/riktig massa jobb/bilar/pengar/etc

The result is that we have never really found any good examples of these items that couldn't be used as an adjective with its supposedly adverb meaning. So we gave up on this idea after a while and treat all of them as adjectives. But we can absolutely be convinced otherwise if the right data comes up.

[larsborin]

Saldo (v. 2.3) has 49 adverb entries for which there is also an identical adjective neuter form (see the list below), sjukt among them (although I would be prepared to accept Gerlof's argument in this case, at least, which then instead would yield an additional sense of the adjective sjuk).

These are probably mostly remnants from the original entry set that Lennart Lönngren used as the basis for SAL, the lemmas from the first Svensk ordbok (which Språkdata sold to him at a not insubstantial price!).

In some cases, the direction of derivation is probably adv > adj (gratis, turvis), and this of course is a different case from the SAG analysis.

absolut..ab.1 ackurat..ab.1 akut..ab.1 alternativt..ab.1 blott..ab.1 bra..ab.2 direkt..ab.1 djävligt..ab.1 fast..ab.1 fett..ab.1 flott..ab.1 fruktansvärt..ab.1 förfärligt..ab.1 galant..ab.1 gratis..ab.1 grymt..ab.1 hejvilt..ab.1 helt..ab.1 hemskt..ab.1 insides..ab.1 just..ab.1 jävligt..ab.1 klockrent..ab.1 knappt..ab.1 kort..ab.1 litet..ab.1 långt..ab.1 nätt..ab.1 oavvänt..ab.1 obehindrat..ab.1 osökt..ab.1 platt..ab.1 relativt..ab.1 runt..ab.1 rätt..ab.1 saklöst..ab.1 sjukt..ab.1 självfallet..ab.1 skarpt..ab.1 snart..ab.1 spritt..ab.1 stint..ab.1 styvt..ab.1 särskilt..ab.1 turvis..ab.1 tätt..ab.1 vackert..ab.1 vansinnigt..ab.1 visst..ab.1

[AleksandrsBerdicevskis]

So what would your suggestion be, Lars?

[larsborin]

It would be along the lines of what SAG says, what Saldo does, and the Koala solution, namely that they are treated as adjectives, and that, syntactically, adjectives can function as adverbials, in which case they appear in the t-form (the strong neuter singular form, which as SAG points out should be treated as the unmarked adjective form also on other evidence). Consequently, the t-adverbials in question should be lemmatized as adjectives.

Only in cases of clear semantic differences coinciding with the difference between attributive/predicative position and adverbial position should a separate adverb lexical enttry be posited. This is a judgement call, of course, and different individuals will come up with different sets. Among the 49 Saldo lemgrams that I listed there are perhaps a few clear cases (e.g. spritt [as in spritt galen]), but most of them are debatable. Note that this is a separate consideration from the main conclusion above.

For the items that (at least etymologically) represent a derivation ab > av, we could still consider them to fall under the main principle in a strictly synchronic perspective and list them only as adjectives in Saldo. Hence, gradvis would be only an adjective, whose "t-form" is also gradvis (but alternatively also gradvist). This is perhaps the more attractive alternative: remove all adjective-adverb doublets from the lexicon, regardless of their origin, unless there are clear semantic arguments to the contrary.

[AleksandrsBerdicevskis]

In practical terms then:

• I can do changes to TalbankenSBX;
• it is up to @jnivre and @LarsAhrenberg to decide what happens to TalbankenUD, LinES and PUD (which seems to opt for POS=ADV, lemma=t-form);
• you make changes to Saldo, if necessary (very strictly speaking, this has to come first, if we want the corpus annotation to depend on Saldo decisions, but I guess it will affect a very small proportion of cases);
• what about Sparv? How do we ensure that it lemmatizes (most) t-adverbials as adjectives?

[larsborin]

The only reasonable strategy wrt Saldo is to make any required changes for v. 3.0 (which has been underway for much too long :-(), but since Saldo by and large (i.e., with the exception of the around 50 cases that I listed) already implements the model lemma=adj, this shouldn't matter very much. As you say, they are marginal: there are about 21,000 adjective lemmas in Saldo 2.3.

[AleksandrsBerdicevskis]

Do we leave SUC as is (POS=ADV, lemma=t-form)? If yes, won't poor Sparv be confused by heterogeneous input from Talbanken and SUC? (As it must be also now.)

[larsborin]

The only reasonable solution in the case of SUC would be to prepare and release a revised version (SUC 3.1 or SUC 4?). The present SUC 2 and 3 must be around and available for reproducibility. The revisid version could be the default in Korp, and the SUC license of course will apply to it, so that it can be bundled with original SUC and stor-SUC.

In that connection it's worth recalling that SAG makes a similar analys wrt verb particles, which is only a syntactic function according to them, but not a part of speech. If the particle can also appear as a preposition, it is analyzed as a preposition even in its particle function (e.g. på, where SAG does not recognize a homonymous adverb).

[AleksandrsBerdicevskis]

Yes, I've already suggested creating SUC 3.1 (https://github.com/SwedishJewellery/discussions/issues/3), but the question is, do we want to change the lemmatization of t-adverbials? (Given that the current SUC is internally consistent.)

(I'm coming to particles, too.)

[larsborin]

I would be in favor of revising the lemmatization, yes.

[LarsAhrenberg]

Efter att ha tänkt på detta ett tag har jag kommit fram till att det nog inte är så skadligt om olika resurser gör på olika sätt. Däremot bör ju varje trädbank vara internt konsekvent, och, vad gäller UD, att alla svenska UD-trädbanker bör göra likadant. Läser man vad UD säger om lemman får man inte så mycket vägledning: varje språk kan välja efter tradition, samtidigt som man anger att lemmat ska ange 'the semantic content' av en ordform. Eftersom UD inte har någon särskild kolumn för 'word sense' är det frestande att göra de semantiska skillnader som går under Lemma. Oavsett val kommer det, som Lars B påpekar, alltid att finnas gränsfall. Andra ord som bör ingå i diskussionen är adverb som slutar på -en (egentligen, verkligen, möjligen etc.) där SAG verkar tycka att -en kan ses som ett alternativ för -t i vissa av deras adverbiella användningar.

[larsborin]

Apropå -en-adverben: För mig finns kärnan i SAG:s argumentation i det här (och det kan hända att jag tillskriver dem åsikter som de inte har): Adjektivets t-form finns i vilket fall som helst, (oberoende av adverbialanvändningen) och den har dessutom andra (icke-prototypiska) användningar som redan nästan tar den in i adverbialzonen. Det är ju så att både finita och icke-finita satser karakteriseras just med adjektivets t-form: Att segla är nödvändigt; Att de måste segla just här är väldigt olyckligt. t-formen dyker även upp i andra sammanhang där man kanske inte skulle ha förväntat sig det, som med många sorters icke-personnamn (proprier som inte betecknar personer): Uppsala är outhärdligt på vintern. Därför är det inte en helt orimlig tanke att t-formen även kan tas i bruk för att uttrycka adverbial (och även med tanke på att även t.ex. substantiv kan fungera som adverbial, så det är inte enbart förbehållet ordklassen adverb).

Med en-formerna färhåller det sig annorlunda. De har ingen annan funktion än just den adverbiella, så då ligger det närmast till hands att betrakta -en som en ordbildare, som inte gör något annat än bytrer ordklass från adjektiv till adverb, och därmed bildar en ny lexikoningång med en egen grundform (lemma). Samma sak gäller för particip- och verbalsubstantivbildarna (-ande, -ad/-en, -ande). De har inga andra funtioner än just att bilda particip (adjektiv i nästa Saldo, i motsättning till SAG, som antar participen som egen ordklass på grumliga grunder) och substantiv av verb.

Angående "semntic content": Där är det väldigt mycket en fråga om beskrivningstradition. Den ryska lexikologi som Saldos föregångare SAL härrör ur brukar anse att enbart ordklassbyte (conversion) inte räcker för att man ska anse att det semantiska innehållet är ändrat (vilket väl också är tanken bakom den svenska traditionellare beskrivningsmodellen där participen hör hemma i verbets böjingsparadigm, trots att de är av en annan ordklass). I SAL fanns bara en ingång "alternativ", eftersom substantivet och adjektivet har samma semantiska innehålll; ordklassen avspeglar enbart deras kombinerbarhet i en syntaktisk kontext. Saldo har i alla såna fall två ingångar, som en eftergift till en tradition där olika ordklass är ett tillräckligt villkor för att man ska ansätta olika betydelser (som i WordNet, där synseten har ordklass per definition).