projectfluent / fluent

Fluent — planning, spec and documentation
https://projectfluent.org
Apache License 2.0
1.4k stars 45 forks source link

How to do arithmetics, i.e. add/subtract numbers? Or how to parse building's floor levels? #227

Open rugk opened 5 years ago

rugk commented 5 years ago

Is there any way to manipulate integers in the translation to e.g. add +1 or subtract -1?

I am going to present you the use-case, but I would like to know the syntax before that, if it exists.

rugk commented 5 years ago

So finally my use case:

I want to translate/parse building levels/floor numbers. As you probably know, in some countries (US, Russia) the floor 1 = ground floor, while in (most?) others it is floor 0 = ground floor. This results in a hard problem, and as per this issue, I e.g. need to do calculations on the integer data.

As for some background info: I am coming from https://github.com/westnordost/StreetComplete/issues/1270 and the source format of the data is used in OpenStreetMap (OSM).

Gist: I cold come up with this gist. But it is totally a WIP and due to the issues here, they only work properly in the range of -1 to +1. https://gist.github.com/rugk/b7886c7332dac68d9b83ed534b5062e0

(Automatic loading into playground does not seem to work [https://github.com/projectfluent/play/issues/7], so you have to copy/paste it.)

stasm commented 5 years ago

That's an interesting use-case, thanks for bringing it to my attention, @rugk!

Fluent doesn't have a built-in way to add or subtract numbers. It was a deliberate design decision; we wanted to keep the language simple and less powerful, so that it's easier to learn, validate and to reason about.

We left a gateway for enabling more complex computations in the form of custom functions. These need to be defined in the runtime in which Fluent translations are used. In other words, it's up to the developers of the application to add them; they cannot be added by localizers inside of the .ftl files. This is also deliberate: custom functions can be written in the same language as the rest of the app and can use the entire power of the available standard library. Plus, the logic they provide can be properly tested by the test suite of the app.

To serve your use-case, I'd suggest the following two approaches, both of which leverage custom functions. I'm going to use the translations you provided in your gist.

1. Custom Arithmetic Functions

In the first approach, I would suggest adding a very simple custom function INCREMENT(x) which literally just adds 1 to the number value it receives.

floor_ENGLI_in_EN = { $level ->
    [-2] on basement floor B{ $level }
    [-1] on basement floor B
    [0] on ground floor
    [one] on floor 2
   *[other] on floor { INCREMENT($level) }
}

Alternatively, you could implement ADD(x, y) which would allow more complex arithmetic, including subtraction.

floor_ENGLI_in_EN = { $level ->
    [-2] on basement floor B{ $level }
    [-1] on basement floor B
    [0] on ground floor
    [one] on floor 2
   *[other] on floor { ADD($level, 1) }
}

We don't currently have good docs on how to write custom functions, sadly. Let me briefly explain how they work here, and I'll work on improving the docs before the 1.0 release. You may find the implementation of the default built-in functions somewhat helpful. The first argument is a list of positional arguments the function receives. All values are instances of FluentType subclasses. To work with native JS values, you first need to call .valueOf() on each argument. The return value of the function must be a FluentType instance as well. Here's how I'd implement the INCREMENT function:

function INCREMENT([num]) {
    return new FluentNumber(num.valueOf() + 1, num.opts);
}

Such function can be passed to the FluentBundle constructor as a property of the functions option.

2. Custom Formatter Functions

Another approach, which would also be my preferred, would be to create a custom function which performs the locale-aware adjustment of the floor number. I think this is an improvement over the first approach because the logic provided by the function is more specific to the domain of the translations. In your use-case, I would find it logically sound that a mapping app has custom functions related to handling floor numbers.

floor_ENGLI_in_EN = { $level ->
    [-2] on basement floor B{ FLOOR($level, ground: 1 ) }
    [-1] on basement floor B
    [0] on ground floor
    [one] on floor 2
   *[other] on floor { FLOOR($level, ground: 1) }
}

The FLOOR function might be implemented as follows:

function FLOOR([num], {ground}) {
    if (num.valueOf() < 0) {
        return num;
    }
    return new FluentNumber(num.valueOf() + ground.valueOf(), num.opts);
}

I'm using a named argument called ground to specify what floor number is associated with the ground level. Perhaps even a better way would be to skip that named argument entirely and instead define this data for each locale you support. It would be as if FLOOR was an i18n formatter for floor levels. However, as of today, custom functions don't have access to the information about the current language, so you'd need to pass it explicitly from inside of the translations, just like I did with the ground argument above. I filed https://github.com/projectfluent/fluent.js/issues/330 to consider enhancing custom functions in the JS implementation of Fluent in a way which would allow this.

(I kind of wish floor numbering was something CLDR exposed, same as they do with the measuring systems, first day of the week, or even the standard paper size. If you think this is something worth proposing, feel free to file an issue in their Trac.)

Lastly, you can also take advantage of custom functions to provide the functionality you mentioned in https://github.com/projectfluent/fluent/issues/228. For instance, you could write a custom function which takes a number and returns one of (under, ground, over). You'd then use it like so:

floor_ENGLI_in_EN = { FLOOR_TYPE($level) ->
    [under] on basement floor B{ FLOOR_NUMBER($level, ground: 1 ) }
    [ground] on ground floor
   *[over] on floor { FLOOR_NUMBER($level, ground: 1) }
}

See https://github.com/projectfluent/fluent/issues/177 for more discussion about using custom functions to effectively create enumerations of named ranges for numbered data.


I hope this helps :) Please let know if this answers your questions. I know that we need to improve the documentation for these little-known features; I'll be working on this in the near future. Thank you for offering a great use-case and for using Fluent!

rugk commented 5 years ago

BTW, as for docs: If you want to explain custom functions, that use-case may be quite well-suited, I'd say.

Personally, I would not implement FLOOR. It's not really descriptive. (I mean, translators also have to know what the function does. INCREMENT/ADD is obvious, while FLOOR is not. So you would have to document your custom functions for translators. And I have no idea how to do that properly - I mean, it's not the samne as documenting for programmers. Maybe this requires a new issue/thought?)

Also, in your "Custom Arithmetic Functions" you forgot one CEIL or so for negative floor levels (e.g. basement floor).

rugk commented 5 years ago

That all said, I still would think such a common thing as arithmetics support would be nice. Especially, if you could prevent this function-syntax and just write it as in programming languages with the "usual" operators. I e.g. initially tried to write it like this:

floor_ENGLI_in_EN = { $level ->
    [-2] on basement floor B{ $level - 2*$level }
    [-1] on basement floor B
    [0] on ground floor
    [one] on floor 2
   *[other] on floor { $level + 1 }
}

(I know, that ceiling is awkward there then, but possibly this could also be a implemented function.)

I do however, understand, that it may be too much for such a language as this.

rugk commented 5 years ago

(I kind of wish floor numbering was something CLDR exposed, same as they do with the measuring systems, first day of the week, or even the standard paper size. If you think this is something worth proposing, feel free to file an issue in their Trac.)

Yeah, why not? I mean, it is certainly worth a try: https://unicode.org/cldr/trac/ticket/11793 Feel free to add any information. I have no idea what else I am supposed to write/explain there.

stasm commented 5 years ago

I do however, understand, that it may be too much for such a language as this.

This has been our guiding principle in designing Fluent. Fluent is first and foremost a declarative transport format for storing translations. It allows some limited logic to be defined but anything more complex should be outsourced to custom functions.

Yeah, why not? I mean, it is certainly worth a try: https://unicode.org/cldr/trac/ticket/11793 Feel free to add any information. I have no idea what else I am supposed to write/explain there.

Thanks! I think it would be a valuable addition to the CLDR.

rugk commented 4 years ago

So what shall we do with this issue here and the closely related https://github.com/projectfluent/fluent/issues/228? Has any conclusion been reached what/how things may be changed in fluent (for this use case or related ones)?


BTW great news I did not notice, because apparently Unicode migrated from Trac to Jira and I also did not got any mails in Trac, but apparently they have decided to include this information in CL;DR. (Or do they? TBD = to be done? or "to be decided"? And is the tag "new" better than "accepted" uff what?? Phase "dsub" – sry, whaat?)

zbraniecki commented 4 years ago

So what shall we do with this issue here and the closely related #228?

Does the ADD/INCREMENT solution suit your needs?

I think we'd like to shy away from arithmetic expressions for as long as possible. :)

rugk commented 4 years ago

Yeah, maybe, I guess that works.