ClojureScript support - Githubissues

ptaoussanis commented 11 years ago

This is a completely unbaked idea; have given it zero consideration. Would love to see something happen long-term though, so open to ideas/comments/use-cases to start warming this up...

ptaoussanis commented 11 years ago

Maybe some places to get started: http://stackoverflow.com/questions/3084675/internationalization-in-javascript http://stackoverflow.com/questions/9640630/javascript-i18n-internationalization-frameworks-libraries-for-client-side-use

asmala commented 11 years ago

I've been thinking about this as well. We could probably provide a very similar or even identical API for i18n and l10n for both Clojure and ClojureScript. The l10n functions could be implemented with simple wrapping around native JS or perhaps goog.i18n. For i18n it may make sense to make t into a macro that does part of the translation look-up during compile time.

The one part where alignment between the two implementations would get tricky is string formatting, since, I think, the two platforms use a different string format. goog.string.format seems to come pretty close to the Java implementation, though.

ptaoussanis commented 11 years ago

We could probably provide a very similar or even identical API for i18n and l10n for both Clojure and ClojureScript

That would be the ideal, certainly. t can probably be converted as-is already (or nearly as is) since it's all Clojure. Don't foresee any problems there.

The L10n stuff should probably wrap some platform-specific lib as you suggest, basically as the L10n stuff does currently. I've zero experience with i18n in JS so no idea of the relevant pros & cons. Any insights there?

For i18n it may make sense to make t into a macro that does part of the translation look-up during compile time.

Hmm, that's an interesting idea I hadn't considered. What draws you in this direction? My inclination would be to say let's just try dump the entire t implementation as-is into Cljs and see if it works. If there's any tweaks necessary, they'll probably be minor. For example, we may want an alternative to the dictionary resource loader (Any ideas there? Maybe a web accessible .clj URL?).

Am in no rush for any of this btw - we can take some time to think about it / experiment.

Cheers :-)

asmala commented 11 years ago

I'm by no means a JS expert. I only looked into the JS i18n matter briefly while playing around with Clj18n. The implementation of the vanilla toLocaleString method is not consistent enough to rely on, so a library is a must. Given ClojureScript's affinity with the Closure compiler, I'm naturally drawn to the i18n functionality provided by the Closure library. Between goog.i18n and goog.string.format, I think we should be good.

The motivation behind my suggestion of implementing t as a macro is to avoid sending a massive dictionary to the client. In fact, this might be something to consider for Tower as well, if library size ever starts to be a drag. Granted, a possible case of premature optimization but just thought I'd put that out there. Here's a sample of what I had in mind:

; Assuming a dictionary that looks something like this…
{:en-US {:my {:namespace {:hi "Hi", :bye "Bye"}}}
 :fi {:my {:namespace {:hi "Terve", :bye "Näkemiin"}}}}

; …then the following…
(t :my.namespace/hi)

; …would compile into something like this:
({"en_US" "Hi", "fi" "Terve"} (goog.locale/getLocale))

ptaoussanis commented 11 years ago

I only looked into the JS i18n matter briefly while playing around with Clj18n.

Any plans to experiment with it on the horizon? I'm just getting underway with a new startup now, so would prefer to put my lib involvement on the back-burner for a while as much as possible. The recent sprint with Tower was unanticipated and mostly to put it into a stable medium-term position after your irresistible suggestions.

I will need JS-side i18n at some point, but am going to try put it off for as long as possible.

The motivation behind my suggestion of implementing t as a macro is to avoid sending a massive dictionary to the client.

Hmm, I don't know. My guess is that the client-side dictionary will (should?) usually be pretty small - esp. post-gzip in relation to something like the Cljs overhead itself. It's a worthwhile point to consider though - I'd definitely be open to exploring it.

In fact, this might be something to consider for Tower as well, if library size ever starts to be a drag.

Library size? You mean dictionary size? I would absolutely not worry about this on the server-side, the amount of memory even a 1m+ entry dictionary would occupy is insignificant. And products that have such large dictionaries would likely be running on serious hardware anyway.

Now that you mention it, it should actually be possible (+ easy) to restructure the compiled dictionary format to be considerably more storage efficient by structuring entries as [:key :locale] rather than [:locale :key]. I'll make a note of that. Either way, really wouldn't worry about it.

ptaoussanis commented 11 years ago

BTW just pushed https://github.com/ptaoussanis/tower/commit/7521cf84c4e9232df299da188480ad4106da900f which makes the compiled dict format > 2x as storage efficient for most common dict types.

ptaoussanis commented 11 years ago

To clarify: this suggests that an alternative approach may be to keep t as a fn client-side, but to precompile the dict. Then we could, for example, offer a macro to compile the dict, but leave t otherwise unchanged.

And the [new] compiled format is actually more efficient than treating t as a macro, since we're essentially deduplicating t content compared to an in-place t expansion.

Does that make sense?

This also carries the advantage of keeping the compiler [the most complex code now and probably in future] Clojure-side. Which gives us the freedom to use non-Cljs deps in future if we decide to, etc.

asmala commented 11 years ago

Moving locale to the end of the of compiled dictionary path? That's an interesting idea. Why not indeed.

I don't have any immediate plans for ClojureScript I18n, but neither did I have plans to revisit Clojure I18n in the first place. :-) What I might do is experiment with ClojureScript implementations for Clj18n (simpler, less featureful code base), and then see if the implementation approach would also work for Tower. Will keep you posted.

ptaoussanis commented 11 years ago

Moving locale to the end of the of compiled dictionary path? That's an interesting idea. Why not indeed.

While it seems like a small change, it's actually quite a significant diff. in terms of storage needs since keys are usually: a) Far more numerous than locales and b) Far longer per key. This way the total content we need to store is minimized.

In any event, like I said - think the storage needs are insignificant anyway. If I were at all concerned about this I wouldn't have gone with the dictionary expansion strategy, for example.

Will keep you posted.

Great. And I'll update here if anything comes up on my end.

ptaoussanis commented 11 years ago

Hi Janne - any more thoughts on this issue? I've got a project coming up that may need ClojureScript-side Tower support, so I want to start thinking about concrete next steps to get this implemented. May or may not materialize, but want to be ready.

asmala commented 11 years ago

Hey Peter,

I've been rather swamped with my day job, but one thing worth adding to the above is that goog.string.format isn't locale aware. In other words:

(def my-tconfig
  {:dictionary
   {:en {:title {:day "Today is %A"}}
    :fi {:title {:day "Tänään on %A"}}}})

; Tower in Clojure
(t :en my-tconfig :title/day current-date) ;=> "Today is Friday"
(t :fi my-tconfig :title/day current-date) ;=> "Tänään on perjantai"

; Tower in ClojureScript using goog.string.format
(t :en my-tconfig :title/day current-date) ;=> "Today is Friday"
(t :fi my-tconfig :title/day current-date) ;=> "Tänään on Friday"

Rewriting goog.string.format and adding locale support would be a lot of code although quite straightforward.

ptaoussanis commented 10 years ago

Rewriting goog.string.format and adding locale support would be a lot of code although quite straightforward.

Thanks for pointing this out. May end up just punting on it for now since the formatting fn was made user-interchangeable a while back. Might be best to add the Cljs support in stages.

ptaoussanis commented 10 years ago

Hi Janne,

Quick update: just pushed a commit to the cljs branch that gives preliminary, working ClojureScript translate & t fns.

Unlike Clojure-side config, the Cljs dictionaries need to be explicitly compiled:

;; (:require-macros [cljs.taoensso.tower.macros :as tower-macros])
(comment ; ClojureScript Tower dictionaries (this is cljs)
  (def my-dict-inline   (tower-macros/dict-compile {:en {:a "**hello**"}}))
  (def my-dict-resource (tower-macros/dict-compile "slurps/i18n/utils.clj")))

The dict-compile macro just calls the standard Tower dictionary compiler - so the only difference here is that it's being done compile-time and the compiled dictionary embedded in the output .js. This works quite nicely IMO since we get the exact same dictionary feature set, and we can even load ClojureScript dictionaries from Java resource files.

Haven't done any research on a locale-aware formatting fn, or on any other localization features. It's likely I won't have time for either of those for at least a couple months since they're low-priority for my own projects.

You're very welcome to take a look at the https://github.com/ptaoussanis/tower/blob/cljs/src/cljs/cljs/taoensso/tower.cljs ns if you feel like playing around with extending any other features some time.

Otherwise, feedback welcome on the translation stuff. Won't be cutting a release any time soon so plenty of time to experiment.

Cheers! :-)

asmala commented 10 years ago

Looks good.

The formatting function looks straightforward enough, and easy to build in stages by incorporating one pattern at a time. Here's a rough sketch of how it could work:

Split string into static components and formatting patterns (e.g. "Today is %A" => ("Today is " "%A")) using a regex
For each pattern: i. Identify type (e.g. "%A" represents a DateTime instance) ii. Call appropriate goog.i18n formatter with the pattern and the matching arg
Merge

The goog.i18n formatters are locale aware, using goog.LOCALE. The only tricky bit that I can see off the bat is multipart patterns (e.g. "hh:mm").

robert-stuttaford commented 10 years ago

Just a ping to see what the state of ClojureScript support is like, now? We're about to start our own i18n journey with Tower and I'm curious how easily we'll be able to extend to the client as well.

Thanks for such a great library!

ptaoussanis commented 10 years ago

Hi Robert,

Thanks for such a great library!

You're welcome, thanks for the shout-out :-)

Just a ping to see what the state of ClojureScript support is like, now?

Sure. There's unofficial support on the cljs branch & on Clojars as "2.1.0-SNAPSHOT". This gives simple (but working) dictionary/t functionality. I'm using this for an app in development right now.

There's no localization utils, and no localization formatting. I'd be very open to PRs for these but I don't have a need for them myself and unfortunately won't have time to implement them myself any time soon.

Depending on your needs, the 2.1.0 snapshot may already be sufficient. Let me know if it isn't (& how), but I may need assistance bringing other features over if you need them soon.

Hope that helps, cheers! :-)

robert-stuttaford commented 10 years ago

It certainly does help! We'll give the cljs stuff a spin.

What does the library do about getting the dictionary into the client? Is it compiled in or is it meant to be loaded at runtime?

ptaoussanis commented 10 years ago

What does the library do about getting the dictionary into the client? Is it compiled in or is it meant to be loaded at runtime?

It's being compiled directly into the cljs file during compile-time. This gives a number of nice advantages and one disadvantage: it's a pain during development since changes to the dictionary tend not to reflect immediately or easily.

It wasn't a major problem for me so I didn't spend much time looking into it, but I think it should be possible to tweak so that tools like cljs-build can more easily identify dictionary changes.

Feedback would be very welcome btw.

robert-stuttaford commented 10 years ago

Ok. I'll let you know how we progress :-)

ptaoussanis commented 10 years ago

Just a quick heads-up that the latest snapshot makes dictionary changes more comfortable.

You'll have something like this in your Cljs:

(ns foo (:require [cljs.taoensso.tower :as tower :include-macros true]))
;; This exposes tower/t (a fn), tower/dict-compile (a macro), and tower/with-tscope (a macro)

(def ^:private tconfig
  {:fallback-locale :en
   ;; Inlined (macro) dict => this ns needs rebuild for dict changes to reflect:
   :compiled-dictionary (tower/dict-compile "my-dict.clj")
   :log-missing-translation-fn
   (when utils/dev-mode?
     (fn [{:keys [locale ks scope] :as args}]
       (warnf "Missing translation: %s" args)))})

;; (def t (partial tower/t (:locale init-state) tconfig))
(def t (partial (tower/make-t tconfig) (:locale init-state))) ; Updated example
(comment (debugf "Tower debug: %s" (t :debug)))
(comment (debugf "Tower dict: %s" (str (:compiled-dictionary tconfig))))

Then you can call t like you would on the server. Only significant difference is that dictionary entries can't contain locale-sensitive formatting patterns. Markdown and the rest work as normal.

Note that you can share the same dictionary between the client+server.

robert-stuttaford commented 10 years ago

Awesome, we'll be digging in next week.

Off-topic: (I'm curious) what do warnf and debugf do?

ptaoussanis commented 10 years ago

Off-topic: (I'm curious) what do warnf and debugf do?

Oh, excuse me - those aren't in Tower, just some utils I use:

(defn format "Removed from cljs.core 0.0-1885, Ref. http://goo.gl/su7Xkj"                                                                                                       
  [fmt & args] (apply gstr/format fmt args))                                                                                                                                    

(defn log* [x]                                                                                                                                                                  
  (if (js* "typeof console != 'undefined'")                                                                                                                                     
    (.log js/console x) ; Nb no forcing to a string, may be js obj                                                                                                              
    (js/print x))                                                                                                                                                               
  nil)                                                                                                                                                                          

(defn log [x] (when dev-mode? (log* x)))                                                                                                                                        

(defn- fmt-level [level]                                                                                                                                                        
  ;; (case level :debug "Debug - " :info "Info - " :warn "Warn - " :error "Error - ")                                                                                           
  (case level :debug "" :info "" :warn "WARN: " :error "ERROR: "))                                                                                                              

(defn sayp [& xs]           (js/alert (str/join " " xs)))                                                                                                                       
(defn sayf [fmt & xs]       (js/alert (apply format fmt xs)))                                                                                                                   
(defn logp [level & xs]     (log (str/join " " (cons (fmt-level level) xs))))                                                                                                   
(defn logf [level fmt & xs] (log (str (fmt-level level) (apply format fmt xs))))                                                                                                

(def debugf (partial logf :debug))                                                                                                                                              
(def infof  (partial logf :info))                                                                                                                                               
(def warnf  (partial logf :warn))                                                                                                                                               
(def errorf (partial logf :error))                                                                                                                                              

(set-print-fn! log)

robert-stuttaford commented 10 years ago

Aha :-) Nice! Thanks!

ptaoussanis commented 10 years ago

Okay, just summarizing the current state of things for anyone that might be tracking:

ClojureScript support has moved to the dev branch.
The dev branch CHANGELOG is available here and will be kept up-to-date.
2.1.0-SNAPSHOT is currently available via Clojars and tracks the dev branch.
It's useable (I'm using it), but the API should still be considered unstable and subject to change. Thankfully the API is quite small, so any changes can hopefully be kept relatively painless.
There's also an up-to-date example in the dev branch's README at the end of the linked-to section.
ETA: I'd like to get a 2.1.0-alpha1 out ~by the middle of March.

Any other questions, please feel free to ping here. Cheers :-)

ptaoussanis commented 10 years ago

Hi @robert-stuttaford,

Would like to cut a v2.1.0 release soon that officially includes ClojureScript support as currently implemented. I've been using the 2.1.0-SNAPSHOT in development for a few weeks and am happy with it myself but wanted to get your feedback too if you've been using it? Any problems?

robert-stuttaford commented 10 years ago

@ptaoussanis We haven't quite gotten underway with it yet, but the spiking we've done has gone swimmingly. From our side, you're clear to cut a release :-)

ptaoussanis commented 10 years ago

Great, thanks - will likely do later today. Cheers :-)

holyjak commented 10 years ago

I see 2.1.0-RC1 is out. Just what I needed, thank you for the good job! <3

ptaoussanis commented 10 years ago

Closing, will be releasing v2.1.0 final shortly.

taoensso / tower

ClojureScript support #31