Collaborate with other libre font editor web app projects on common libraries

davelab6 commented 10 years ago

Wow! Massive congratulations on the kickstarter - I'm sure you'll go way beyond your modest target! :)

There are a few other font tool web apps that I'd like to suggest collaborating with:

Typism (by @xxyxyz) is a traditional outline font editor for SVG fonts. It exports SVG fonts to be compiled offline, the same as Prototypo.

Glyphr Studio (by @mattlag) is a traditional outline font editor for OpenType-CFF. It exports TTX xml to be compiled offline.

Metapolator (by @simonegli and many others) is a parametric font editor that applies 'convention over configuration' to hijack metafont; it can do interpolation of masters made with outline editors (so-called 'superpolation') and it can derive new masters from 1+ existing ones, using 'off the peg' metafont programs ('metapolation'.) It is a reboot of the Metaflop project, which has a client-side UI of slides that set parameters of true (hand-coded) metafonts, and a server-side system to run metapost on the metafont with the given parameters and return an OTF as a web font. The first prototype of metapolator reused this design. The new prototype is 100% client-side, using NodeBox's opentype.js for superpolation and a metapost.js (made with Emscripten) for metapolation.

Font Bakery (maintained by @hash3g) and now is a continuous integration server and web interface for UFO and TTX format fonts stored in git. It clones a repo, locates font files to build, runs tests on UFO and TTX source fonts, compiles them with FontForge and fontTools (and ttfautohint) and runs more tests on the font binaries. It provides a utilitarian UI to proven server-side font tools, and Bootstrap is more than enough. However, a fonts dashboard to droni.io might be better, as that CI server has good momentum and would allow Font Bakery to focus on the things that make it unique.

Impallari Testing Tool by @impallari is a document-driven typeface test app, to test the design of a type. By improving this and making the tests encapsulated, I hope it can be embedded in other tools - all those I just listed, and prototypo.

I think there is a lot of value in the fundamentally different approaches each project takes, and I do not propose merging them. Yet I see 3 clear common areas that all projects will benefit from, by pooling our limited resources on our common goals to maximize our impact :)

Git and Github repo reading and writing

Its great for studios of designers to store known-good versions of their projects in git, even if they aren't using for every little revision of every file.

The Github REST API is powerful enough that it can serve as a 'file system' to web apps, and http://prose.io is a great example.

Font Bakery already uses git to structure its input of UFO/TTX font files, and it would be great if all font editor web apps also had this option.

Once a web app has access to a binary font file, of course, then its time for....

Client Side SFNT Reading

opentype.js by Frederik @fdb De Bleser, is seeing active community contributions and core development thanks to Prototypo :) So this currently seems to be the leading candidate, and both prototypo and metapolator use this.

ttf.js At the Libre Graphics Meeting, I met in person with Denis @moyogo Jacquerye (of DejaVu fonts fame, now a software engineer at Dalton Maag) who is working on the development branch of ttf.js, which is very similar to opentype.js, written in coffeescript. It would be great to not duplicate this effort.

ufoJS Also at LGM was Lasse @graphicore Fister, who worked on ufoJS a while ago, and will be much more active on this in the coming months as part of metapolator. Perhaps this code can partner with another.

binary-parser Mike @Pomax Kamermans worked on this a while back, and many other fonts-in-browsers experimental utilities, and of course was involved in pdf.js. The spec file approach is neat, and follows Brandon @Benvie Benvie (also at Mozilla)'s approach from 2 years ago in reified ttf example

opentype Bram @bramstein Stein works in Adobe Typekit and made this, which already (I think) has more OTL table support than ttf.js or opentype.js. Again, it would be great not to duplicate this effort.

fontTools In theory, @behdad's fontTools could be transpiled to C and then transpiled to JS. This brings us to the next item...

Client Side SFNT Writing

Louis-Rémi emailed me a while back about funding opentype.js to add font binary generation, and I've since emailed a bit with Frederik about it. But neither thread reached a clear conclusion...

I know Glyphr Studio, Prototypo and Metapolator would love to have this. The immediate option is to do it server-side, using fontTools or FontForge.

Glyphr offers TTX files as a download format, and requires users to run fontTool's ttx offline to build binary files.

If we make a fontTools.js, then Glyphr could immediately provide binaries 100% client-side.

All other font editor web apps would then merely need to export TTX files in the same way.

It would also be possible to put FontForge through Emscripten, although a fontforge.js would be heavier than a fonttools.js. (The next stable release of fontforge is coming soon - June?)

Eventually @adobe will make the FDK libre, then we could also make a fdk.js

The nice thing about FontForge is that is has so many features. The pity is that its UI is from the 90s (or earlier :) Which leads me to...

Font Editor UI Toolkit

Both metapolator and prototypo use AngularJS as an MVC framework. Glyphr Studio uses its own.

This month Alex @Troush has been shedding Metapolator's python parts and taking it full client, starting with a directive for an opentype.js interpolator using paperjs and-or two.js for <canvas> rendering. He has also set up bower and gulp (instead of grunt) to structure the setup process.

Angular promises the possibility of making a 'font editor web app toolkit'.

I would like to suggest we share as much as possible.

mattlag commented 10 years ago

Firstly, I would like to second the whole-hearted congratulations on the Prototypo effort. Maybe it was just coincidence, but I feel like there is a great font/web movement afoot, and Prototypo's success is an indicator of such.

Secondly, I would like to cast an internet vote for your second item, Client Side SFNT Writing... and just the 'we need to collaborate' message on whole. From my perspective, it's the last step in an end-to-end font design workflow for client side that is the least-well met today.

I started to integrate TTX support into Glyphr Studio out of necessity, due to my lack of binary file writing knowledge. It's a solid goal of Glyphr Studio to hide a majority of this complexity, and ultimately just client-side serve up a real font file to our users. Even with this in mind, I think investing in something like fontTools.js and the TTX XML file format would be fantastic to the community at large, so this is what I would like to advocate (for what it's worth).

And really, if I'm able to get some sort of font JSON format to TTX XML, it shouldn't be that hard to modularize or make generic that process, and apply it to something, like, opentype.js. Binary file writing is kind of my cliff at the moment, but I'd be happy to contribute some time to "opentype.js export to TTX" - iff there was some sort of last step like fontTools.js to complete the journey and export a font.

Or, you know, I'm relatively young in this field, so if there's some other obvious way I or Glyphr can help out, let me know.

Let's get this party started ,\,,/

bramstein commented 10 years ago

Great project. I agree with @davelab6 and @mattlag, there is no need to duplicate efforts on font reading and writing. I'd be happy to contribute code, or merge opentype into another project, or otherwise help out.

davelab6 commented 10 years ago

@bramstein Are you coming to TypoBerlin or the automated type design event at anrt in Nancy? Greg and Ivan said they will be in Berlin, could be good if you can't make Nancy

bramstein commented 10 years ago

@davelab6 I'm coming to Typo Berlin, let's meet up then. Didn't know about the automated type design event, looks interesting.

louisremi commented 10 years ago

Thank you guys!

And thank you Dave for organizing this discussion and keeping everyone synchronized to collaborate efficiently on reusable component to use and produce fonts on the web :-)

A little bit of history We've been thinking about client-side binary font export since we started working on "Prototypo on the web". At that time I was aware of the svg-opentype-workshop built by roc of Mozilla, but it's very incomplete and "SVG in OpenType" only. I didn't feel like diving into the OpenType spec, so when I discovered opentype.js, I approached Frederik to see if it would be possible to outsource that task to him. It seemed like the closest thing we had to achieve client-side binary export, and I liked the fact that it was a new tool built specifically to be used in the browser, and written in a language I'm familiar with. Frederik said he was interested in implementing export into opentype.js for his NodeBox project and we agreed to give him a part of our funding, if it succeeded.

Our use-cases

client-side SFNT reading: one of the stretch-goals of the campaign is to allow people to import their own font into Prototypo and gradually replace vector coordinates with parameters. There's no specific performances constraints for the library that will allow us to read those binary fonts: it can be loaded dynamically when the feature is first used, and parsing the font will only happen once.
client-side SFNT writing: a parametric font authoring tool must update the glyph preview in real-time as the user interacts with the app. Currently we display glyphs using SVG paths and position them manually on the page. The problem is that the user cannot modify the preview text directly. This would mean reimplementing the "text shaping" and "editor" components of the browser and this is something extremely complex that we'd better avoid. The solution is to generate subsetted binary fonts on each frame and let the browser render that itself. So we need a library able to generate binary font files in less than 16ms, to achieve 60fps. The second use-case is to llow users to export their work to autohinted binary font-files.
font editor UI toolkit: see the first use-case of client-side SFNT above :-)

See you guys in Nancy, for those who can make it.

moyogo commented 10 years ago

Hi, I've started playing with ttf.js by @ynakajima but can work on opentype.js. A single Server or client to-and-from binary JS library would be good.

The layout of opentype and ttf.js is better at the moment, everything gets compiled into one file instead of working on a single file. A modular system would also be better. But I see there’s an issue for that on opentype.js.

For GPOS, GSUB, BASE, JSTF and MATH tables parsers, the approach of binary-parser or fontTools is better.

davelab6 commented 10 years ago

The binary-parser approach seems wisest to me. What do the other projects do that binary-parser does not?

moyogo commented 10 years ago

A-binary-parser-generator’s OpenType.spec has an incomplete GSUB support, and is missing BASE, GDEF, GPOS, glyf or CFF as well as other tables.

Pomax commented 10 years ago

@moyogo aye, but mostly because I've not had time to work on it more since stopping there. I'm looking at revisiting how it does its job so it can convert specs to a cleaner code, ideally generating something like the struct based source in https://github.com/pomax/CFF-glyphlet-fonts, but I'm also working on a lot of other things so I have no idea how soon or far in the future that's going to be done.

davelab6 commented 10 years ago

Since fontTools has full support for everything, I think the first thing will be to get fonttools.js running through Emscripten, then look at the performance, because if we adapt it to input/output JSON instead of XML, and the performance is good, maybe we are done.

louisremi commented 10 years ago

Yep

On 16 April 2014 23:15, Dave Crossland notifications@github.com wrote:

I think the first thing will be to get fonttools.js running through Emscripten, then look at the performance, because if we adapt it to output JSON instead of XML, and the performance is good, maybe we are done. On 16 Apr 2014 15:30, "Denis Moyogo Jacquerye" notifications@github.com wrote:

Hi, I've started playing with ttf.js by @ynakajima< https://github.com/ynakajima>but can work on opentype.js. A single Server or client to-and-from binary JS library would be good.

The layout of opentype https://github.com/bramstein/opentype and ttf.jshttps://github.com/ynakajima/ttf.jsis better at the moment, everything gets compiled into one file instead of working on a single file. A modular system would also be better. But I see there’s an issue for that on opentype.js.

For GPOS, GSUB, BASE, JSTF and MATH tables parsers, the approach of binary-parser or fontTools is better.

— Reply to this email directly or view it on GitHub< https://github.com/byte-foundry/prototypo/issues/115#issuecomment-40582398>

.

— Reply to this email directly or view it on GitHubhttps://github.com/byte-foundry/prototypo/issues/115#issuecomment-40653214 .

Pomax commented 10 years ago

:+1:

davelab6 commented 10 years ago

@troush will get into this next week, as he is working on mpost.js first with some help from @max99x :)

davelab6 commented 10 years ago

http://www.rfk.id.au/blog/entry/pypy-js-first-steps/ and http://www.rfk.id.au/blog/entry/pypy-js-poc-jit/ seem very promising :)

davelab6 commented 10 years ago

@behdad is tracking JSON for TTX at https://github.com/behdad/fonttools/issues/117 and estimates that this will be 10x slower than native Python going the repl.it route, so using PythonJS may be better (or writing the JS by hand...)

davelab6 commented 10 years ago

Quick update... mpost.js proved tricky for Alex, so its now postponed to probably July while @troush and @graphicore work on ufoJS - details on the mp roadmap.

I think ufoJS can help with this prototypo ks stretch goal,

implement and editor allowing to turn glyphs of an existing font to parametric glyphs, or draw a new glyph and make it parametric, all of this right into Prototypo

as it will soon allow users to upload UFOs into a browser and access the fonts using RoboFab style penProtocol methods, and thus easily parse them in to your parametric language.

louisremi commented 10 years ago

Hi Dave,

Thank you for the update. Could you or Alex provide more info on what difficulty were encountered with metapost.js? Was that perf related or somethin else?

ufo.js will definitely be useful for interoperability, I'm glad you guys are working on that : )

davelab6 commented 10 years ago

Getting it work at all. I'll have a engineer with a lot of Asm/C/C++ experience look at it :)

davelab6 commented 10 years ago

Another quick update :) ufoJS is now accepting ufo.zip files and Alex posted a simple glyph inspector here, http://beta.metapolator.com/glyph-inspector/

I've also started https://github.com/fontforge/fontforge/issues/1497 for a simple python httpd to provide a REST API for compiling UFO and TTX font sources. That seems like maybe the next point of collaboration?

louisremi commented 10 years ago

Excellent, I'll look into ufoJS and your fontforge issue.

graphicore commented 10 years ago

Actually ufo.zip files are not included in ufoJS as it is not in here: https://github.com/graphicore/ufoJS However, http://beta.metapolator.com/glyph-inspector/ uses ufoJS. should it be part of ufoJS? By now ufoJS is not so deep into specializing for Browsers/the Web API, but I could add a WebAPI submodule.

davelab6 commented 10 years ago

I suggest @troush submit a pull request for https://github.com/graphicore/ufoJS to support ufo.zip :)

davelab6 commented 10 years ago

Another update. Metapolator abandoned metafont directly, so no progress on that.

OpenType.Js has some form of export now. If it is insufficient, I think we will get Fontforge set up as a web service to compile UFO to TTF. However, with the FDK now libre, maybe we can Emscripten that as another option.

Did you look at ufoJS?

fdb commented 10 years ago

Indeed, I'm happy to announce that opentype.js has write support now! It's still early, but my tests ( reading in an existing font and writing it back out) show that the library produces good results that work on multiple platforms and in the browser.

We only support CFF outlines, I'm not sure if I'll ever implement TrueType outlines.

I'm still working on getting the API right to allow control over all parameters without confusing new users. I'm more than happy to work with the community to support additional use cases that I haven't covered.

Oh, and big shout-out to @Pomax for his CFF-glyphlet-fonts project, that was a huge help in getting this done.

mattlag commented 10 years ago

@fdb :+1: :+1: This is very exciting, nice work!

Pomax commented 10 years ago

TTF outlines are pretty easy though, as low hanging fruit probably worth putting it in for ttf to ttf round trips. Unless you're generating the CFF with subroutine optimisations, you're probably making the font bigger than it original was by upcasting all the outlines to cubic curves. If you keep a record of whether the outline origins were cff or ttf, generating the ttf outlines is incredibly straight forward.

behdad commented 10 years ago

Yes, they are much easier!

davelab6 commented 9 years ago

http://byte-foundry.github.io/plumin.js/ was just released, and is powering the latest Prototypo - open the page with Chrome, uncomment the final piece of code, click run, and you see fonts generated 30+ times a second to give the impression of smooth animation of the font :)

Plumin integrates @fdb's OpenType.js to do the generation from its new JS font object model which is RoboFab inspired in structure, and (like Metapolator) is skeleton inspired by Metafont.

louisremi commented 9 years ago

Ok, we probably tried to put too much in the presentation and ended up confusing the audience : ) So, to clear things up: plumin.js is just a combination of the paper.js API/scene-graph and opentype.js font creation capabilities.

It adds a font/glyph oriented wrapper API on top of paper.js' API, as well as some methods to allow the font to be used instantly in the page or downloaded as .otf. Achieving 30fps is only really possible in browsers implementing the css font-loading API: currently Chrome has it, and Firefox nightly as well, but behind the layout.css.font-loading-api.enabled pref (I'm also about to report a bug related to the .delete method of the API). In Safari, the font switching currently produces an annoying flash, but I haven't had time to try to tackle the problem. And I haven't tried in IE.

The skeleton approach that we're adopting in Prototypo is about to be reimplemented on top of Plumin. The next steps for Plumin is to allow "ufo as json" fonts to be loaded (using UFO.js?) and to create a proper site and documentation.

We rushed things a little to introduce new stuff at Typomad, so some of it isn't ready for prime time yet. Let us know if you have additional questions. : )

graphicore commented 9 years ago

Who is outputting "ufo as json"?

davelab6 commented 9 years ago

The next steps for Plumin is to allow "ufo as json" fonts to be loaded (using UFO.js?)

Who is outputting "ufo as json"?

No one :) @louisremi means he wants to load UFO into JS objects... which ufoJS does (right? :)

louisremi commented 9 years ago

Yep, that's the idea. Specifically we'd like to avoid doing the parsing/conversion in the browser, but rather have some kind of task that can do the conversion to js(on) Le 15 nov. 2014 19:03, "Dave Crossland" notifications@github.com a écrit :

The next steps for Plumin is to allow "ufo as json" fonts to be loaded (using UFO.js?)

Who is outputting "ufo as json"?

No one :) @louisremi https://github.com/louisremi means he wants to load UFO into JS objects... which ufoJS does (right? :)

— Reply to this email directly or view it on GitHub https://github.com/byte-foundry/prototypo/issues/115#issuecomment-63181537 .

graphicore commented 9 years ago

to load UFO into JS objects... which ufoJS does (right? :)

yes, at least it's very easy to do and we do it frequently. You can also initialize your object model directly using the pen protocol.

we'd like to avoid doing the parsing/conversion in the browser

Even without loading the the data as xml you would certainly want to validate it in the browser, when it's coming from an untrusted source (like a user upload). Therefore we'd need to change ufojs a bit. Also, we don't validate data above glyph and glyphset level yet.

louisremi commented 9 years ago

Whether the font can be trusted or not is not plumin's concern. What is really important is that we have a json format to exchange fonts between apps. It could be ufo converted to json, or something else. Le 15 nov. 2014 19:43, "Lasse Fister" notifications@github.com a écrit :

to load UFO into JS objects... which ufoJS does (right? :) yes, at least it's very easy to do and we do it frequently. You can also initialize your object model directly using the pen protocol.

we'd like to avoid doing the parsing/conversion in the browser Even without loading the the data as xml you would certainly want to validate it in the browser, when it's coming from an untrusted source (lika a user upload). Therefore we'd need to change ufojs a bit. Also, we don't validate data above glyph and glyphset level yet.

— Reply to this email directly or view it on GitHub https://github.com/byte-foundry/prototypo/issues/115#issuecomment-63183148 .

graphicore commented 9 years ago

Whether the font can be trusted or not is not plumin's concern.

ufoJS does validation it must not be the concern of plumin. It's not only about trust, it's much more about data integrity, e.g. after you opened a glyph using ufoJS (or robofab) you can be sure that the outline etc. has a valid format. You shouldn't need further checks.

Is XML an unpopular ground for web based development, because it is in a way harder to handle than JSON? I would myself criticize the way how UFO uses XML e.g. it's not using proper XML tools like XML-Schema (XSD) or XML-Namespaces. But there are great benefits of one (and only one) format for interchange. Since UFO-XML is already there and widely used to exchange fonts between apps, I think it makes a much better interchange format than anything else.

Why is it so important for you to use JSON as serialization format? It is important to have a good reasoning behind such a "bold" decision, because then people will will be able understand why you did it and can follow your move.

That being said, it is entirely possible to create a JSON serialization of UFO using ufoJS. It would require some work (more at the side of reading the JSON back in) and it would of course have to obey the specs at http://unifiedfontobject.org/ because otherwise it's pretty much nonsense. However, at that point the difference between a JSON serialization and a XML serialization is not very big anymore.

We should analyze what your needs/the needs of apps in the future are/might be (i.e. make a wishlist) and what can be done best. I for example would prefer YAML over JSON, only because YAML has proper support for comments, which JSON lacks. So maybe the way to go is to write a proper serialization api description and let it read and write to any format you want? Another nice thing I can imagine would be a format that stores the entire UFO in one single file instead of a directory, because that would make it really easier to send it to other apps.

louisremi commented 9 years ago

Sorry for not getting back to you earlier Lasse. Being compatible with UFO is a mid-term goal for us. Right now all we need is a file where we can store info about the font, the parameters it uses and the glyphs. Our short-term solution is a JSON file that has a structure as close as possible to a UFOv3 archive. I've looked at ufoJS, and it seems that we might be able to use it to produce ufo-structured JSON files.

davelab6 commented 9 years ago

I thought I'd update this issue

opentype.js

This has been actively developed by a vibrant developer community, and supports basic SFNT writing - which the prototypo team have combined with paperjs to produce www.pluminjs.com :)

ufoJS

This has been actively developed by its maintainer, and support UFOv3

adobe will make the FDK libre

They did! https://github.com/adobe-type-tools/afdko

I've looked at ufoJS, and it seems that we might be able to use it to produce ufo-structured JSON files.

In the short term, there is https://github.com/byte-foundry/unzip-ufo :)

Also something not captured on this thread is the need for OpenType Layout processing in JavaScript. The @prezi company have done something similar to what is described at the top, and ported Harfbuzz to JS with emscripten!

See a prezi about this at https://prezi.com/zybglpjalpkl/conquering-multi-platform-text-rendering-wwx2013/ and video of the presentation at https://www.youtube.com/watch?v=yvQpt6hCzU8

davelab6 commented 9 years ago

There's now also https://github.com/devongovett/fontkit

davelab6 commented 9 years ago

@fdb I wonder if you could comment on what new support you think opentype.js will ship over the next 6 months. Everyone else, what area of OpenType standard do you think improvement is most needed in?

mattlag commented 9 years ago

@fdb @davelab6 at least for Glyphr Studio, it would be great to have:

Kern writing support (GPOS)
Table writing (font.tables.name, font.tables.os2, etc)
Ligature writing support (GSUB)

In that order :-)

louisremi commented 9 years ago

@fdb @davelab6 What @mattlag said : ) We are willing to sponsor some of that work, once our financial situation improves.

fdb commented 9 years ago

@davelab6 @mattlag @louisremi I have some time in July-August where I would love to have more robust support for font writing. I think it would be cool to have deep control over what gets written in the tables. With that as a foundation, you would already be able to use it for kerning and ligatures, although it would be pretty low-level.

On top of that would be more "user-facing" support for things like kerning and ligatures, like I have now with glyphs. That is: you provide the info, and OpenType.js makes sure everything gets written to the correct table(s).

I wouldn't mind at all for this work to be sponsored. I think the collaboration with @louisremi last year, where we agreed on a work package and compensation, worked out really well for everyone.

davelab6 commented 9 years ago

Do you want to continue the discussion publicly or privately?

fdb commented 9 years ago

@davelab6 I don't mind discussion feature scope here, or as issues/milestones in opentype.js. Talk about sponsored development, I'd prefer we discuss privately (email is at https://github.com/fdb).

fdb commented 9 years ago

Okay I've added a "Font Writhing" milestone to OpenType.js that covers the issues mentioned by @mattlag. You're free to add comments / suggestions there.

Pomax commented 7 years ago

This issue is collecting quite a bit of dust in my github.com/issues/mentioned list because it's been open for 2 years since the last comment: is it possible to close this and capture the information in it on a repo wiki page instead?

davelab6 commented 7 years ago

Sure, no need to capture it elsewhere for me.

Since the last comment, a few summary thoughts:

https://www.google.com/search?q=opentype+variable+fonts happened - which is really great :)
@devongovett has done an amazing job with https://github.com/devongovett/fontkit (eg http://fontkit-demo.now.sh using it as a polyfill for variable fonts) and it seems to be leading the other js sfnt libraries discussed above
https://github.com/byte-foundry/prototypo/ has been continuously developed, although not as a typical libre software project
@metapolator went very far but not far enough to be really useful to end users as prototypo now is, although I expect @graphicore will continue to push it forwards
glyphr studio is undergoing a complete rewrite, which as a hobby project is fine per https://www.joelonsoftware.com/2000/04/06/things-you-should-never-do-part-i/ ;)
@adrientetar launched https://github.com/trufont/trufont which is a Python 3 font editor platform that uses many of the foundation libraries of RoboFont, but with PyQt instead of Apple's Cocoa library

byte-foundry / prototypo