Tibetan Font Rendering on Android, iPad, and iPhone 4

June 14th, 2010 § 3

Thanks to Chris Fynn and the Dzongkha Development Commission, there is now a Tibetan font that can display correctly on Android, iPad, and iPhone 4, with some conversion and munging of the Unicode data.

In a previous post, I described how to render Tibetan on a rooted Android phone, through replacing the built-in fallback font with one (Jomolhari ) that supported the Chinese Tibetan Unicode standard GB/T20524-2006, which resides in the 0xF300 to 0xF8FF space.  This technique had a severe limitation, in that it wouldn’t work on an unrooted phone, and also removed support for Japanese, Chinese, and Korean (quite important languages in their own right).

I mentioned in that previous post that it would be possible to render Tibetan without having to root your Android phone, if you could use the technique described here, which embeds the fonts in the application and sets it on TextView objects. Unfortunately, it didn’t work with Jomolhari when I tried it, showing lots of beautiful boxes instead of Tibetan letters.  Evidently, some fonts just “don’t work” with this technique, and it was unclear why that happened; additionally this technique would have required a 2.3MB font to be contained in every Tibetan-supporting Android app.  It was large because this font was optimized for printing, so was specified in a lot more detail than cell phone screens needed.

A while back, Chris Fynn sent me his new font, “DDC Uchen,” which was designed to be used for web sites, and is only 860k, distributed under the Open Font License.   This font and Jomolhari are the only two fonts I’m aware of that have support for the Precomposed Tibetan standard.  I’ve placed it for download here.  (The DDC stands for Dzongkha Development Commission, which paid for Chris Fynn to develop the font, and is making it freely available to everyone. Dzongkha is the language spoken in the country of Bhutan.  It is very closely related to Tibetan and uses an identical alphabet.)  Just for grins, I thought I’d try it out by placing the font in my Android project under assets/fonts/DDC_Uchen.ttf, with this fairly simple code:

        TextView tv=(TextView)findViewById(R.id.output);
        Typeface face=Typeface.createFromAsset(getAssets(), "fonts/DDC_Uchen.ttf");
        // Precomposed Tibetan version of a supplication to Tara
        String tibetanText = "\u0F04\u0F05\u0F0D \u0F0D\u0F60\u0F55\u0F42\u0F66\u0F0B\u0F58\u0F0B" +
        "\uF397\u0F63\u0F0B\u0F58\u0F0B\u0F63\u0F0B\u0F56\uF43A\u0F51\u0F0B\u0F54\u0F0B\n" +
        "\uF4BA\u0F42\u0F0B\u0F60\u0F5A\u0F63\u0F0B\u0F56\u0F0B" +
        "\uF3ED\u0F0B\uF5BE\u0F0B\uF53F\u0F0B\u0F42\uF3B7\u0F42\u0F0B\u0F54" +
        "\uF591\u0F0B\uF5AA\u0F58\u0F0B\u0F54\u0F0B\uF473\u0F0D\u0F0D";
        tv.setText(tibetanText);
        tv.setTypeface(face);

It rendered perfectly on my Android phone:

android_tibetan_cropped

To convert text from Tibetan Unicode, you either have to use the libraries I’ve uploaded at the GyutrulDrawa project, hosted on Google Code, or use Andrew West’s BabelPad (Windows only).  To convert it using BabelPad, use the menu option Convert -> Tibetan -> Unicode to Precomposed Tibetan (Set A).  For these program examples, to paste it into Java code, I then used “Convert to Universal Character Name”, which generated Hex codes in \u format, and pasted it into the Android Java code.

This technique will only work with widgets which are derived from TextView, which means that this can’t be used to easily create a Tibetan Web browser.  However, other widgets such as EditText, Button, CheckedTextView, CheckBox, CompoundButton, RadioButton, and ToggleButton, as well as Chronometer, DigitalClock, AutoCompleteTextView, ExtractEditText, and MultiAutoCompleteTextView, all inherit from TextView, and so should work with this technique (though I haven’t tried it out with them).

Getting it to work on iPad 3.2 and iPhone 4

Now that we have working Tibetan font support on Android, what about the Other Phone, the iPhone?  Well, it turns out that 3G and earlier iPhones don’t support setting embedded fonts easily, but 3.2 for the iPad supports it, as described in this post.  As on Android, the iPhone doesn’t have the complex script support necessary to render Tibetan properly (in fact, the default rendering mode looks worse than on Android), but the exact same technique we used on Android works on iOS as well.

I’d worked through some basic tutorials on iPhone development, but this was my first excursion into writing my own program in Objective C, so it took me a while to get everything set right.  There are a few steps to this:

  1. Put the DDC_Uchen.ttf font in the Resources directory of your Xcode project.
  2. In your projects plist file, you have to add a Key named “UIAppFonts”, change the Value Type to Array, and then set the first (only) value to “DDC_Uchen.ttf”.
  3. Make sure you refer to the font by its actual name, not the file name, in this case “DDC Uchen,” when you use it.

Here’s some code to get it to work:

	textView = [ [ UITextView alloc] initWithFrame: rect ];
	UIFont* tibFont = [UIFont fontWithName: @"DDC Uchen" size: 22];

	if (tibFont != nil) {

		[ textView setFont:tibFont];

		textView.text = @"\u0F04\u0F05\u0F0D \u0F0D\u0F60\u0F55\u0F42\u0F66\u0F0B\u0F58\u0F0B\uF397\u0F63\u0F0B\u0F58\u0F0B\u0F63\u0F0B\u0F56\uF43A\u0F51\u0F0B\u0F54\u0F0B\n\uF4BA\u0F42\u0F0B\u0F60\u0F5A\u0F63\u0F0B\u0F56\u0F0B\uF3ED\u0F0B\uF5BE\u0F0B\uF53F\u0F0B\u0F42\uF3B7\u0F42\u0F0B\u0F54\uF591\u0F0B\uF5AA\u0F58\u0F0B\u0F54\u0F0B\uF473\u0F0D\u0F0D";

	} else {

		textView.text = @"Failed.";

	}

This results in the following screen shot, which I took from the 3.2 simulator:

screen-shot-2010-06-13-at-43756-pm

This will definitely not work on any iPhone 3GS or earlier, but I presume that it will work with the iPhone 4 (though I haven’t tested it).

The conversion routines which Andrew West offered to this project are written in C++ (here and here).  I haven’t yet ported them to Objective C, so if anyone is inspired to do this, it should allow for dynamic display of any Unicode Tibetan text by converting it into Precomposed Tibetan.  Otherwise, you’ll have to use the Windows-based BabelPad to convert each of your strings into Precomposed Tibetan, or use the C++ code and convert the strings between CString and NSString.

The next steps will be to firm this code up.  There are a few Tibetan projects (TBRC, the Trace Foundation, and Sonam Rinchen‘s iPhone apps) that are looking to use this code, so it would be best if everyone involved can keep this in the Open Source spirit by building on this bare foundation to create a common display, text entry, and reading environment for Mobile Tibetan.  Enjoy.  Gelek Pel and  Sarva Mangalam!

New Tibetan Phone Project on Google Code

January 29th, 2010 Comments Off

I just started an open-source project, “Gyutrul Drawa” for a Tibetan phone environment, seeding it with the code to render Tibetan properly on Android, at:

https://code.google.com/p/gyutruldrawa/

gyutrul-drawa

“Gyutrul Drawa” is Tibetan for the “Magical Net,” which is one of the core texts of the Nyingma tradition, translated into Tibetan in the 8th century from Indian sources.  Since our own modern Magical Net has appeared in the last 20 or so years, it felt appropriate to bring the name forward and use it again.

It doesn’t do much yet, but at least it renders a Tibetan web page properly in Android once you’ve unlocked your phone and installed the Jomolhari font, following the instructions at Nathan Freitas’s blog. (Note that the Tibetan Machine Uni font he suggests installing will NOT work with this, you must use Jomolhari.)

Future plans are to write a Tibetan landing page, improve the browser, and add the following applications:

  • Connections to the various IM systems that support Unicode
  • E-mail client
  • Tibetan-English dictionary
  • PDF reader with Tibetan-language buttons (to support reading scanned texts such as those from TBRC)

It may be possible to perform a similar technique to render Tibetan properly on the iPhone (and iPod Touch and iPad, of course).  I hope to test that out when I get a chance.

Feel free to contact me if you’d like to help out with this project.

Proper Rendering of Tibetan in Android

January 18th, 2010 § 12

Summary: I managed to get Tibetan script rendering accurately on Android, even without Complex Script support, thanks to a Tibetan Unicode font with support for the GB/T20524-2006 standard.

Since I’ve been studying Tibetan, and have started programming on Android, I thought I’d try out Nathan Freitas’ instructions on enabling Tibetan font support in Android.  I’m not going to duplicate his instructions here, but they involve rooting your phone and installing a Tibetan Unicode font to replace the built-in DroidSansFallback font, used to display Chinese, Japanese, and Korean.

However, once I got it working, I realized that, though the characters were indeed Tibetan, they weren’t properly laid out.  Tibetan is a fairly complex language to render properly, since it allows for quite complex vertical stacks of characters with idiosyncratic joining rules.  Here is an example, taken from a supplication to Tara, showing how it was originally rendered on my cell phone:

badtibetansample

OM JE TSUN MA PAK MA DROL MA (Om, the Revered Lady Noble Tara)

If you don’t know Tibetan, that could look decent (and certainly mysteriously beautiful), so let me show the end result, how it’s supposed to look:

goodtibetansample

OM JE TSUN MA PAK MA DROL MA (Om, the Revered Lady Noble Tara)

Note that the letters don’t overlap anymore, and some of them have changed shape to accommodate their companions.  For example, the first syllable, an OM, has had its accents adjusted so they don’t overlap, and the shape of the NA-RO, the seagull-like accent, has been changed to anchor it off of the middle line.  The second syllable has more drastic changes, where the RA is simplified to blend in with the JA below it properly.  In the third syllable, the SHAB-KYU, the hook accent underneath, has been moved up snug to its letter.  The last major change is toward the end, where a RA subscript has been moved down to join up properly with the bottom of its stack.

Though the individual glyphs were being rendered perfectly, they were not being combined properly.  It turns out the need for better language support is a long-standing complaint about Android, and is in the Issues list as #5925.   Comment 15 is a very good description by an unidentified Google engineer of the core problems with Complex Script Support.  (If you’d like to encourage Android to support this properly, please add a comment to that issue.)  The main problem is that, though FreeType (used in Android for font rendering) renders fonts perfectly, there is no support for the ligatures necessary to render complex writing systems such as Arabic, Thai, Devanagiri, and Tibetan.

Chris Fynn is one of the main people who’s been working on Tibetan Unicode standards, and his font, Jomolhari, is my favorite Tibetan font (you can see it in the above pictures, and it’s available under the Open Font License).  He helped with Tibetan rendering in Pango, HarfBuzz, and ICU, any of which would render Tibetan properly, but none of these have yet been integrated into Android.  He lives in Bhutan, and I met him when I visited there.

Chris mentioned that there is a less-well-known Tibetan text encoding named GB/T20524-2006, which resides in the 0xF300 to 0xF8FF space. The “right way” to handle Tibetan has always been to combine glyphs on the fly.  However, since the Chinese are used to having a separate encoding of each unique combination of characters, they created a Chinese National Standard using the Unicode Private Use Area (PUA) to encode each combination of Tibetan glyphs as a single precomposed ligature.  Chris implemented support for this in Jomolhari partly in an effort to get Tibetan fonts working properly in some products (such as Adobe InDesign and Photoshop) that didn’t fully support Tibetan at the time.

He suggested using Andrew West’s BabelPad, which is a Unicode text editor that also supports all sorts of interesting conversions (including rewriting various Tibetan, Mongolian, Manchu, and Vietnamese encodings as Unicode), to re-encode whatever I wanted to display in Tibetan.  This worked once I installed Jomolhari on the handset as the DroidSansFallback.tff font.

I realized that, using Android’s WebView and WebViewClient classes, I could build a Tibetan web browser that intercepted url requests, downloaded the HTML, and rewrote any Tibetan Unicode as precomposed characters.  I would just have to implement a massive conversion table to do it.  Fortunately, Andrew West gave me a code sample in C++ that implemented this 1500-line table, and I was able to quickly port it to Java.  (BTW, for future reference, don’t cut-and-paste large C++ code files directly into Eclipse’s java editor.  It grinds away for a long time, patiently highlighting all the syntax errors for you, before it lets you fix any of them.)  It worked the first time I got it to compile, resulting in a beautiful, properly rendered web page.

A screenshot of the Tibetan WebView app.

A screenshot of the Tibetan WebView app.

I’m planning to clean up the rough edges on the browser, and release that code soon as the seed of a Tibetan open-source Android suite.  I thought that I’d make the Java code implementing the conversion available right away, even though it could also use some optimizing & clean-up, so that others can work with it.  It’s available at:

http://tom.to/code/TibConvert.java

It’s very easy to use:

composedData = TibConvert.convertUnicodeToPrecomposedTibetan(data);

I look forward to being able to help out with other useful Tibetan apps, such as a custom keyboard, IM and e-mail apps, and a dictionary.  Please contact me if you’re interested in working together on these types of things.

I’ve really been enjoying the Android development process.   Running code and debugging code directly on the handset is working well, and I like how the libraries are structured.  This might be what it takes for me to get back into having fun with developing client-side Java.

May there be a great flourishing of Tibetan apps on the Android!  Ge-lek pel!

BTW, I spent some time trying to get Tibetan font support working on a stock Android phone, without having to root it, using the font embedding technique described here.  I couldn’t get it to work (just lots of beautiful boxes showing up instead of Tibetan), but they say that some fonts just don’t work to embed.  Of course, embedding a 2.2MB font inside an app might be pushing the envelope a bit, but it would be great if someone could get this to work.  Though it appears that the only widgets you can set the font for are TextView widgets, so you wouldn’t get a web browser that way, and you’d also have to re-implement a new widget set derived from TextView to get it to show up properly.

Where Am I?

You are currently browsing entries tagged with Android at tom at tom dot to.