Note
This post was written only for Django 0.96.1 in GAE.
Two days ago, I started to create another Google App Engine application. This application will be internationalized when it’s finished. I tried searching for some solution, then I realized that there is no very simple way to achieve.
Normally, you can handle gettext stuff on your own, but our Google App Engine applications usually use templating from the SDK, which is from Django actually. One way or another, we have to incorporate with Django partially.
The goal here is:
- Use minimal Django stuff, only import the essential stuff in order to get Django’s I18N support to work.
- Messages in template must be translated, too.
- Capable to decide the language from the cookie, django_language, or the request header, HTTP_ACCEPT_LANGUAGE.
I have already made a sample code, which you can read here and you can see it at http://yjltest.appspot.com/i18n.
Note
http://yjltest.appspot.com/i18n is gone. (2015-12-14T06:40:18Z)
Before we go into the code, please read the I18N1 and Settings2 of Django.
Contents
1 Setting Up
We need to use Django Settings to make I18N work. The reason of using Setting was due to Django’s gettext helper will require Settings module and decide location of message files by the location of Settings module.
If we want to use Django Setting, we must run the following code:
from google.appengine.ext.webapp import template os.environ['DJANGO_SETTINGS_MODULE'] = 'conf.settings' from django.conf import settings # Force Django to reload settings settings._target = None
Note that you must import the google.appengine.ext.webapp.template module, or you might get error about conf.settings is not able to be imported.
We need to set the environment variable DJANGO_SETTINGS_MODULE to the location of Setting module, conf.settings in this case. conf is the package and settings is a module file, our Settings module.
Why conf? Because when we generate message files from Python scripts and templates — we will see how to generate later, the Django message file generator, make-messages.py, will create files under conf/locale/ from where it’s run.
2 Settings
What do we need in conf/settings.py?
USE_I18N = True # Valid languages LANGUAGES = ( # 'en', 'zh_TW' match the directories in conf/locale/* ('en', _('English')), ('zh_TW', _('Chinese')), # or ('zh-tw', _('Chinese')), # But the directory must still be conf/locale/zh_TW )# This is a default languageLANGUAGE_CODE = 'en'
3 Mark the messages
Wraps those need to be translated with _("message") in Python script and {% trans "message" %} in template files. Please read I18N1 for more usages.
4 Generate message files
Before you run the helper script, we need to create conf/locale, the helper won’t create it for us.
Make sure you are at root of Google App Engine application’s directory, then run:
$ PYTHONPATH=/path/to/googleappengine/python/lib/django/ /path/to/googleappengine/python/lib/django/django/bin/make-messages.py -l en
/path/to/googleappengine/ is the Google App Engine SDK’s location. This command should generate the conf/locale/en/LC_MESSAGE/django.po. Now you can open it to translate.
Don’t forget to set CHARSET, Usually UTF-8 will be fine, the line would read like:
"Content-Type: text/plain; charset=UTF-8\n"
Once you finish translating, you need to run:
$ PYTHONPATH=/path/to/python/googleappengine/lib/django/ /path/to/googleappengine/python/lib/django/django/bin/compile-messages.py
It will generate django.mo files in each language directories. You also need to update when you modify scripts or template, run:
$ PYTHONPATH=/path/to/googleappengine/python/lib/django/ /path/to/googleappengine/python/lib/django/django/bin/make-messages.py -a
This will update all languages in conf/locale.
5 Working?
If you run your application, now it should show the language in conf.settings.LANGUAGE_CODE.
This is a per application setting, which is not normally that we want. We will expect each user can choose their own language. Django has a helper that calls LocaleMiddleware can do the job, unfortunately, it needs Django’s request and response class to work normally.
6 Do the dirty job
In order to do what LocaleMiddleware does, we need to make Google App Engine’s request/response objects have same behavior as Djagno’s. For easing the complexity, we create a new class, I18NRequestHandler, which inherits google.ext.webapp.RequestHandler. You only need to replace with it in your handlers.
import os from google.appengine.ext import webapp from django.utils import translation class I18NRequestHandler(webapp.RequestHandler): def initialize(self, request, response): webapp.RequestHandler.initialize(self, request, response) self.request.COOKIES = Cookies(self) self.request.META = os.environ self.reset_language() def reset_language(self): # Decide the language from Cookies/Headers language = translation.get_language_from_request(self.request) translation.activate(language) self.request.LANGUAGE_CODE = translation.get_language() # Set headers in response self.response.headers['Content-Language'] = translation.get_language() # translation.deactivate()
Where Cookies is from http://appengine-cookbook.appspot.com/recipe/a-simple-cookie-class/ (dead link with long gone Cookbook). When request comes in, it can automatically activate the language from what Cookies/Headers specify.
7 Caching problem
It’s not so perfect. I have noticed a problem in development server. If you change code and/or the message file, recompile the message file while server still runs, those message in entry script may not be translated for reflecting to cookie django_language‘s change. I believe that is about the caching.
I am not sure the natural problems, so I couldn’t solve it. However, this may not be severe problem.
8 Encoding
If you use unicode string (not str string) in {% blocktrans %} template tag, you may get error, encode it to utf-8 first, e.g. s.encode('utf-8').
9 Language Code
You must use underscore not dash for messages directory, e.g aa_BB, or Django would not recognize directory named as aa-BB or aa-bb. But in conf.settings you can use aa-bb, this means the language code and directory can be different, e.g. zh-tw for the language code in Python and zh_TW as message directory name.
10 Conclusion
Although this will work, but it may be broken if any changes to Django framework within Google App Engine. There isn’t a good solution for I18N in Google App Engine if Google doesn’t natively support it.
11 Updates
- 2009-11-25: Added not about template module first and encoding issue, and updated the path of Python lib in GAE SDK.
- 2009-12-24: Added a note about Language Code format, thanks BRAGA, again.
- 2010-02-04: Added a note about the Language Code and message directory name.
- 2013-02-17: fix dead links and typos.
- 2013-07-24: remove “.rst” from title, update link.
[1] | (1, 2) Django 0.96 documentation http://www.djangoproject.com/documentation/0.96/i18n/ is gone, the link is for Django 1.4. |
[2] | Django 0.96 documentation http://www.djangoproject.com/documentation/0.96/settings/ is gone, the link is for Django 1.4. |
I have read this post, and it seems to be useful. Could you compare this approach with BabelDjango ? For example, most of the time templates include information include contextual data. Some others, because of the nature of Django templates, some visual aspects in a site are hidden outside templates since they'r coded in Python scripts. Is there a way to extract all this messages at once in order to maintain a single catalog PO file? Besides I am not sure about this funny {% trans "message" %} since everything inside templates is a potential target for translations, so they could be full of {% trans "message" %} blocks. IMO Genshi templates are cleaner because of this, and its quite easy.
ReplyDeleteYes I know ... you still need to upload babel to GAE ... but I dont agree with you when you say that "There isn't a good solution for I18N in Google App Engine if Google doesn't natively support it." since you can always upload the one that better fits your particular purposes and, you know ... just use it. The only thing left is that yes ... if Google supports a better approach natively, users preserve their quotas ... ;)
I am sure that {% trans "message" %} will be recognized by make-message.py as well as _() in Python scripts. And make-message.py will put all into one .po. As of BabelDjango, I am sorry that I don't want to dig into that. Currently, I am satisfied with Django's I18N.
ReplyDeleteWhen I wrote "There isn't a good solution for I18N in Google App Engine if Google doesn't natively support it." That is simply because GAE takes some parts from different open source projects and also made some changes for GAE. For example, the template from Django. Now if you want to use nearly all support Django's I18N, you have to go through this blog posting, which includes workarounds in different level. Can you guarantee any framework's I18N support could work just out-of-box? That's why I said we need the native support from GAE, which implies that GAE has integrated its own I18N or some open source's I18N.
I compared it with Babel. The message stores are compatible though the request handler here is preferred. My app updates messages only with compile-messages and we can get common messages from the sdk and add on more to the predefined. Thanks
ReplyDeleteis it just me or this doesn't work any more with 1.2.3 version of GAE SDK?
ReplyDeleteI have this error message.
ReplyDeletewhat can i do?
Traceback (most recent call last):
File "C:\Program Files\Google\google_appengine\lib\django\django\bin\make-mess
ages.py", line 4, in < module >
from django.conf import settings
ImportError: No module named django.conf
@make, have you set PYTHONPATH environment variable? You need to add additional path which points at the directory django locates in GAE SDK.
ReplyDelete@redduck666, I don't know if it still can run on SDK (development server), but it has no problem on production server. My example still works.
Simple correction to the post:
ReplyDeleteThe file is [compile-messages.py] instead of [compile-message.py], as stated here.
Thanks for the post. It was very useful.
@BRAGA, Bruno, thanks for it! I have corrected it.
ReplyDeleteHi, thanks for the correction (the sample code in Google Code is still not updated - documentation part).
ReplyDeleteI would like to add another note, that might be useful to other people.
I had hard time to make this solution work as the example above because of a stupid mistake: the language code syntax.
From HTML, the syntax is more commonly accepted as hyphen (-), such as pt-BR, ja-JP. The standard is registered at: http://www.rfc-editor.org/rfc/bcp/bcp47.txt (section 2.1 syntax)
However, Django understands underscore (_) ONLY instead. I don't know why this is hard-coded there (or if it has been fixed already in newer versions).
@BRAGA thanks again. I met that problem when I firstly tried to make it work, therefore I put `zh_TW` in sample code. But I forgot to mention the point. Thanks again!
ReplyDeletehttp://eflorent.blogspot.com/2010/08/internationalization-under-google-app.html
ReplyDeleteHere I follow Livebetter and additional explanation on Internationalization and Localization under the Google App Engine framework.
ReplyDeleteHi im a bit slow on the uptake - can someone tell me - im using aptanna studio where and how do i run these python commands? if my app engine path is - D:/program files/google/google-appengine -
ReplyDeleteI have tried from the command prompt but that dosnt work
can someone give me some simple help on how i do this
many thxs
No, you don't need the PYTHONPATH. Do you code run as described in Setting Up section?
ReplyDeleteSorry about dumb problems..... But I follow this tips, and I got an error. When I try to access the page, I read a trace that ends with:
ReplyDeleteEnvironmentError: Environment variable DJANGO_SETTINGS_MODULE is undefined.
Do I need to start app engine setting enviroment variable PYTHONPATH like the folowing command?
$ PYTHONPATH=/home/hugo/bin/google_appengine/lib/django/ /home/hugo/bin/google_appengine/dev_appserver.py ./
It would be really helpful if this post can be updated to the current django version (1.2) that is recommended by google using use_library.
ReplyDeleteI followed all these steps and got no error but I have a problem. Strings are not translated. When I launched your project for the first time (smart-gk), strings were translated. But when I modify strings translation, nothing change. The old values are showed. Can you help me please?
ReplyDeleteIf you have read "caching problem" section, you know I had also noticed the issue. (Note: this post was written long ago and some may not be same under the current GAE environment. I have not used I18N or GAE for long time.)
ReplyDeleteRestart the dev server. I guess instances will be ended under the current GAE after deployment, if not kill all the instances.
Thank you. I restart the dev server and it works for your project. I tried to add internationalization to my own app but translations are not done, even the first time. I also notice that files inside "conf" folder are not compiled. When run app, normally Python files which are not for request handler must be compiled. Maybe it's the problem?
ReplyDeleteIf there are not compiled, then your script/Django doesn't know about stuff conf. You didn't configure correctly, some code might be missed. That's all I can guess.
ReplyDeleteIt's been a long time, GAE changes a lot, you may want to find GAE I18N support for newer Django, if there anyone has written about it.