Internationalization

As you may or may not know, I'm French and I'm writing the story and dialogs of Superfluous Returnz in the French language. But fairly early in the development of the game, I thought it would be nice to be able to translate it, especially in English, just in order to broaden the potential audience.

So I'm going to talk to you about internationalization, or i18n for short. As the game does not have any recorded voices to date, it is mainly a question of translating the texts… and a few images.

The general principle

I want to continue to write my game in French, my native tongue, and to take care of the translation separately. So I'm moving all the translated lines of the game into a separate file, locale.yaml, which looks like this:

locales:

  - { id: "fr_FR", description: "Français (France)" }
  - { id: "en_US", description: "English (US)" }

lines:

  - fr_FR: "Je ne sais pas trop par où commencer..."
    en_US: "I'm not sure where to start ..."

  - fr_FR: "Vous devriez peut-être examiner ce joli tableau."
    en_US: "Maybe you should take a look at this lovely painting."

Firt, I list the available languages: for the moment, I only support French and English, but I want to keep open the possibility of adding other languages ​​later. Note that I'm using the usual code language_COUNTRY, which theoretically makes it possible to distinguish several variants of the same language (Quebec French, British English, etc.).

Then I list every line of dialog (or interface) in both languages. Of course, I'm much too lazy to do this by hand: I wrote a script that gathers, from all the game data (level and object descriptions), the lines to be translated, and which then, for each line, adds to the locale.yaml:

  - fr_FR: "(Whatever line it found)"
    en_US: TODO

All I have to do then is replace these TODOs. The script also checks that all lines that were previously listed still exist in the game data: if I ever corrected a spelling mistake in the French version, for example, the line in question simply does not exist any longer. In this case, the line is moved to a separate section (which allows me not to lose it: if I fixed the spelling mistake, I want to be able to find the translation I made for the corrected line).

For now, I'm storing all the lines in a single file, but I may segment it if it gets too big as I add levels to the game.

About images

The vast majority of images will not change from one language to another. However, there are bound to be differences: for example, some puzzles require the observation of some documents, and these documents must thus appear in the chosen language.

For example, this treatement form found inside the first level is translated (even if it's not very believable in the real world, of course):

In practice, in the game data, the image images/windows/fiche_secu.png will be accompanied with an image images/windows/fiche_secu.en_US.png. On loading, the engine will therefore simply search, for each image, if there is a variant with the country code added at the end.

In the code

For lines of dialogues, it's very simple: I store, for each language (different from the base language, here French) an object of the type std::unordered_map<std::string, std::string>: This is basically a hash table that maps one string to another.

Thus, when the engine looks for a line of dialogue, it either looks for the line in French, or looks in one of these tables for the translation into the wanted language. In terms of performance, the hash table should be efficient enough to accommodate thousands of lines of dialog without taking a noticeable amount of time to load a line.

For images, it's a pretty much the same principle: translated images have a hash table that associates a different image object with each language. On display, the engine therefore simply looks at the selected language before choosing a particular image.

The only thing I had to be careful with, is when the language is changed by users: images and text already displayed must be deleted and reloaded in the new language so as not to retain “outdated” info (in another language) on the screen.

Automatic language detection

If the engine obviously gives the option to change language at any time, it seems unthinkable today that a game wouldn't start in the system language automatically. So let's see how to detect this language when launching the game for the first time.

In theory, the C++ Standard Library provides a function to access various defined locales, std::locale. To retrieve the famous language_COUNTRY code used, all you have to do is:

    std::string locale = std::locale("").name();

And of course, the only system on which this function works correctly is… GNU/Linux.

I had already briefly discussed this in my articles on the Android and Windows ports, so I'm not providing details here, but basically, we manage to reconstruct this code using the features specific to each system, which gives something like this:

#if defined(SOSAGE_ANDROID)
#  include <SDL.h>
#  include <jni.h>
#elif defined(SOSAGE_WINDOWS)
#  include <wchar.h>
#  include <winnls.h>
#elif defined(SOSAGE_MAC)
#  include <CoreFoundation/CoreFoundation.h>
#else
#  include <locale>
#endif

std::string get_locale()
{
#if defined(SOSAGE_ANDROID)
  JNIEnv* env = (JNIEnv*)SDL_AndroidGetJNIEnv();
  jobject activity = (jobject)SDL_AndroidGetActivity();
  jclass jni_class(env->GetObjectClass(activity));
  jmethodID method_id = env->GetStaticMethodID(jni_class, "getLocale", "()Ljava/lang/String;");
  jstring jstr = (jstring) env->CallStaticObjectMethod(jni_class, method_id);
  const char *str = env->GetStringUTFChars(jstr, 0);
  std::string out (str);
  env->ReleaseStringUTFChars(jstr, str);
  env->DeleteLocalRef(jstr);
  env->DeleteLocalRef(jni_class);
  return out;
#elif defined(SOSAGE_WINDOWS)
  wchar_t name[LOCALE_NAME_MAX_LENGTH];
  int l = GetUserDefaultLocaleName(name, LOCALE_NAME_MAX_LENGTH);
  if (l == 0)
    return "";
  std::wstring ws (name, name + l);
  return std::string(ws.begin(), ws.end());
#elif defined(SOSAGE_MAC)
  CFLocaleRef cflocale = CFLocaleCopyCurrent();
  CFStringRef language = (CFStringRef)CFLocaleGetValue(cflocale, kCFLocaleLanguageCode);
  CFStringRef country = (CFStringRef)CFLocaleGetValue(cflocale, kCFLocaleCountryCode);
  char lstr[256], cstr[256];
  CFStringGetCString(language, lstr, 256, kCFStringEncodingUTF8);
  CFStringGetCString(country, cstr, 256, kCFStringEncodingUTF8);
  CFRelease(cflocale);
  return std::string(lstr) + "_" + std::string(cstr);
#else
  try // std::locale might throw runtime error if no locale declared
  {
    return std::locale("").name();
  }
  catch (std::runtime_error&)
  {
  }
  return "";
#endif
}

Note that SDL provides such a function, but only since a newer version than the version of SDL I support. If I update it, I'll probably dump my custom function for what the SDL provides.

Our get_locale() function therefore returns us a code language_COUNTRY or an empty string if the language was not found. It's debatable, but here's how I then select the language:

• if the returned language_COUNTRY is supported by the game (here, if it is fr_FR or en_US then), I select this language;
• otherwise, we forget the country and we check if another variant of the language is available (on a Quebec computer, we will detect fr_CA which is not supported, but we will use fr_FR because the language code is the same);
• otherwise, if the language is not available (or if get_locale() returns an empty string), English is used by default, which seemed preferable to using French.

Of course, there is always the option to change the language from the menu afterwards, but it's still much more comfortable if it works directly in the right language.

Conclusion

With this small mechanism, I manage to deal with an additional language, English, without disturbing the writing of the game which I keep doing in French. Nothing prevents me from adding other languages ​​later, whether they are variants such as Quebec French or completely different languages ​​such as Spanish.

There's room for improvement of course. I could replace the single text hash table with a mini-table for each line of dialog: it would probably be more efficient, but as I said, for now there is clearly no performance issue, so I'm keeping my current mechanism.