The fight of Man against Machine is a hot button topic. Both our clients and our colleagues are probably pondering it a lot: “AI is translating text about as well as a human could — so about half of the human translator workforce is probably going away, and that might be just as well since Google Translate has made leaps in quality since 2005.” “Why should I pay a whole bunch of people if a robot can do the same job better?” “Are soulless wirebrains coming to take my job?” “Why bother studying at all? An entire profession—librarians—has all but disappeared before our eyes, and we are next in line for elimination.”
Just 5-10 years ago in this industry, adequate machine translation was but a twinkle in the science geek’s eye. Today, every conference of note brings up this issue. The industry is changing, and machine translation is becoming its own segment within it. A new job has emerged and is in high demand: Machine Translation Post-Editor.
Every year we need more professionals working with AI and automation while the demand for industry-scale manual labor is dwindling.
Below we have some data demonstrating the rise in demand for AI technology:
Machine translation (MT) is already becoming a part of the toolkit for translators and localizers.
But do localizers really need it? Or is it a T-1000 in sheep’s clothing? Could this be the panacea? Is this way of the future?
Clients have their own questions. Namely, if MT is so great, so flexible and so fast, why should they pay more for human talent?
To discuss these pressing matters, we have all but pressured some industry professionals to give their candid opinions:
Managing Partner at Levsha
Head of Localization at Levsha
Head of Localization at Social Quantum
How do you translate games at your company? Do you use machine translation?Daniil Sokolov
We sometimes have to deal with MT, but we don’t use it ourselves. People send us their machine translation for editing and we do that, but we don’t do machine translation ourselves. I don’t know if this arrangement falls under “use of MT.”
We tried to set up machine translation from Chinese to Russian once: we wanted to offer a cheap option to our clients. At the time—that is to say, about 18 months ago—we were not satisfied with the results.
Most of the problems we ran into were related to some game-specific features of the text we were working with. We are localizing the body of text that is not intended for a human to process: it’s input for the game engine. This, in turn, means that the text includes some special framing elements, unknown parameters, and more. The game engine processes all this and then puts the resulting text in front of the player. 18 months ago MT could not handle this challenge. Moreover, the quality of Russian translation was unfit for further editing.Mikhail Gorbunov
We prefer the tried and true way: RU>EN>LOC. We work with 12 languages in total: Russian, English, German, French, Spanish, Portuguese, Italian, Turkish, Japanese, Korean, Simplified and Traditional Chinese.
We use Memsource and Gridly. We have an English editor on our staff who doesn’t even know our source language (Russian), but we have full confidence in his ability to make English text good :)
We have a team of freelancers, 36 people. It’s a tight-knit group, some of them have been with us for 8 years now.
We do use MT—quite a lot, actually—but I can’t say that we love it. For us it’s a tool that can save us time and money.
The hard part about using MT was showing our translators that it is not their enemy. In fact, it can save them some time.
How good is machine translation in terms of price, quality, and speed?Fedor Kulikov
Machine translation is mostly useless for the majority of our cases. Movie translation, translation of individual lines of dialogue, football commentator speech, descriptions of some regions or spells — for all these things you have to be aware of the broader context to produce a good translation. In dialogues specifically, it’s not just about word-for-word translation, it’s about the things that happen before the line is spoken and how this line ties in with the next. In games, it’s about references to in-game lore. In descriptions, it’s about naming conventions we have to discuss and agree upon before we even touch the main body of the text: which synonyms are appropriate for the game’s setting, which are not to be used, things like that.
Another problem with MT is its complete incompetence when it comes to idioms people use in speech. Even if you have a good human editor who is capable of utilizing Google, Urban Dictionary, and their certain je-ne-sais-quoi, it’s going to be hit or miss. When you set about editing a machine translation like that, you often end up rewriting the translation from scratch to breathe life into the text. And if you don’t do it, that MT flavor will not go away.Daniil Sokolov
In my opinion it’s reasonably cheap and definitely fast. Whether or not it’s good depends entirely on how challenging the source text is. First thing you have to consider with MT is whether or not you should apply it to the kind of text you have. Secondly, you would have to train the translation engine for your specific needs. If you skip these steps, you will end up in a situation where your linguists will curse you and throw the text away entirely to translate it from scratch: sometimes this could be worse than starting from a blank page.Mikhail Gorbunov
MT is cheap, quick, and easy. I’m joking when I’m saying this, but only in part. It really depends on what you apply it to and what kind of results are you looking for. In certain scenarios it can work great right out of the box: usually it’s for uncomplicated texts on some generic topics. DeepL, for instance, can handle our Megapolis city builder and Wild West farming games pretty well.
In other scenarios, MT has to be trained. You have to feed it bodies of text and glossaries to make it learn terminology and professional jargon. This approach works in legal, technical, and medical translation.
There are, however, some scenarios where MT just plain doesn’t work. We have a game in development right now with a metric ton of pop culture references and the kind of English language that you will never read in a book. We have experienced translators with degrees who can’t reliably handle it. There is no text body we could possibly feed to a machine translation engine to teach it how to translate something like that. It’s impossible to make the machine learn something like this.
Fine-tuning a machine translation engine is very difficult. Using other people’s text bodies is expensive: this sort of data is not exactly easy to come by. To train a machine translation engine you need your own text body, and building one up can take years. That’s before you account for NDA, copyright, things like that.
What’s more profitable for a business, staff linguists or one tuned MT engine?Fedor Kulikov
These are two instruments for achieving different goals. For some purposes (like legal documents, technical documents, and—rather bewilderingly—TCG rule descriptions, a neural network trained on a large body of text could work fine) machine translation could be a viable option, but if you don’t have a relevant body of text or need to be careful with the details, then it just won’t. Machine translation just isn’t the right tool for every job.Mikhail Gorbunov
It really depends on what kind of texts you have to translate. If you have similar texts, like, for instance, different games in the same genre or hyper-specialized texts, and your MT engine seems to handle it well, you won’t need linguists on staff once you train your engine properly.
If you work with games of different genres, software, literary translation, marketing translation — you will need human resources. MT will not handle it on its own. In my opinion, we should view MT as a tool that can be used to solve a limited number of rather specific tasks. It’s not a one stop shop solution for your business.
Besides, MT output is never perfect, so you will have to keep editorial staff instead of a translator staff anyway.
How does a localization studio decide whether or not they need machine translation?Fedor Kulikov
You have to understand the scope of your business. In the game industry, there are only a couple areas where you could apply MT. One could, of course, sell it to their customers as the latest and greatest bleeding edge solution, that’s how some people do it. There are, however, some reputational risks associated with that. You also don’t get to be proud of that kind of work. And you get all the hate from the player base.Mikhail Gorbunov
To implement MT, I would say you have to experience these factors:
- your studio can’t process the volume of orders it receives with the linguist pool it has at its disposal, and you are unable to expand your pool of linguists for any reason;
- your margins are getting thinner as the translation rates rise and MT is getting better and cheaper;
- you have run some MT experiments just for fun and realized that you could apply it to at least some of the work you have coming in to save yourself time and/or money.
When should localization avoid machine translation?Fedor Kulikov
For most games. You can see the remnants of MT during review very clearly if the translator has used it. There are exceptions though: some oddball texts like TCG rules are not actually a living text, they are a lot more like programming language with their if-then conditions.Daniil Sokolov
“When should localization (in the foreseeable future) consider (NOT) using MT?” :-)
Red flags are mostly derived from the contents of the text:
- colloquial/street language;
- cultural references/wordplay/jokes/sarcasm;
- hidden untranslatable elements (e.g. variables);
- sometimes you will see unfinished lines that don’t even make a complete sentence — that’s often the case with effects from bonuses or spells.
In terms of general attitude, you should not even try MT if you don’t have a substantial body of text for a given language pair.Mikhail Gorbunov
MT can’t stomach literary translations, marketing translations, metaphors, allusions, jokes, and wordplay. Don’t even think about MT if you don’t have editors who will dredge through its output and fix it.
In my company, we never use MT on new projects where we have no translation memory bank or a glossary.
Is it feasible to combine MT with human talent services under the roof of the same studio?Fedor Kulikov
I don’t think that’s the right question. Is it feasible to put pig cartilage and watermelon in the same fridge? Yes, it is. I believe what you want to know is whether or not a human translator can work side by side with a machine translation engine on the same project. It can be done if you can isolate translation memories, setting the texts aside for MT that cover themes unrelated to other aspects of the project.Daniil Sokolov
I would say it’s hard not to. The best MT can do is give you something that will get the general ideas across, something good enough for internal use. It will not give you a complete product. If you intend to use MT, you will need some human linguists to work with it.
Raw MT output is only good enough for translating things like instructions to a utility you want to use internally. You can explain to your people that yes, the quality sucks, but the general idea is there and it’s better than trying to read Chinese. You can also just double-check something you don’t understand. What you should not do is sell this kind of translation or use it in a product that’s going up for sale.Mikhail Gorbunov
It is, and in fact you should do that. In our company, at least, we are doing it and it works fine for us. In most cases MT post-editing and translation are handled by different people, but in our company it’s the same linguists doing both.
It’s very important to delineate the scope of application for MT and for people. You don’t want your human teams feeling like they are about to be replaced.
How often does machine translation make mistakes? When do people have to correct it?Fedor Kulikov
When don’t they need to correct it?Mikhail Gorbunov
I can’t evaluate that without more input data. It depends on the source text and on the engine you are using. In some cases you will have 99% of segments translated with errors, in some cases you will get 40%. I have never seen error rate less than 40% personally, and I’m basing this off of my own projects and some feedback from my colleagues. People will always have to step in. You have to check the output of MT: it could cost you a client if you don’t.
Is machine translation quality different for different language pairs?Fedor Kulikov
Of course it is. The difference can be quite stark. It all depends on the bodies of texts it uses and features of given languages. For English to Russian language pair, there is a substantial body of text; for Korean to Russian it’s not so big. In Korean specifically, a lot of obvious details within a given context can be straight up omitted from a sentence in colloquial speech — that kind of thing is pretty dang rough.Mikhail Gorbunov
It’s going to be different engine to engine, language to language, text to text.
Translation quality for a given language pair depends on algorithms powering the MT engine as well as the volume and quality of text bodies it was trained on. The more common either language in a pair is, the better the translation quality since it’s easier to find people who can work with them. It’s easier to develop algorithms and find text bodies to train them on.
MT quality in the KO>RU pair is low because most of that translation is done as KO>EN>RU, therefore it’s more likely to run up errors.
Do you feel like MT has made progress in the last 10–15 years? What got better, what got worse?Fedor Kulikov
There is now a hope for models that you can train for specific projects, but in reality that kind of thing is still in its infancy. Often you just don’t have the right body of text for the project you have.Mikhail Gorbunov
It definitely got better, especially in the last 5 years with the introduction of neural networks. It used to be that engines translated text using internal databases and templates, the algorithms involved were rather primitive. Today it’s a fully fledged artificial intelligence that can learn a lot—not everything, but a lot—and make things easier for people.
What do you think is next for MT? Will it get more user-friendly/precise/fast?Fedor Kulikov
Well, everything is developing. For live speech with jokes and humor that translators and editors have to get across, MT won’t catch up just yet. People put a piece of their soul in their work, and you will get very different results if you give the same project to different translator/editor teams. What we do lies somewhere between translation and transcreation, and the ratio of the two varies greatly across different projects. Here’s an example: Halloween is coming up, and we have an article with the following title:
Where wolf? There wolf!
Среди темных подворотен, где гуляет оборотень
Оборотней бояться — в лес не ходить.
I shudder at the thought of how MT would butcher a title like that.Mikhail Gorbunov
I don’t know if it can go any faster, and I don’t believe it should. MT can already handle thousands of words per minute. What should improve is its accuracy and ease of implementation, because right now training an MT engine is a complicated technical process that takes a lot of time, and don’t even get me started on integrating one with a CAT suite.
Will MT ever replace human translators?Fedor Kulikov
No. Depending on where you stand, it could be a relief or a pain.Daniil Sokolov
Discussing something like this with no timeframe given is senseless. MT won’t replace people completely in the foreseeable future, but the scope of its application may expand.
There will come a point where MT will suffice for all practical purposes. Neural networks drive better than people, diagnose better than people, and there is no reason they won’t surpass people in translation. There will remain a small fraction—a fraction of a percent—of cases where people will have to be involved because improving the technology past a certain point will produce diminishing returns on invested effort.Mikhail Gorbunov
I would like to rephrase that. Will MT replace human translators in every industry, and if it will, how soon will it happen?”
In some fields of translation it’s already happened. There are fields where people don’t need high-quality translations, and in those fields people have switched to MT results already.
Literary translation and transcreation are a human domain and I don’t think there will be any incursions on it within the next 5 years.
We should, however, see some progress on the spectrum that lies between these polar opposites. Every year MT engines are getting better. Where an editor used to spend an hour of their time, they now spend 30 minutes or less because there are fewer segments that need to be rewritten start to finish.
What advice do you have for a beginner translator?Fedor Kulikov
Don’t use MT in your work and don’t go editing MT results — it won’t do you any good, especially when you are just starting out.Daniil Sokolov
Accept that machine translation is not going away and will take over more and more routine jobs. Consider that various flavors of MT Post-Editing will now join the trifecta of Translation, Editing, and Proofreading. Learn to spot mistakes in MT output and when to expect these mistakes; figure out when it could be technically correct but not necessarily good, and when it is undeniably right. You should be able to spot potential problems by just looking at a translation. If you are able, study the principles of neural networks that power machine translation and understand what it means to train it; learn its hard limits.Mikhail Gorbunov
Improve your professional skills, broaden your horizons, and look for good customers — I wish you the best of luck with that last part :-)
Get familiar with MT and learn MT Post-Editing. The demand for that kind of work is skyrocketing across the job market.