How Accurate Is WIPO’s Patent Machine Translation System?

The Word Intellectual Property Organization has expanded their patent machine translation tool – a neural machine translation software – to include ten languages for international patent research. Neural networks are considered the latest technological advancement in the translation services industry, and we decided to test out just how accurate a patent translation the WIPO platform could produce.

The Growing Influence of Neural Machine Translation

The potential disruption of Artificial Intelligence looms over the translation industry.

While there has yet to be a real substitute for humans, especially when it comes to extremely technical or creative language, machine translation (MT) systems are improving at what seems to be a  lighting pace.

The latest in translation software, neural machine translation (NMT), has in fact significantly helped improve the accuracy of machine generated translations.

Neural machine translation has allowed MT to shift from rule and phrase-based translation models, to one that resembles the functioning of neurons in the human brain. Using an encoding and decoding process that renders entire sentences, the technology can even factoring in some cultural and technical contexts.

The end result is a translation product that is closer to what a human translator can accomplish, relative to standard MT.

Many are looking to integrate this technology into processes previously handled only by human translators. The benefits are evident: faster results (albeit still less accurate), and cheaper costs.

A recent example of this is The World Intellectual Property Organization (WIPO), who has adapted this idea for their patent machine translation system.

NMT and International Patent Research

WIPO has specially designed and “trained” a new specialized NMT software to translate patent documents online. It is easily accessible to people around the world, and the system is able to correctly use more complex terminology than previously possible.

A version of the patent machine translation tool launched in 2016 was initially available only in English and Chinese. However, the platform has expanded and is currently available in 16 language pairs on the WIPO site, PATENTSCOPE database.

The language pairs for patent machine translation available include:

• English <-> Arabic
• English <-> French
• English <-> German
• English <-> Japanese
• English<->Chinese
• English <->Korean
• English <-> Russian
• English <-> Spanish
• English <-> Portuguese

 

Despite significant improvement over previous methods of machine translation, this method is not, and does not claim to be, 100% accurate. The platform is mainly useful for conducting an international patent search or obtaining general information regarding a patent filed in one of the official languages of the Patent Cooperation Treaty.

Highlighting this, the platform has a 2,000 word at a time limit on text that can be translated, and patent applications translated in this way are not considered legally binding as stated on WIPO’s site:

 

“This automatic translation is provided for information only, it may contain discrepancies or mistakes and does not have any juridical value“.

 

While this technology is not as of yet posing a severe threat to human patent translators, because NMT is known for providing better translations than past MT technologies we couldn’t resist from testing out the platform’s performance ourselves.

We wanted to determine two basic things:

1. How accurate a translation the NMT platform could produce for technical text
2. Whether or not it could be clearly understood by native speakers

To get answers to these questions our team decided to translate a portion of a public, English patent in four of the languages offered by the WIPO platform. We then had some of our in-house speakers of those languages analyze the results.*

*Please note the following information is not meant to provide anything more than a general review of current patent machine translation technology. Our team members are not legal professionals and are not providing any legal advice. Measurements used were estimated and did not undergo actual statistical analysis.

Could WIPO’s Neural Machine Translator Replace A Human?

For our experiment we chose a public chewing gum patent. We made sure to test a portion with some technical language to see just how well the NMT could handle complex verbiage.

Here is the English text in full:

We tested out the WIPO translate tool for four languages:

• Spanish
• Portuguese
• French
• German

This is what we found:

Spanish Patent Translation

“All in all the translation gets the main points across and can be understood. But there are some issues with certain words and phrases.” – LC Spanish Speaking Team Member

Some of the Problems Found:

• The English word “cud” has not been translated into Spanish and is used throughout the translation.
• Depending on the target audience, the word “chicle” is more appropriate for “chewing gum” as opposed to “goma de mascar” (the word “goma” in Spanish means rubber as in the material).
• Some of the phrases are not optimally translated in the target language. For example the phrase translated from “The chewing gum of claim 1” –  “La goma de mascar de conformidad con la reivindicación 1” – is a bit awkward.

Although it stays true to the source, sentence structure is around 70% accurate for the translation.

Portuguese Patent Translation

“For the most part, the text can be understood, but there were a few issues/mistakes with the Portuguese version.” – LC Portuguese (European) Team Member

Some of the Problems Found:

• The word “cud” is used throughout the text in English and has not been translated.
• While not a mistake it is important to note that, depending on the target audience, the word for chewing gum can vary. “Goma de mascar“, which was used throughout the translation, is used in Brazil; however, “pastilha elastic” is more commonly used in Portugal.
• The gender of the verb used wasn’t always correct, such as in the phrase: “quando da mastigação” (“when chewing”). It should be “quando mastigada“.
• The spelling of the verb in the phrase “médio pondera” (from the 8th and 9th English points “having a weight average molecular weight”) is incorrect. The correct spelling is “médio ponderado”.
• When denoting the “portion comprises 55 to 70% by weight”  in the 9th English point, the translation stated “de 55 a 70% em”. The correct form is “55 a 70% do peso”. This same correction can be made for the translation of the 8th point.

All in all, we’d say the translated text was approximately 75% accurate for the Portuguese translation.

French Patent Translation

“Although the translation is not always easy to understand, it does convey the main idea.” – LC French Team Member

Some of the Problems Found:

• The English word “cud” has not been translated into French and is used throughout the text.
• Certain phrases such as “Le chewing gum de la revendication” which translates to “Claiming chewing gum” (from “The chewing gum of claim” in the source text) are unusual and not translated well.

In total, the translation was approximately 60% accurate.

German Patent Translation

“The main ideas can be understood by a native German speaker with the exception of a few issues.” – LC German Team Member

Some of the Problems Found:

• The English word “cud” was not translated but rather remained as “cud“.
• The word “chewed” was translated into the German word meaning “combed”, which made the sentence sound very funny.
• The word “claim” was translated to the German word “Anspruch” (to claim ownership of something) instead of “Behauptung” (to claim something to be true) which distorted the meaning of many of the points made in the abstract.

Overall the German translation performed the worst out of the four, with only a rough 30% accuracy score (mostly due to comprehension issues caused by the mistranslated “Claim”).

In Summary – Patent Machine Translation Technology Needs Some Work

Comparing the translation products between English, Spanish, French, German, and Portuguese, it is clear that native speakers of these languages should be able to discern what is meant by the translated content.

However poor sentence structure and incorrect terms (such as “cud” which remained untranslated in each test) render the translations as “gist” worthy only.

One of the unique offerings the WIPO NMT does offer is it’s suggested alternative translations that can be accessed by hovering over a sentence (as seen below). This could help users to clarify some of the less-accurate translations.

Interestingly though, the command offered by the WIPO translate tool – “Please hover your mouse over parallel segments of text”- used for reviewing portions of the content, was rendered incorrectly in both French and German.

The German translation was particularly bad, as “Please hover your mouse” was translated as “Plissierhoffyour-Maus”. Needless to say our German colleague had never encountered this command before.

Evidently, the tool still needs work on some of its more basic language.

That being said, there are instances where the patent machine translation tool may be a helpful tool when filing. For example, if it is used in conjunction with a human translator it may help reduce time and costs associated with translation.

However, to ensure you avoid potential ramifications when filing, we do agree with the WIPO recommendations to only use this type of patent translation software for preliminary information research.

To conduct your own patent machine translation experiment, head over to the WIPO Patent Translation Tool and let us know how your test goes in the comments!

Professional Patent Translation Is Safe…For Now

All in all our neural patent machine translation experiments were approximately 59% accurate (by rough estimation).

This should come as no surprise seeing as the system itself was created for information collection only. Whether or not this changes in the future as MT continues to advance is yet to be seen.

For now, those looking to move beyond the research of foreign patents to file with an international office, should continue to use professional patent translators.

As seen during our tests, the language is at risk even with a sophisticated tool like an NMT. This is especially true for technical language, and the legal implications of a poor translation can range from rejection to complications with IP protection down the line.

Besides, after all the hard work of researching and interpreting semi-accurate translations of patent claims for your target filing countries, it would be a shame to put your own patent at risk with a bad translation!

Learn More About Our Legal Translation services!

About Language Connections:

Language Connections is one of the top language service companies in the US. Over the last 30 years, we’ve focused on providing the best business translation services, interpreting services, as well as interpreter training and customized language training programs. In addition to top-tier corporate language training, we offer certified corporate interpreters and professional business translation services in 200+ languages. Our network includes linguists with backgrounds in all major industries. They’re ready to meet your needs, whether they’re for technical translation services, legal translation, government translation services, international development translation services, education translation services, life sciences translation, or something else. Reach out to us today for a free quote on our cost-efficient and timely translation services, interpreters, or other linguistic services.

Language Connections Inc.
2001 Beacon Street, Suite 105,
Boston, MA 02135
Phone: +1-617-731-3510
Email: service@languageconnections.com