moinsen / Software historyDE ↗
← 2017

Event

Der Transformer für maschinelle Übersetzung

Context and research status
  • Kennung: 2017-transformer
  • Zeitraum: 2017
  • Datumsgenauigkeit: Jahr, sofern im Text nicht genauer / year unless specified in text
  • Art: historische Entwicklung / historical development
  • Kategorien: Sprachen und Techniken, Ereignisse
  • Akteure und Rollen: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser und Illia Polosukhin: Forschung
  • Region: International

The 2017 research paper Attention Is All You Need introduced the Transformer architecture. The authors evaluated the attention-based architecture, without recurrence or convolution, on machine translation tasks. Research paper Source

The paper names eight people and marks their contributions as equal. One later development building on this architecture is BERT in 2018. Authorship

In simplified terms, self-attention can weight relationships between different parts of a sentence. In the original translation model, an encoder processes the input and a decoder generates the translation step by step. The conference paper reported 28.4 BLEU on the WMT 2014 English-to-German test. This is an automatic score for that test, not a percentage measuring general language understanding. Experiment and explanation

For English-to-French, the conference version reports 41.0 BLEU. The later arXiv version v7 gives 41.8 in the abstract and table but retains 41.0 in its results section. The reason and correct figure remain unresolved. Version comparison

A further discrepancy appears on the NeurIPS metadata page: its HTML abstract gives 41.1 for English-to-French and 27.5 for English-to-German, whereas the linked conference PDF gives 41.0 and 28.4 respectively. The arXiv abstract changed from 41.0 in v4 in June to 41.8 in v5 in December 2017. The reasons for these differences are not established. Original versions

The conference paper compares 41.0 for the large single Transformer model with 41.29 for a ConvS2S ensemble in the same table. Its results section therefore claims an improvement over earlier single models. The later arXiv table gives Transformer 41.8; comparisons need to identify the version. Original tables

Sources

Offen

Ursache und zutreffenden Englisch–Französisch-Wert unabhängig klären; arXiv v7 ist auch intern uneinheitlich. / Independently resolve the cause and correct English-to-French value; arXiv v7 is internally inconsistent.