Training-data credits

Sonja PhraseFlow · Sonja Projects
Author: Sonja Sahebzad · Utrecht, the Netherlands

Tatoeba

The eight additional models use sentences by Tatoeba contributors, downloaded from the official exports dated 26 September 2026, under Creative Commons Attribution 2.0 France. Text was normalized, segmented, sampled and aggregated into statistical models. Tatoeba does not endorse this app.

The files below preserve the credited contributor, sentence ID and original sentence URL for each training sentence. A missing owner in the original export is preserved as supplied; the sentence page provides its history. Held-out evaluation answers are not included.

• Dutch: training contributors and sentence URLs (TSV, gzip)
• French: training contributors and sentence URLs (TSV, gzip)
• Korean: training contributors and sentence URLs (TSV, gzip)
• Mandarin Chinese: training contributors and sentence URLs (TSV, gzip)
• Portuguese: training contributors and sentence URLs (TSV, gzip)
• Italian: training contributors and sentence URLs (TSV, gzip)
• Hindi: training contributors and sentence URLs (TSV, gzip)
• Spanish: training contributors and sentence URLs (TSV, gzip)

Earlier models

English, German, Finnish and Russian use the official Coursera SwiftKey corpus. The project retains its source archive and checksum. No course quiz text or answers are included in these attribution lists.

Software and method

The app uses R, Shiny, data.table, jsonlite and stringi. Chinese segmentation uses ICU dictionary word boundaries. The compact language models use interpolated Kneser-Ney smoothing. Download checksums, runtime versions and transformations are recorded with the experiment scripts.

Return to the user guide and results