Why Getting the TMX Format Right Is More Consequential Than It Looks
Anyone working in machine translation model training eventually runs into the same wall: a perfectly clean Excel file full of bilingual segments that simply will not import correctly into a training platform. The data looks fine. The columns are there. The text is aligned. But the TMX file the platform receives throws validation errors and refuses to cooperate.
This is not a minor inconvenience. When you are training a custom MT model — say, inside Systran Model Studio — the quality and structural integrity of your training data directly determines how well the resulting model performs. A TMX file with encoding issues, malformed headers, or segments that exceed certain character-handling thresholds does not just fail silently. It produces a model trained on incomplete or corrupted data, which means poor translation output that erodes trust in the entire workflow.
The stakes are real whether you are building a domain-specific model for legal translation, technical documentation, or any specialized field. Getting the Excel-to-TMX conversion right the first time saves weeks of retraining and debugging.
What Proper Excel-to-TMX Conversion Actually Requires
The instinct is to treat this as a simple export task — run a macro, use a converter tool, move on. In practice, the work is more structured than that.
A valid TMX (Translation Memory eXchange) file is an XML-based format governed by the TMX 1.4b specification. That means every structural element has a defined place, every attribute must conform to expected values, and the file encoding must be declared and consistent throughout. Excel, by contrast, stores text in a flat cell structure with no inherent awareness of XML rules, character encoding, or segment-level metadata.
Done well, the conversion process requires four things to happen correctly. The source and target language codes must match the IANA BCP 47 standard (for example, "en-US" and "fr-FR", not "English" or "French"). The XML header must declare the correct source language, creation tool, and segmentation type. Every translation unit must be wrapped in the right tag hierarchy — <tu>, <tuv>, and <seg> — with no stray characters or encoding artifacts. And segments longer than roughly 100 characters need to be validated for whitespace normalization and special character escaping before they enter the XML structure.
Skipping any one of these produces the kind of validation errors that a platform like Systran Model Studio will surface immediately.
How the Conversion Work Actually Gets Done
Auditing the Source Excel Data First
Before any conversion happens, the Excel source file needs a structural audit. The bilingual data typically lives in two columns — source language in column A, target language in column B — and the first step is confirming that alignment is exact. A single misaligned row, where a source segment sits across from the wrong target segment, corrupts that translation unit permanently. At scale, with thousands of rows, even a 0.5% misalignment rate introduces dozens of bad training pairs.
The audit also checks for empty cells, duplicate segments, and encoding anomalies. Excel files saved in older .xls format can introduce non-UTF-8 characters that survive into the TMX and break XML parsing. The safest approach is to open the file, save it explicitly as UTF-8 encoded CSV, and inspect the raw output in a plain-text editor before proceeding. Hidden characters — particularly non-breaking spaces (Unicode U+00A0) or smart quotes — are invisible in Excel but cause XML validation failures.
Building the TMX Structure Correctly
A valid TMX file opens with an XML declaration and a TMX header. The declaration must read <?xml version="1.0" encoding="UTF-8"?> exactly. The header tag carries attributes including creationtool, creationtoolversion, datatype, segtype, and srclang. For a sentence-level training corpus, segtype should be set to "sentence" and srclang must match the BCP 47 code for the source language column.
Each row from the Excel file becomes one <tu> (translation unit) element. Inside each <tu> sits two <tuv> elements — one for the source, one for the target — each carrying an xml:lang attribute. The actual text goes inside a <seg> tag within each <tuv>. A correctly structured unit for an English-to-French segment looks like this in practice: the <tuv xml:lang="en-US"> block contains <seg>The system will restart automatically.</seg>, and the corresponding <tuv xml:lang="fr-FR"> block contains the French equivalent in the same tag structure.
For segments longer than 100 characters, special attention goes to escaping reserved XML characters. An ampersand in the source text must become &, a less-than sign must become <, and quotation marks inside attribute values must be escaped as ". Failing to escape these characters in longer segments is one of the most common reasons a TMX file that looks valid in Excel throws parsing errors on import.
Using Scripted Conversion for Reliability
Manual assembly of TMX files from Excel is error-prone at any scale beyond a few dozen segments. A Python script using the openpyxl library to read the Excel file and xml.etree.ElementTree to build the TMX structure is the standard reliable approach. The script reads each row, strips leading and trailing whitespace from both cells, escapes reserved characters, and writes each translation unit into the XML tree. At the end, the tree is serialized with encoding="unicode" and written to file with an explicit UTF-8 BOM declaration.
For a dataset with segments of varying length — including many over 100 characters — the script should also include a validation pass that checks segment length, flags any cell containing raw HTML or markup tags, and logs any row where either cell is empty. Running that log before the final TMX export gives a clean picture of data quality before submission to the training platform.
What Goes Wrong When This Work Is Underestimated
The most common failure is skipping the pre-conversion audit entirely. Teams export directly from Excel using a generic TMX converter tool, see that a file is produced, and submit it — only to hit validation errors they cannot diagnose because the converter gave no feedback about the underlying data issues.
A second pitfall is language code formatting. Tools like Systran Model Studio are strict about BCP 47 codes. Submitting a TMX with srclang="EN" instead of "en-US" or "en-GB" often triggers a header validation error that looks unrelated to language codes on the surface.
Encoding drift is a third issue that compounds quietly. If any segment in the Excel file was pasted from a PDF, a web page, or a Word document, it may carry invisible Unicode characters — particularly soft hyphens (U+00AD) or zero-width spaces (U+200B) — that survive into the TMX and cause the XML parser to fail mid-file. A 5,000-segment corpus with three bad characters buried in row 3,847 will fail entirely, and the error message will point to a line number in the XML, not a row number in Excel.
Fourth, teams often underestimate the difference between a TMX that opens without errors and one that trains a model well. A structurally valid file can still contain duplicate translation units, inconsistently tokenized segments, or mismatched segment boundaries that reduce training quality significantly. Deduplication and segment-length normalization — keeping segments between roughly 5 and 200 words for most MT model types — are not optional polish steps; they directly affect model performance.
Finally, building the TMX as a one-off manual effort rather than a repeatable scripted pipeline means the next batch of training data requires the same error-prone process from scratch.
What to Take Away From This
The path from a bilingual Excel file to a valid, training-ready TMX is not a single-step export. It is a structured workflow: audit the source data for alignment and encoding issues, build the XML structure to specification with correct language codes and character escaping, validate the output before submission, and script the process so it can be repeated reliably on future data batches.
The work is doable with the right tools and patience. If you would rather have it handled by a team that does structured data and file conversion work every day, Helion360 is the team I would recommend.