<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[dev_kevinxiao]]></title><description><![CDATA[dev_kevinxiao]]></description><link>https://kevinxiao.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Sun, 15 Nov 2026 18:41:39 GMT</lastBuildDate><atom:link href="https://kevinxiao.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[German Excel CSV breaks your React importer: semicolons, decimal commas, and quoted line breaks]]></title><description><![CDATA[Originally published on DEV. A first CSV import often looks like this: It works on the Google Sheets export you used for testing. Then a support ticket…]]></description><link>https://kevinxiao.hashnode.dev/german-excel-csv-breaks-your-react-importer-semicolons-decimal-commas-and-quoted-line-breaks</link><guid isPermaLink="true">https://kevinxiao.hashnode.dev/german-excel-csv-breaks-your-react-importer-semicolons-decimal-commas-and-quoted-line-breaks</guid><category><![CDATA[React]]></category><category><![CDATA[TypeScript]]></category><category><![CDATA[JavaScript]]></category><category><![CDATA[csv]]></category><dc:creator><![CDATA[Kevin Xiao]]></dc:creator><pubDate>Sun, 15 Nov 2026 02:46:33 GMT</pubDate><enclosure url="https://cdn.hashnode.com/uploads/covers/6af90de2d94103036f0c1199/7ca03f41-2c34-4cc1-a7c3-08883e2ed5ea.jpg" length="0" type="image/jpeg"/><content:encoded><![CDATA[<p><em>Originally published on</em> <a href="https://dev.to/_dev_kevin_xiao/your-csv-import-works-on-your-laptop-then-a-customer-in-germany-uploads-a-file-10m3"><em>DEV</em></a><em>.</em></p>
<p>A first CSV import often looks like this:</p>
<pre><code class="language-ts">const text = await file.text();
const rows = text.trim().split('\n').map((line) =&gt; line.split(','));
</code></pre>
<p>It works on the Google Sheets export you used for testing. Then a support ticket arrives:</p>
<blockquote>
<p>"The import says every row is invalid. It's the same spreadsheet we use every month."</p>
</blockquote>
<p>You open their file:</p>
<pre><code class="language-csvs">ID;Name;Email;Plan;Seats;MRR;Country
C-3001;Jos� Mart�nez;jose@example.es;Pro;6;299,00;ES
</code></pre>
<p>That one line hides three everyday Excel problems outside the US — and your two-liner hits all of them:</p>
<ol>
<li><strong>Encoding isn't UTF-8</strong> → accented letters become <code>�</code> (<code>file.text()</code> always decodes UTF-8).</li>
<li><strong>Delimiter is</strong> <code>;</code> → European Excel uses semicolon when comma is the decimal separator.</li>
<li><code>299,00</code> <strong>is a number</strong> → decimal comma, not "invalid".</li>
</ol>
<p>Add two more traps that show up the same week: <strong>newlines inside quoted fields</strong>, and dates like <code>05/03/2025</code> that mean different days in the US and Europe.</p>
<p>Below is how to handle each in the browser with plain TypeScript — no CSV library required. The pattern that keeps support tickets short: <strong>validate strictly, suggest generously</strong>.</p>
<p>If you remember only three things:</p>
<ol>
<li><strong>Detect the delimiter and locale first</strong> — don't hard-code a comma, and don't guess inside your API.</li>
<li><strong>Treat newlines inside quotes as valid CSV</strong> — they look like a broken file; they aren't.</li>
<li><strong>Show the parse result to the uploader</strong> so they can fix rows in the browser before you submit.</li>
</ol>
<p>Encoding pitfalls and ambiguous dates show up in the same files; the sections below cover them as supporting detail, not as extra takeaways.</p>
<h2>1. Decode bytes — don't trust <code>file.text()</code></h2>
<pre><code class="language-ts">export function decodeCsvBytes(buf: ArrayBuffer): {
  text: string;
  encoding: string;
  warning?: string;
} {
  const bytes = new Uint8Array(buf);

  if (bytes[0] === 0xef &amp;&amp; bytes[1] === 0xbb &amp;&amp; bytes[2] === 0xbf)
    return { text: new TextDecoder('utf-8').decode(bytes.subarray(3)), encoding: 'utf-8' };
  if (bytes[0] === 0xff &amp;&amp; bytes[1] === 0xfe)
    return { text: new TextDecoder('utf-16le').decode(bytes.subarray(2)), encoding: 'utf-16le' };
  if (bytes[0] === 0xfe &amp;&amp; bytes[1] === 0xff)
    return { text: new TextDecoder('utf-16be').decode(bytes.subarray(2)), encoding: 'utf-16be' };

  try {
    return {
      text: new TextDecoder('utf-8', { fatal: true }).decode(bytes),
      encoding: 'utf-8',
    };
  } catch {
    return {
      text: new TextDecoder('windows-1252').decode(bytes),
      encoding: 'windows-1252',
      warning:
        'This file was not UTF-8, so we read it as Windows-1252. Check accented characters.',
    };
  }
}
</code></pre>
<p><code>{ fatal: true }</code> is the whole trick. Without it, bad bytes become <code>�</code> forever. With it, you get an exception you can recover from.</p>
<p>Also watch for <strong>mojibake already in the file</strong> (<code>JosÃ©</code>): UTF-8 was once decoded as Latin-1 and saved again. Detect <code>Ã</code>/<code>Â</code> patterns and <em>suggest</em> a repair (re-encode as Latin-1 bytes, decode as UTF-8). Don't auto-apply — show the before/after.</p>
<h2>2. Detect the delimiter from the data</h2>
<pre><code class="language-ts">export function detectDelimiter(text: string): string {
  const lines = text.split(/\r?\n/).filter(Boolean).slice(0, 10);
  let best = ',', bestScore = -1;

  for (const d of [',', ';', '\t', '|']) {
    const counts = lines.map((line) =&gt; {
      let n = 0, inQuotes = false;
      for (const ch of line) {
        if (ch === '"') inQuotes = !inQuotes;
        else if (ch === d &amp;&amp; !inQuotes) n++;
      }
      return n;
    });
    if (!counts[0]) continue;
    const consistent = counts.filter((c) =&gt; c === counts[0]).length;
    const score = consistent * 100 + counts[0];
    if (score &gt; bestScore) { bestScore = score; best = d; }
  }
  return best;
}
</code></pre>
<p>Consistency beats raw frequency. <code>299,00</code> contains commas that vary per row; semicolons usually don't.</p>
<blockquote>
<p>Sampling with <code>split(/\r?\n/)</code> is fine for a quick guess. Your real parser must ignore newlines inside quotes, or a multi-line address in row 2 skews the counts.</p>
</blockquote>
<h2>3. Parse quoted fields (including embedded newlines)</h2>
<p>Don't do this:</p>
<pre><code class="language-ts">// breaks on commas / quotes / newlines inside fields
text.split('\n').map((line) =&gt; line.split(','));
</code></pre>
<p>RFC 4180 essentials: fields may be <code>"..."</code>, a literal quote is <code>""</code>, newlines inside quotes are part of the field. Prefer CRLF when you <em>write</em> CSV (Excel is happier).</p>
<p>Sketch of the state machine:</p>
<pre><code class="language-ts">export function parseCsv(text: string, delimiter = ','): string[][] {
  const src = text.charCodeAt(0) === 0xfeff ? text.slice(1) : text;
  const records: string[][] = [];
  let row: string[] = [], field = '', inQuotes = false, i = 0;

  const endRow = () =&gt; {
    row.push(field); field = '';
    if (!(row.length === 1 &amp;&amp; row[0].trim() === '')) records.push(row);
    row = [];
  };

  while (i &lt; src.length) {
    const ch = src[i];
    if (inQuotes) {
      if (ch === '"') {
        if (src[i + 1] === '"') { field += '"'; i += 2; continue; }
        inQuotes = false; i++; continue;
      }
      field += ch; i++; continue;
    }
    if (ch === '"' &amp;&amp; field === '') { inQuotes = true; i++; }
    else if (ch === delimiter) { row.push(field); field = ''; i++; }
    else if (ch === '\r' || ch === '\n') {
      endRow();
      i += ch === '\r' &amp;&amp; src[i + 1] === '\n' ? 2 : 1;
    } else { field += ch; i++; }
  }
  if (field !== '' || row.length &gt; 0) endRow();
  return records;
}
</code></pre>
<p>If a data row's cell count ≠ header width, surface a <strong>warning</strong>. Unquoted commas in free text are the usual cause — later values are shifted into the wrong columns.</p>
<p>Keep every cell as a <strong>string</strong> through parse. Auto-coercion (<code>dynamicTyping</code>, naive <code>Number(...)</code>) eats leading zeros in IDs and ZIP codes. Let the schema decide types.</p>
<h2>4. Don't reject <code>299,00</code> — suggest <code>299.00</code></h2>
<p>Strict validation is right for your API. Rejecting the whole row with "invalid number" is wrong when the fix is obvious. Separate <em>validating</em> from <em>suggesting</em>:</p>
<pre><code class="language-ts">const NUMBER = /^[+-]?(\d+\.?\d*|\.\d+)(e[+-]?\d+)?$/i;

function suggestNumber(raw: string): string | undefined {
  let s = raw.replace(/[\s$€£¥]/g, '').replace(/^\((.*)\)$/, '-$1');
  if (/^[+-]?\d{1,3}(,\d{3})+(\.\d+)?$/.test(s)) s = s.replace(/,/g, ''); // 1,200.50
  else if (/^[+-]?\d+,\d+$/.test(s)) s = s.replace(',', '.');             // 299,00
  else if (/^[+-]?\d{1,3}(\.\d{3})+,\d+$/.test(s)) s = s.replace(/\./g, '').replace(',', '.'); // 1.234,56
  return NUMBER.test(s) ? s : undefined;
}
</code></pre>
<p>UI copy that works: <code>MRR expects a number, got "299,00". **Use 299.00**</code> + "Apply all in this column". Order the regexes carefully: strip US thousands <em>before</em> treating a lone comma as a decimal, or <code>1,200</code> becomes <code>1.200</code>. European Excel also writes <code>1.234,56</code> (period for thousands, comma for decimal), which needs its own branch. Keep the decimal comma required there, so a bare <code>1.234</code> is never silently rewritten as 1234. (Thanks to QRiQRa Team for catching this.)</p>
<h2>5. Dates: never guess day-first vs month-first</h2>
<ul>
<li><code>14/03/2025</code> → only 14 March is calendar-valid → safe to suggest <code>2025-03-14</code>.</li>
<li><code>05/03/2025</code> → 5 March <em>or</em> 3 May → <strong>flag it</strong>, let the user choose.</li>
</ul>
<p>Rules that don't silently corrupt renewals:</p>
<ol>
<li>Accept only the formats declared on the field.</li>
<li>Validate the calendar (no Feb 30).</li>
<li>Suggest a fix only when <strong>exactly one</strong> other known format yields a single ISO date.</li>
</ol>
<p>European Excel also sends <code>14.03.2025</code>. Support <code>DD.MM.YYYY</code> explicitly if your customers are there — <code>Date.parse</code> will not save you.</p>
<pre><code class="language-ts">// Suggest only when exactly one other format produces one ISO date.
function suggestDate(raw: string, accepted: DateFormat[]): string | undefined {
  if (parseWithAny(raw, accepted)) return undefined;
  const hits = ALL_FORMATS
    .filter((f) =&gt; !accepted.includes(f))
    .map((f) =&gt; tryParse(raw, f))
    .filter(Boolean);
  const unique = [...new Set(hits)];
  return unique.length === 1 ? unique[0] : undefined;
}
</code></pre>
<h2>The pattern</h2>
<table>
<thead>
<tr>
<th>Do</th>
<th>Don't</th>
</tr>
</thead>
<tbody><tr>
<td>Decode bytes strictly, fall back visibly</td>
<td>Trust <code>file.text()</code></td>
</tr>
<tr>
<td>Detect delimiter outside quotes</td>
<td>Hard-code <code>,</code></td>
</tr>
<tr>
<td>Parse quotes / embedded newlines</td>
<td><code>split('\\n').map(split(','))</code></td>
</tr>
<tr>
<td>Validate strictly, suggest fixes</td>
<td>Auto-"fix" ambiguous dates</td>
</tr>
<tr>
<td>Export an error report (row, field, value, fix)</td>
<td>"Row 4812 failed" with no cell</td>
</tr>
</tbody></table>
<p>Client-side checks are UX. <strong>Re-validate on the server</strong> before you write.</p>
<p>Those three takeaways again: detect the delimiter and locale first, treat quoted line breaks as legal CSV, and let the uploader fix the parse result in the browser before you submit.</p>
<h3>Try it / go further</h3>
<p>You can copy the snippets above into any React or vanilla TypeScript app. To see the same German Excel traps in the browser — semicolon delimiters, decimal commas, and quoted line breaks — try the <a href="https://xiaoyangge1993.github.io/patchrow-demo/?utm_source=hashnode&amp;utm_medium=referral&amp;utm_campaign=republish">live demo</a>. It runs locally in your browser and does not upload the file.</p>
<p>If you'd rather drop in a guided flow (upload → map columns → cell-level errors → one-click fixes → export) instead of assembling the pieces, I packaged this approach as <a href="https://xiaoxiaoyang.gumroad.com/l/usgiy?utm_source=hashnode&amp;utm_medium=referral&amp;utm_campaign=republish"><strong>PatchRow</strong></a> — a React + TypeScript CSV import kit (source you own, one-time $29). <em>Disclosure: it's my product.</em> It does not parse <code>.xlsx</code>; users export CSV.</p>
<p><strong>Question for the comments:</strong> what's the worst CSV your import has eaten — encoding, dates, or something weirder? I read every reply.</p>
]]></content:encoded></item></channel></rss>