Scraping a site that isn't in English.

Most big sites mark their Next button in ways that do not depend on the language: a rel="next" link, an arrow, an aria-label, a numbered pager. Magic Scraper reads all of those, which is why Amazon.co.jp, Kleinanzeigen and LDLC crawl the same as an English shop. The word only matters on sites that write nothing else, and there it reads the word, in more than 80 languages, and the Load more button too.

† With the next update, 2.0.2 next update. The rest is in the stores now. In all, Next and Previous are read in 87 languages and Load more in about 68: every major European language, and Turkish, Azerbaijani, Indonesian, Malay, Filipino, Vietnamese, Swahili, Arabic, Persian, Urdu, Hebrew, the languages of India from Hindi and Bengali to Tamil and Malayalam, Sinhala, Thai, Lao, Khmer, Burmese, Georgian, Armenian and Amharic. Previous-page words are read too, so Zurück, Anterior or 前へ is never taken for Next. And outside Latin script the word has to come first, so 目次 (“contents”), Читать далее (“read more”) and आगे पढ़ें (“read on”) are never taken for it either.

Excel guesses a CSV file's encoding, and without a hint it guesses wrong: Müller becomes Müller and 東京 becomes æ±äº¬. Magic Scraper's CSV starts with the byte-order mark that tells Excel the file is UTF-8, so the letters arrive as the page wrote them. The Excel (.xlsx) export avoids the question altogether, and Google Sheets reads either.

A CSV from another tool that is already garbled can be rescued: in Excel, use Data → From Text/CSV and set File origin to 65001: Unicode (UTF-8).

Most of Europe writes 1.234,56 where English writes 1,234.56, and France writes 1 234,56. A spreadsheet that reads the dot as a decimal point turns a German table's 1.234 (one thousand two hundred and thirty-four) into 1.234, a thousand times too small, with nothing to say so.

From the next update next update, the Excel export looks at the whole column. When it shows European signs (83.200.000, 12,50, 1.234,56) every value is read that way, so the column sums. Thousands written apart with a space, the French way, are read in any column. A column of nothing but 1.234-style values could be either, so it stays text: never wrong, only not summable. The CSV keeps every value exactly as the page wrote it.

A table's own headers come through as written: 価格, Preis, Precio. Where a page has no headers, columns are named from the markup, and those names are English: Price, Link, Image. Double-click a header in the popup, or press F2, to rename it; the name is remembered for that site.

The extension's own buttons and messages are in English. The data never is.

Before this page was written, the extension ran on Amazon.co.jp search, 価格.com, Yahoo!ニュース and はてなブックマーク; Kleinanzeigen, Chefkoch, heise online and Gelbe Seiten; Allociné, Marmiton and LDLC; and the German, Spanish and French Wikipedias. Some sites showed a “prove you're human” check instead. On those it stops and hands you the tab, and never tries to get past the check.

In your language: 日本語 · Deutsch · Español · Français · English.