Plain words, not legal advice
Is it legal to scrape a business directory?
Often, yes — but not always, and not for every use. It comes down to three questions: what the directory's own rules say, whether the listings contain personal data, and how you contact the businesses afterwards. Here is each one in plain words, with the sources.
This page is general information, not legal advice. Laws differ by country and change; for a decision that matters, ask a lawyer where you are.
1. The directory's own rules
Most directories have terms of use, and some say plainly that automated collection is
not allowed. Read them before you collect from a site you will use often. A site's
robots.txt file also says which parts it would rather automated tools left
alone.
In the US, courts have generally not treated reading pages anyone can see as “unauthorised access” under federal anti-hacking law — but a site's terms can still bind you as a contract, so breaking them is a risk of its own. Getting past a login, a paywall or a “prove you're human” check is a different and much riskier matter, which is why the Magic Scraper extension never does it.
2. Company details or personal data?
A company's name, address, main phone and website are business information. A named person — a sole trader, or an employee's name and email — is personal data, and in the EU and UK that brings GDPR into play even when the details are public.
France's data-protection authority, the CNIL, has said that people who publish their contact details online do not expect them to be collected for sales prospecting, and that scraped contact data still needs a lawful basis to be reused (summary of the CNIL's guidance). In January 2025 it fined Kaspr, a contact-data company, €240,000 for collecting contact details without a valid legal basis or enough transparency (CNIL).
The simplest way to stay on the right side: collect company-level fields, leave out named people unless you have a clear reason, and keep only what you will use.
3. Contacting the businesses afterwards
Collecting a list and emailing or calling everyone on it are covered by different rules, and the second is where most trouble starts.
UK. You may send marketing emails to companies (“corporate subscribers”) without their prior consent, if you say who you are and offer a way to opt out. Sole traders and some partnerships are treated like individuals: you need their consent, or an existing customer relationship (ICO guidance).
EU. Rules on marketing email vary by country; several require consent even for business addresses. Check the country you are emailing into.
US. Commercial email must follow the CAN-SPAM Act: no misleading subject lines, a real postal address, and a working unsubscribe that you honour (FTC guide). Calls and texts are stricter, especially to mobile numbers.
A short checklist
Read the terms
Of the directory you collect from, especially if you'll use it regularly.
Keep to business details
Company, address, main phone, website. Leave out named people unless you need them.
Go at a human pace
One page at a time, with pauses. Stop if the site asks you to.
Contact people properly
Say who you are, why you're writing, and honour every opt-out.
What the Magic Scraper extension does about it
It reads only pages you open, in your own browser, at a human pace with a randomised pause between pages. If a site answers “slow down”, it waits. If a site asks you to prove you're human, the run stops and hands you the tab; it never tries to get past a check, a login or a paywall. What you collect stays on your computer — there is no server of ours for it to pass through.
How you use the list is up to you, and the rules above are yours to follow.