Technical SEO tools audit the layer of your site that search engines deal with before content ever matters: crawlability, indexing, speed, rendering, and structured data. They surface the broken links, redirect chains, blocked pages, and slow templates that quietly cap rankings. This post is a working-stack piece, not a countdown: it sorts the tools by the job each class does, names the ones we actually run, and is honest about the part vendors skip, which is that free tools cover most of the job for most sites. Nothing here is an affiliate link and nobody paid to be named. The aim is that you finish knowing exactly where free stops being enough for your site, and what to buy at that point.
The short version
- Google Search Console is mandatory for every site, and it costs nothing.
- Screaming Frog is the standard crawler; the free tier handles 500 URLs.
- PageSpeed Insights plus Chrome DevTools cover the whole speed workflow.
- Paid buys convenience, history, and scale, not capability free lacks.
- Enterprise crawlers and log analyzers earn their cost only on large, complex sites.
Search Console comes before everything you pay for
Google Search Console is the only tool in this post that shows how Google itself sees your site, which is why it opens every audit we run. The page indexing report says which URLs are indexed and names the reason for every one that is not. The crawl stats report shows what Googlebot requests, how often, and with what response codes. The Core Web Vitals report carries real field data from Chrome users, the numbers rankings actually use, and the manual actions panel is the only place a penalty announces itself. The robots.txt report shows the exact file Google fetched, a check our robots.txt post walks through in detail. Recent versions added quality-of-life features worth turning on: annotations that pin notes to dates on the performance chart, so a migration or a redesign stays visible next to the traffic it caused, weekly and monthly performance views, and a branded query filter that splits brand searches from the growth you are actually working for. Turn on email alerts before doing anything else, so indexing problems reach you the day they happen, not the month traffic falls.
Desktop crawlers do the everyday audit work
A crawler walks your site the way a search engine does and reports what it finds, and a desktop crawler is the workhorse of the whole discipline. Screaming Frog SEO Spider is the industry standard for a reason: it finds broken links, redirect chains, duplicate and missing titles, thin pages, orphaned URLs, canonical problems, and hreflang errors, and the free tier crawls 500 URLs, enough for most small business sites. The paid license, a few hundred dollars a year, removes the limit and adds the features practitioners lean on daily, JavaScript rendering for sites that build pages in the browser, scheduled crawls, and direct connections to the Search Console and Analytics APIs so crawl data and traffic data land in one table. Sitebulb is the other desktop crawler worth naming, and its pitch is different: it explains findings in plain language and scores them, which makes it strong for client reporting and for teams still building crawl-reading instincts. Frog gives you the raw data faster; Sitebulb tells you what the data means.
Cloud crawlers add scheduling, history, and a shared view
The site audit tools inside Ahrefs and Semrush crawl from the cloud on a schedule, track issues over time, and give the whole team the same dashboard, none of which a desktop crawler tied to one laptop does gracefully. That is genuinely worth something on sites that change weekly. The honest tradeoff: both suites price on their keyword and backlink data, so if technical auditing is your only need, you are paying suite rates for a crawler that inspects less deeply than a Frog license costing a fraction as much. Buy a suite for the research data and treat its site audit as a monitoring layer that emails you when something regresses between your real crawls. That combination, deep desktop crawls plus shallow scheduled cloud crawls, is what most working practitioners actually run.
Enterprise platforms earn their cost only at real scale
Lumar, the platform formerly known as DeepCrawl, and Botify sell crawling for sites measured in millions of URLs, with log file ingestion, crawl-budget analysis, and workflow features for teams where SEO findings become engineering tickets. On a large ecommerce or publisher site, pairing what Google actually crawls against what the site serves is exactly how you find the faceted-navigation traps and parameter explosions that waste crawl budget at scale. Under roughly ten thousand pages, that class of problem barely exists, and the subscription, typically thousands per month on an annual contract, buys reporting you could have had from a desktop crawler and a weekend. The honest test is not company size but URL count and architecture complexity.
Speed tools split into lab and field
Speed testing has two kinds of numbers, and confusing them wastes entire quarters. Lab numbers come from a synthetic test machine and exist for debugging. Field numbers come from real Chrome visitors through the Chrome UX Report and are what Core Web Vitals assessments actually use. PageSpeed Insights shows both for any URL, field data on top when the site has enough traffic, lab results below, and it is free with no setup. Chrome DevTools is the debugging environment behind the lab number: the Performance panel records exactly what happens during a load, Lighthouse runs the audit locally, and device mode handles the mobile checks that Google's retired Mobile-Friendly Test and the old Search Console mobile usability report used to cover. WebPageTest remains the deep end, with filmstrip views, connection throttling, and test locations the simpler tools lack. Our post on Largest Contentful Paint shows what this workflow looks like pointed at one metric.
Log files show what Googlebot actually does
Every request to your server, including every Googlebot fetch, lands in the access logs, which makes logs the only ground truth about crawling that does not come filtered through a report. Log analysis answers questions nothing else can: which sections Googlebot ignores, how much crawling gets spent on parameters and redirects, whether the AI crawlers you allowed actually visit. Screaming Frog's Log File Analyser handles this at desktop scale, Botify and Lumar build it in at enterprise scale, and for sites behind Cloudflare, its analytics offer a lighter substitute when raw log access is awkward to arrange, which on shared hosting it usually is. The honest scoping: below roughly a hundred thousand URLs, the crawl stats report in Search Console tells you most of what logs would, with none of the setup.
Validators catch the errors nothing else reports
Structured data fails silently. A schema error costs you rich results without breaking anything a visitor or a crawl report would notice, which is why validators stay in the stack despite being the least glamorous tools in it. The Rich Results Test checks whether a page qualifies for Google's rich result types and pinpoints the failing property. The Schema Markup Validator, which lives at schema.org, checks any structured data against the vocabulary itself and is stricter about syntax. Run key templates through both after every markup change, because a template-level error multiplies across every page built from it.
Pick your stack by site size
| Site size | Stack | Cost |
|---|---|---|
| Under 500 pages | Search Console, Screaming Frog free tier, PageSpeed Insights, Chrome DevTools | Free |
| 500 to 10,000 pages | Add a Screaming Frog license, plus Semrush or Ahrefs if you also need keyword and link data | Low hundreds per month |
| 10,000+ pages or heavy faceting | Enterprise crawling (Lumar, Botify) plus log file analysis | Enterprise pricing |
| JavaScript-heavy at any size | Any crawler with JS rendering, verified against the URL Inspection tool | Varies |
The pattern to resist is buying an enterprise platform for a 200-page site, and its quieter cousin, keeping three overlapping subscriptions because each one was justifiable alone. Tool spend should follow crawl complexity. Most small and mid-size sites never outgrow the middle row, and the honest vendors will tell you the same.
The workflow we actually run
-
Read Search Console before crawling anything
The page indexing report tells you what Google can and cannot see, and a blocked or noindexed money page outranks every other finding in urgency. Starting with the crawl instead means an hour of setup before you learn what mattered.
-
Crawl the site and compare against the sitemap
Screaming Frog or Sitebulb surfaces broken links, chains, duplicate titles, canonical conflicts, and orphan pages. URLs in the crawl but not the sitemap, and the reverse, are both findings.
-
Test speed on the templates that earn money
Run PageSpeed Insights on the homepage, a service or product page, and a blog post. Template-level fixes move numbers sitewide; page-by-page fixes never end.
-
Validate structured data on key templates
One pass through the Rich Results Test per template. Schema errors are invisible everywhere else.
-
Check the logs when the reports disagree
If Search Console says pages are discovered but not crawled, or crawl stats look strange, logs settle the argument. This step is skippable on small sites and decisive on big ones.
-
Fix in severity order and re-crawl
Indexing blockers first, then broken links and chains on important pages, then Core Web Vitals, then markup polish. Severity beats ease of fixing, and the re-crawl proves the fix instead of assuming it.
Where Egochi fits
Egochi runs this exact stack daily as part of our technical SEO services: Search Console first, deep desktop crawls, speed work read from field data, and log analysis where scale justifies it, delivered as a prioritized fix list rather than a raw data export, and implemented by the same team that found the problems. The tools are the easy part, which is the last honest thing this post owes you. Every tool here produces findings; the results come from knowing which ten of the four hundred findings are worth an engineer's week.
Questions people ask about technical SEO tools
What is the most important technical SEO tool?
Google Search Console, and it is not close. It is free, it comes straight from Google, and it shows indexing status, crawl stats, Core Web Vitals field data, and manual actions that no third-party tool can see. Verify it before buying anything.
Is Screaming Frog free?
The free version crawls up to 500 URLs, which covers most small sites. The paid license removes the limit and adds JavaScript rendering, scheduled crawls, saved configurations, and Google API connections. For most practitioners it is the first paid tool worth buying.
Can I do technical SEO with only free tools?
For sites under about 500 pages, yes. Search Console, the free Screaming Frog tier, PageSpeed Insights, and Chrome DevTools cover indexing, crawling, speed, and debugging. What you give up is convenience, history, and scale, not core capability.
Do I need a log file analyzer?
Only at scale. If your site has hundreds of thousands of pages, faceted navigation, or stubborn indexing problems, logs show exactly what Googlebot requests and skips. Smaller sites get what they need from the crawl stats report in Search Console.
Are the paid all-in-one suites worth it for technical SEO alone?
Usually not. Ahrefs and Semrush earn their subscriptions on keyword and link data, with site audits included as a bonus. If technical auditing is the only need, a Screaming Frog license costs a fraction of a suite subscription and crawls deeper.
How often should I run a technical audit?
A full crawl quarterly, a scheduled crawl monthly, and a Search Console check weekly. After anything big, a migration, a redesign, a platform change, crawl immediately, because that is when accidental noindex tags and broken redirects slip through.
