Admissions open Take the eligibility test & see if you can join the next batch. Take the test →

Decoding the Jargon

Crawling

Also called crawl, spidering

SEOTerm

The step where Google's automated programs (Googlebot) discover pages and download them, mostly by following links and reading sitemaps.

Example

Meera publishes a new 'Festive silk kurtas' page. Googlebot finds it through a link from her home page and her sitemap, and downloads it. That is crawling; it does not mean the page is in Google yet.

How you'll hear it

“The page was crawled last week but is still not indexed, so the problem is quality or duplication, not discovery.”

Don't confuse it with

Indexing: storing and understanding the page after it is crawled. A page can be crawled and not indexed.

Want to use this at work? AI-led SEO Excellence is aFactor's 16-week program for SEO work.

Sources

In-depth guide to how Google Search works, Google Search Central

Checked 1 October 2026. Examples use a fictional business; figures are illustrative.

Send this to someone

Knowing the words is the start. Using them well is the job.

The role-fit test shows how your judgment lines up with the role you hold or the one you want next.

Take the role-fit test →