SEO Glossary
What is SEO, and how does it work?
INDEXING SITE RANKING TRAFFIC WEB RESULTS
OPTIMIZED
Terms by Letter:
#   A   B   C   D   E   F   G   H   I   J   K   L   M   N   O   P   Q   R   S   T   U   V   W   X   Y   Z   All

Web spider/ Web crawler

Web crawlers or more usually known as web spiders, or Internet robots (or just called bots). They are software applications that run tasks on the Internet. These tasks are simple and repetitive. Search engines use web crawlers to fetch, analyze and file the information gathered from web servers. Web crawlers can also be used to check links or validate a HTML code.

Web crawlers can copy all the pages they visit and leave it for further/later processing by the search engine.

Back to Glossary