Build a sitemap that describes your site
A sitemap lists permanent URLs you want crawlers to discover. Use the exact HTTPS address that returns useful content, rather than session URLs, priva…
Guides
Practical checks with examples, limits and the next action.
Find and fix sitemap errors
A file can be well-formed XML and still contain poor entries. Start by inspecting missing loc values, repeated URLs, invalid schemes and date formats.…
Test robots.txt rules with real paths
Robots.txt controls crawling for its own protocol, host and port. Place it at the root, such as https://example.com/robots.txt. A rule in a different …
Separate crawling from indexing
An allowed crawl does not mean a page is indexed. Robots.txt tells a crawler whether it can request the page; noindex is an indexing instruction in pa…
Review canonical without hiding useful pages
A canonical link describes the preferred address for similar content. A self-referencing canonical is common on unique permanent pages. Use one link i…
Check hreflang declarations and their counterparts
Hreflang relates language and regional versions of a page. Use language codes such as en or tr, with a region only when you actually target it, such a…