← 返回文章

Static Site SEO: Sitemap, Robots.txt, and Ads.txt Essentials

guide 2026/3/18 6 min read 分钟阅读

中文版本整理中，当前提供英文原文。

Static Site SEO: Sitemap, Robots.txt, and Ads.txt Essentials

Every static site needs proper SEO configuration. This guide covers the essential files: sitemap.xml, robots.txt, and ads.txt.

Why SEO Files Matter

Proper SEO configuration helps:

  • Search engine discovery - Crawlers find your pages
  • Index control - Specify what to include/exclude
  • Monetization - Enable AdSense properly
  • Credibility - Signal legitimate publishing

Sitemap.xml

A sitemap lists all pages you want indexed.

Basic Structure

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://yoursite.com/</loc>
    <lastmod>2026-03-18</lastmod>
    <changefreq>weekly</changefreq>
    <priority>1.0</priority>
  </url>
</urlset>

Best Practices

  1. List all important pages - Homepage, main sections, key content
  2. Update regularly - Keep lastmod current
  3. Prioritize wisely - Higher priority for important pages
  4. Don’t include - 404 pages, redirects, low-value pages

Common Mistakes

  • Including broken links
  • Not updating after content changes
  • Listing pages with noindex
  • Excluding the sitemap from robots.txt

Robots.txt

Robots.txt controls crawler behavior.

Basic Structure

User-agent: *
Allow: /
Disallow: /private/
Disallow: /admin/

Sitemap: https://yoursite.com/sitemap.xml

Key Directives

  • Allow - What crawlers can access
  • Disallow - What to block
  • User-agent - Specific crawler rules
  • Sitemap - Point to your sitemap

Best Practices

  1. Allow public content - Don’t accidentally block indexable pages
  2. Block private areas - Admin, login, private directories
  3. Reference sitemap - Help crawlers find it
  4. Test thoroughly - Use Google Search Console

Ads.txt

Ads.txt is required for AdSense monetization.

Basic Structure

google.com, pub-XXXXXXXXXXXXXXXXXXX, DIRECT, f08c47fec0942fa0

Requirements

  1. Get your publisher ID - From AdSense dashboard
  2. Create the file - At yoursite.com/ads.txt
  3. Include correct line - Your specific publisher entry
  4. Don’t modify - Only add, never change existing lines

Common Issues

  • Wrong publisher ID
  • File in wrong location
  • Multiple conflicting entries
  • Not waiting for propagation

Implementation Order

  1. Create sitemap.xml - List all public pages
  2. Create robots.txt - Allow crawling, reference sitemap
  3. Create ads.txt - If using AdSense
  4. Submit to search engines - Google Search Console, Bing Webmaster Tools

Verification

Check your setup:

  • Sitemap - Submit to Google Search Console
  • Robots - Use robots.txt tester
  • Ads.txt - Check AdSense dashboard status

Conclusion

Proper SEO configuration is essential for static sites. A correct sitemap helps crawlers find your content, robots.txt controls what’s indexed, and ads.txt enables monetization. Take time to configure these files correctly.