Tutorial: Creating Custom Configs

Learn how to create custom configuration files for documentation websites not covered by presets.

Time: 15 minutes | Level: Intermediate | Result: Working custom config

Interactive Config Creation

The easiest way to create a config:

skill-seekers scrape --interactive

Follow the prompts:

1. Enter base URL
2. Test selectors on sample pages
3. Verify extracted content
4. Save config

Manual Config Creation

Create configs/my-framework.json:

{
  "name": "my-framework",
  "base_url": "https://docs.my-framework.com/",
  "selectors": {
    "content": "article.documentation",
    "title": "h1.page-title",
    "code": "pre code"
  },
  "url_patterns": [
    "^https://docs.my-framework.com/guide/",
    "^https://docs.my-framework.com/api/"
  ],
  "exclude_patterns": [
    "/changelog/",
    "/blog/"
  ],
  "max_pages": 200,
  "rate_limit": 0.5
}

Test Your Config

# Estimate page count
skill-seekers estimate --config configs/my-framework.json

# Test on first 10 pages
skill-seekers scrape \
  --config configs/my-framework.json \
  --max-pages 10 \
  --output output/test/

Share Your Config

# Submit to community
skill-seekers submit-config \
  --config configs/my-framework.json \
  --description "My Framework documentation config"

See: Config Format Reference for all available options.