htmlLimitedBots

Source URL: https://nextjs.org/docs/app/api-reference/config/next-config-js/htmlLimitedBots

htmlLimitedBots

The htmlLimitedBots config allows you to specify a list of user agents that should receive blocking metadata instead of streaming metadata.

import type { NextConfig } from 'next'

const config: NextConfig = {
  htmlLimitedBots: /MySpecialBot|MyAnotherSpecialBot|SimpleCrawler/,
}

export default config

module.exports = {
  htmlLimitedBots: /MySpecialBot|MyAnotherSpecialBot|SimpleCrawler/,
}

Default list

Next.js includes a default list of HTML limited bots, including:

• Google crawlers (e.g. Mediapartners-Google, AdsBot-Google, Google-PageRenderer)
• Bingbot
• Twitterbot
• Slackbot

See the full list here.

Specifying a htmlLimitedBots config will override the Next.js’ default list. However, this is advanced behavior, and the default should be sufficient for most cases.

const config: NextConfig = {
  htmlLimitedBots: /MySpecialBot|MyAnotherSpecialBot|SimpleCrawler/,
}

export default config

module.exports = {
  htmlLimitedBots: /MySpecialBot|MyAnotherSpecialBot|SimpleCrawler/,
}

Disabling

To fully disable streaming metadata:

import type { NextConfig } from 'next'

const config: NextConfig = {
  htmlLimitedBots: /.*/,
}

export default config

module.exports = {
  htmlLimitedBots: /.*/,
}

Version History

Version	Changes
15.2.0	htmlLimitedBots option introduced.

---