> ## Documentation Index
> Fetch the complete documentation index at: https://docs.siftly.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Vercel / Netlify Middleware
> Add a single middleware file to any Next.js, SvelteKit, Nuxt, or Astro project to track AI bot traffic.
## How It Works
Framework middleware intercepts every incoming request **before** your page code runs. When an AI bot is detected, the bot's metadata is sent to the traffic-ingest-service as a fire-and-forget background fetch — zero latency added to real user loads.
## Setup
Add the following file to the **root** of your project (same level as `package.json`):
```ts middleware.ts (Next.js) theme={null}
import { NextRequest, NextResponse } from 'next/server';
const AI_BOT_PATTERNS = [
// OpenAI — citations & training
'OAI-SearchBot', 'ChatGPT-User', 'GPTBot',
// Anthropic / Claude — citations & training
'Claude-Web', 'Claude-User', 'ClaudeBot', 'anthropic-ai',
// Perplexity
'PerplexityBot', 'Perplexity-User',
// Google / Gemini — citations & training
'Googlebot', 'Google-Extended', 'Google-Gemini',
// Microsoft / Bing
'bingbot', 'BingPreview',
// Meta
'Meta-ExternalAgent', 'FacebookBot',
// Others
'Bytespider', 'cohere-ai', 'CCBot',
// Traditional search
'YouBot', 'DuckDuckBot', 'Baiduspider',
];
const LOG_ENDPOINT = '/log/vercel';
const ORGANIZATION_ID = '';
export function middleware(request: NextRequest) {
const ua = request.headers.get('user-agent') ?? '';
const isBot = AI_BOT_PATTERNS.some((p) =>
ua.toLowerCase().includes(p.toLowerCase())
);
if (isBot) {
const payload = {
timestamp: new Date().toISOString(),
bot_ua: ua,
url: request.url,
method: request.method,
ip: request.headers.get('x-forwarded-for') ?? request.headers.get('x-real-ip'),
country: request.headers.get('x-vercel-ip-country'),
city: request.headers.get('x-vercel-ip-city'),
};
// Fire-and-forget — never awaited, does not block the response
fetch(LOG_ENDPOINT, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${ORGANIZATION_ID}`,
},
body: JSON.stringify(payload),
}).catch(() => {});
}
return NextResponse.next();
}
export const config = {
// Run on all routes except Next.js internals and static files
matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
};
```
```js middleware.js (SvelteKit / Nuxt / Astro — server hooks) theme={null}
// SvelteKit: place in src/hooks.server.js
// Nuxt: place in server/middleware/bot-tracker.js
// Astro: place in src/middleware.js
const AI_BOT_PATTERNS = [
// OpenAI — citations & training
'OAI-SearchBot', 'ChatGPT-User', 'GPTBot',
// Anthropic / Claude — citations & training
'Claude-Web', 'Claude-User', 'ClaudeBot', 'anthropic-ai',
// Perplexity
'PerplexityBot', 'Perplexity-User',
// Google / Gemini — citations & training
'Googlebot', 'Google-Extended', 'Google-Gemini',
// Microsoft / Bing
'bingbot', 'BingPreview',
// Meta
'Meta-ExternalAgent', 'FacebookBot',
// Others
'Bytespider', 'cohere-ai', 'CCBot',
// Traditional search
'YouBot', 'DuckDuckBot', 'Baiduspider',
];
const LOG_ENDPOINT = '/log/vercel';
const ORGANIZATION_ID = '';
export async function handle({ event, resolve }) {
const ua = event.request.headers.get('user-agent') ?? '';
const isBot = AI_BOT_PATTERNS.some((p) =>
ua.toLowerCase().includes(p.toLowerCase())
);
if (isBot) {
const payload = {
timestamp: new Date().toISOString(),
bot_ua: ua,
url: event.request.url,
method: event.request.method,
ip: event.request.headers.get('x-forwarded-for'),
country: event.request.headers.get('x-vercel-ip-country'),
city: event.request.headers.get('x-vercel-ip-city'),
};
fetch(LOG_ENDPOINT, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
Authorization: `Bearer ${ORGANIZATION_ID}`,
},
body: JSON.stringify(payload),
}).catch(() => {});
}
return resolve(event);
}
```
Replace the two placeholder values in the file:
| Placeholder | Value |
| ------------------------------ | ------------------------------------------------------------- |
| `` | The URL of the traffic-ingest-service (ask your Siftly admin) |
| `` | Your Organisation ID from the Siftly dashboard |
Commit and push (or run `vercel deploy`). The middleware activates automatically on every incoming request.
No additional environment variables, build steps, or packages are required.
## Netlify
For Netlify, use an [Edge Function](https://docs.netlify.com/edge-functions/overview/) instead of middleware:
```js netlify/edge-functions/bot-tracker.js theme={null}
export default async (request, context) => {
const AI_BOT_PATTERNS = [
// OpenAI — citations & training
'OAI-SearchBot', 'ChatGPT-User', 'GPTBot',
// Anthropic / Claude — citations & training
'Claude-Web', 'Claude-User', 'ClaudeBot', 'anthropic-ai',
// Perplexity
'PerplexityBot', 'Perplexity-User',
// Google / Gemini — citations & training
'Googlebot', 'Google-Extended', 'Google-Gemini',
// Microsoft / Bing
'bingbot', 'BingPreview',
// Meta
'Meta-ExternalAgent', 'FacebookBot',
// Others
'Bytespider', 'cohere-ai', 'CCBot',
// Traditional search
'YouBot', 'DuckDuckBot', 'Baiduspider',
];
const LOG_ENDPOINT = '/log/vercel';
const ORGANIZATION_ID = '';
const ua = request.headers.get('user-agent') ?? '';
const isBot = AI_BOT_PATTERNS.some((p) => ua.toLowerCase().includes(p.toLowerCase()));
if (isBot) {
const payload = {
timestamp: new Date().toISOString(),
bot_ua: ua,
url: request.url,
method: request.method,
ip: request.headers.get('x-nf-client-connection-ip'),
country: context.geo?.country?.code,
city: context.geo?.city,
};
context.waitUntil(
fetch(LOG_ENDPOINT, {
method: 'POST',
headers: { 'Content-Type': 'application/json', Authorization: `Bearer ${ORGANIZATION_ID}` },
body: JSON.stringify(payload),
}).catch(() => {})
);
}
return context.next();
};
```
Add the edge function route in `netlify.toml`:
```toml netlify.toml theme={null}
[[edge_functions]]
path = "/*"
function = "bot-tracker"
```
## Verifying It Works
After deploying, visit your site with a spoofed User-Agent to confirm events appear in Siftly:
```bash theme={null}
curl -A "GPTBot/1.0" https://yoursite.com
```
Check the **Traffic** section of the Siftly dashboard within a few minutes.