User-agent: * Disallow: /site.php Disallow: /custom/ Allow: /