User-agent: * Disallow: /usgs/ User-agent: usgs-gsa-crawler Disallow: