# Everything is open to every crawler. # An empty Disallow value is what grants full access: the line must be # present, but it must have nothing after the colon. User-agent: * Disallow: