Current facts
A live fetch may retrieve recent information that a static index has not processed.
Automatic crawling and user-requested retrieval
Perplexity Search and Crawler Access
Perplexity documents separate user agents for search discovery and user-triggered page access, with practical WAF guidance for legitimate requests.
Search discovery and linking
Perplexity describes PerplexityBot as a crawler used to surface and link websites in search results. It is managed through robots.txt and should also be allowed through infrastructure controls when a publisher wants search visibility.
Retrieval initiated by a user request
Perplexity-User may visit a page when a user asks a question that requires current information. Perplexity distinguishes this from automatic web crawling and states that the user-requested fetcher generally does not rely on robots rules in the same way.
The distinction matters because a site can observe different access patterns and should not assume every Perplexity request is building the same index.
Verify user agent and official IP ranges
Perplexity recommends combining user-agent matching with its published IP ranges when configuring a web application firewall. The ranges can change, so the official endpoints should remain the source of truth.
A robots file can permit the bot while a WAF continues to block it. Server logs and external testing are required to confirm access.
Real-time retrieval changes freshness expectations
Page stability
A transient error, regional block or script failure can affect the answer at the moment of the request.
Citation
The interface may link to a page, but source visibility and generated wording remain separate outcomes.
Monitoring
Logs, referral traffic and repeated public observations provide different evidence.
Make permission observable
- Allow PerplexityBot when search discovery is desired.
- Use official IP data in strict WAF environments.
- Return a complete public response without a browser-only challenge.
- Keep freshness-sensitive facts visible and dated.
- Compare the generated statement with the linked source.
Primary material behind this analysis
Perplexity Crawlers
Official roles and access guidance for PerplexityBot and Perplexity-User.
Continue through the discovery stack
Crawling, robots.txt and Indexing Explained
A technical guide to discovery, crawler access, robots.txt, infrastructure blocks, directives, canonical URLs, sitemaps and indexing.
Information Retrieval Explained
A practical guide to lexical, semantic, hybrid, graph and structured retrieval, failure modes, passage design and evaluation.
Search and AI Visibility Measurement Framework
A layered measurement framework for crawler access, indexing, entity recognition, retrieval, citations, representation accuracy and business outcomes.
Search and AI Research Library
A research hub connecting NIST, Stanford HAI, OECD, Google, OpenAI, Perplexity, Microsoft, IETF, W3C, Schema.org and original GEO, RAG and REG sources.