How LLMs Do Web Search

Remember when “Just Google it” was the answer to everything? Now it’s “Just ask AI”.

We’ve come a really long way from searching for hours for a particular piece of information in google search, blogs, forums to few seconds of answer generation by complex LLM models like ChatGPT, Claude, and others. But let’s not talk about the history here. In this article, we’ll discuss how LLM models are able to browse the internet, visit websites, webpages, and are on the way to replace Google search.

Source: Image generated using SORA

This article is divided into multiple sections to help understand the concept clearly. This article is more of a higher-level abstraction than the technical implementation.

A. First, Let’s Understand The ABCs

You can skip this section if you are already familiar with tokenization, function calling, tool calling, and LLMs are.

What Are LLMs?

Let’s start with LLM (Large Language Models). LLMs are transformer-based models trained on massive amounts of data obtained from the internet and various sources that companies utilize. Think of them as incredibly sophisticated pattern recognition systems that have read basically the entire internet and learned how language works.

These models don’t actually “understand” text the way humans do. Instead, they understand in tokens, think in tokens, breathe and speak in tokens. With all the trainings and finetuning they have become really, really good at predicting what words should come next in a sequence.

Tokenization: Breaking It Down

Before an LLM can work with text, it needs to break it down into smaller pieces called tokens. Tokenization is like chopping up a sentence into digestible chunks. For example: “Hello world!” might become [“Hello”, “ world”, “!”] — each piece is a token that the model can work with.

This matters for web search because when an LLM processes a webpage, it’s not seeing the full HTML as one big blob. It’s seeing thousands of individual tokens that it can analyze and understand piece by piece.

You can see how token are made from normal text in this site: https://tiktokenizer.vercel.app/

Function and Tool Calling: The Hands and Legs for LLMs

Here’s where things get interesting. Modern LLMs don’t just generate text — they can also “call functions” or use “tools.” Think of it like giving your AI assistant a toolkit.

When you ask an AI to search the web, it doesn’t magically know how to browse the internet. Instead, it has access to a web search tool (like a search API) that it can call. The process works like this:

1. You ask: “What’s the weather like today?”
2. The LLM thinks: “I need current weather data, so I should use my web search tool”
3. It calls the search function with a query like “weather today”
4. The search tool returns results
5. The LLM processes those results and gives you a human-friendly answer

It’s like having a really smart assistant who knows exactly which tool to grab from the toolbox for each job. You can make your own tools using this feature and give the LLMs more power to perform tasks.

B. The Overall Flow: How LLMs Actually Search the Web

Now, Let`s dive into the flow of the overall web search— how does this whole web searching thing actually work?

1. The Decision Process

First, whenever you ask any specific question which is not in the training data of the LLM and require the web, it needs to decide if it should search the web at all. This isn’t as simple as it sounds. The model has been trained to recognize when its existing knowledge is sufficient versus when it needs fresh information. The labelers have trained the models on a lot of data which makes them catch the pattern and decide whether it needs to browse the web or not.

For example, if you ask “What’s the capital of Nepal?” — the LLM knows this is stable information that hasn’t changed, so it won’t search. But if you ask “What’s happening in the stock market today?” — now it is the time for the web search.

 Example of Training Data on Function Calling. Source: glaiveai/glaive-function-calling-v2

2. The Search Query Generation

When the LLM decides to search, it doesn’t just copy-paste your question into a search engine. It’s been trained to craft effective search queries. Your question “What people are talking about that popular cute Japanese office lady in a suit ” becomes a clean search query like “Saori Araki reviews.”

This training involved showing the model millions of examples of good vs. bad search queries and their results. This process makes the model learn how to search properly. The model learned that shorter, keyword-focused queries often work better than natural language questions.

3. The Search Process

The LLM typically uses a search API (like ChatGPT uses Bing Search API , Gemini uses Google Custom Search, or specialized search services) rather than literally opening a web browser. These APIs return structured data in particularly JSON format including:

Sometimes metadata like publication dates

• Titles of web pages
• Brief snippets/descriptions
• URLs
• Sometimes metadata like publication dates

Source: https://www.ml6.eu/blogpost/how-llms-access-real-time-data-from-the-web

Custom Sites vs. Open Web Browsing

Remember that you can also enter the website URL itself to ask. But, there is a big difference between how LLMs handle pre-specified sites versus how they search the sites on their own i.e. open web browsing:

Custom Sites (Given in Prompt): When you give an LLM specific websites to check, it’s like giving it a reading list. The model will:

• Directly fetch content from those URLs
• Parse the HTML to extract meaningful text
• Treat these sources as more authoritative since you specifically mentioned them

They do not search other sites until you mention in the prompt for these types of query.

Open Web Browsing: When searching the open web, the LLM faces a much more complex challenge:

• It needs to evaluate source credibility on the go
• Filter through potentially millions of results and rank them
• Deal with varying content quality
• Navigate different website structures and formats

Since they are scraping and navigating through these sites on the web. It takes generally more time than the google search which uses predetermined results obtained from algorithms.

The Ranking of the Site: SEO for LLMs

Now there is SEO for the LLMs also. LLMs don’t just grab the first search result and call it a task accomplished. They’ve been trained on ranking signals that help them identify trustworthy sources:

Domain Authority: Sites like Wikipedia, established news outlets, government websites, and academic institutions typically get higher trust scores. The model has learned patterns about which domains tend to provide reliable information.

Content Quality Signals: The LLM looks for indicators of quality content:

• Proper grammar and spelling
• Coherent structure
• Citations and references
• Recent publication dates for time-sensitive topics
• Author credentials (when available)

Cross-Referencing: Good LLMs will often check multiple sources and look for consensus. If three reputable sites say the same thing, that’s more trustworthy than one random blog post.

Web Scraping and Content Extraction

Once the LLM decides which pages to examine based on ranking and other stuff, it needs to extract useful information. This isn’t as simple as copying everything — web pages are full of navigation menus, ads, comments, and other noise.

The model has been trained to identify and extract the main content from web pages:

• Article text vs. sidebar content
• Main paragraphs vs. captions
• Factual information vs. opinion pieces
• Current information vs. outdated content

The Content Processing Pipeline

After scraping content, the LLM processes it through several steps:

1. Cleaning: Remove HTML tags, ads, navigation elements
2. Tokenization: Break the content into processable chunks
3. Relevance Filtering: Identify which parts actually answer the user’s question
4. Fact Checking: Cross-reference with other sources when possible
5. Synthesis: Combine information from multiple sources into a coherent response

Training for Web Browsing

LLMs aren’t born knowing how to browse the web effectively. They’re trained through several methods:

Supervised Learning: The model is shown millions of examples of:

• Good search queries and their results
• High-quality web content vs. low-quality content
• Proper citation and source attribution
• Effective information synthesis from multiple sources

Reinforcement Learning: The model learns from feedback about whether its search results and summaries were helpful. If users consistently rate certain types of responses as better, the model learns to replicate those patterns.

Constitutional AI Training: Some models are trained with explicit rules about web browsing behavior, like “always cite sources” or “prefer recent information for current events.” as we have seen in Perplexity.

Handling Different Content Types

Web pages come in all shapes and sizes, and LLMs have learned to handle different formats:

• News Articles: Look for headline, publication date, author, main content
• Academic Papers: Identify abstract, methodology, conclusions
• Forum Posts: Distinguish between original posts and replies
• Product Pages: Extract specifications, reviews, pricing
• Social Media: Handle informal language, hashtags, threading

Summary

Let me break this down in simple terms: LLMs search the web by using specialized tools (like search APIs) that they’ve been trained to use intelligently. When you ask a question, the AI decides whether it needs fresh information, crafts a good search query, evaluates the credibility of sources it finds, extracts the relevant information, and then synthesizes it all into a helpful response.

The key difference between custom sites and open web browsing is that custom sites are treated as pre-approved sources, while open web browsing requires the AI to make real-time decisions about what sources to trust and what information is relevant.

The whole process relies on extensive training that taught the model how to be a good web researcher — not just how to find information, but how to find good information and present it clearly.

TL;DR: LLMs search the web using APIs or scrapers like selenium, evaluate source credibility through learned patterns, extract relevant content from web pages, and synthesize information from multiple sources to answer your questions — all while being trained to distinguish between reliable and unreliable information.