Few have tried OpenAI’s Google killer. Here’s what they think.

The limitations of the prototype search tool suggest that OpenAI, whose ChatGPT has inspired predictions that some Silicon Valley giants could become sidelined, still has major work to do before it can begin to directly threaten Google’s lucrative search business.

“We’re going to take the best features and merge them into ChatGPT,” OpenAI spokeswoman Kayla Wood said in a phone interview about SearchGPT. When asked if OpenAI’s service would include ads, like Google and other established search engines, Wood said the company’s business model was based on subscriptions. But she added that OpenAI hasn’t announced if SearchGPT will be offered free or as part of a ChatGPT subscription.

Since OpenAI unveiled ChatGPT in November 2022, tech pundits and industry insiders have predicted that AI chatbots will revolutionize the way people find information online, potentially disrupting Google’s lucrative, decades-old position as the dominant gateway to the web. OpenAI stoked further excitement in July when it announced SearchGPT, and in recent weeks, the company began making the prototype available to 10,000 early testers.

More than 200 million people use ChatGPT every week, the company said last month. Many ask questions that might have previously been typed into Google. But that doesn’t appear to have cut into Google’s business yet: The search giant’s revenue continues to grow, which CEO Sundar Pichai in a July investor call credited in part to “tremendous ongoing momentum in search.”

Microsoft added a chatbot powered by OpenAI’s technology to its Bing search engine last year, but the company has seen only modest growth in its market share, according to search marketing firm BrightEdge. Microsoft’s search engine remains dwarfed by Google, which has recently added its own AI search features.

In its current version, OpenAI positions SearchGPT as a one-stop shop for whatever information a person seeks, rather than a conventional search tool that offers up a list of options to click on. After a user types in a query, SearchGPT offers up a concise answer with subheadings and bullet points, links to sources and related images. The tool uses Bing and other data sources for the information that it presents, OpenAI said.

SearchGPT’s interface looks more similar to a search engine than ChatGPT and other conversational chatbots do, but OpenAI’s search tool also encourages follow-up questions. “People already turn to ChatGPT for information, and with the search features in the prototype, we want it to become a go-to place for even more questions,” Wood said in an email.

Google has steadily moved in a similar direction over the last decade, displaying certain answers drawn from webpages at the top of its search results for many queries. The company says that approach is more helpful for users but has been criticized by web publishers who say Google uses their content in a way that discourages people from visiting their webpages.

OpenAI declined to provide The Washington Post access to SearchGPT and said in an email last month to those on the wait list for access that “we’ve had an overwhelming response and have filled the initial spots for the prototype.”

Ananay Arora, a software engineer and AI and cybersecurity researcher, is among those with access to the SearchGPT prototype but says it doesn’t seem to pose much of a threat to Google so far. He was pleased with the results on a query about local restaurants. But for other searches, he has been underwhelmed by the images SearchGPT provided alongside its results and found the way sources are labeled occasionally confusing.

“From a company like OpenAI, you’d expect a breakthrough, given their history of state-of-the-art models,” Arora said in a phone interview. In comparison to ChatGPT, he said, SearchGPT “isn’t exactly too impressive.”

Daniel Lemire, a tech professional who runs the educational organization AI Mistakes, was more positive in an interview about his own experiences as an early user of SearchGPT. He said he thought OpenAI’s search tool is better than the AI-generated answers, or “overviews,” that Google has added to its results pages. “I would choose SearchGPT over Google any day,” Lemire said.

But both Arora and Lemire said SearchGPT’s interface and answers were less impressive than how a much smaller start-up, Perplexity, has attempted to reinvent web search with AI. Perplexity uses AI to search the web and provide one-shot answers, with links to sources. The company has built its own “web index,” or giant catalogue of websites on the internet, but also uses some data from Google and Bing, its CEO has previously said.

In a YouTube video posted this month, AI enthusiast Matt Berman shared some of his own experiences with SearchGPT, including comparisons of results from Perplexity and Google on queries that included Olympics results and the assassination attempt on former president Donald Trump. He judged the AI search tools to beat Google results on queries about event planning or how to fix a coding issue, and said SearchGPT “nailed it” when asked to list the top three movie theaters in his neighborhood and explain why.

But Berman also ran into an example of the problem of AI tools providing incorrect, or “hallucinated,” information that has plagued ChatGPT and its rivals. “A big downside to AI search is it will tell you things with complete confidence that are just false,” Berman said in his video. In July, a video demo of SearchGPT in OpenAI’s blog post announcing the tool also showed an error, providing the wrong dates for a music festival.

In his video, Berman showed an example in which SearchGPT was asked who’s speaking at the upcoming Dreamforce tech conference in San Francisco. In response, it named OpenAI CEO Sam Altman, who is not listed on the event’s website but did speak at last year’s conference. When asked about the error, OpenAI’s Wood said SearchGPT is “an initial prototype, and we’ll keep improving it.”

Google said Friday that it was limiting how its own AI search results respond to queries related to November’s election because “this new technology can make mistakes as it learns or as news breaks.” OpenAI has said ChatGPT will respond to questions related to election procedures by directing users to authoritative information.

BrightEdge, the search marketing company, didn’t extensively study hallucinations for a report on SearchGPT issued last month but did come across a few instances. Jim Yu, founder and executive chairman of BrightEdge, estimated in an interview that less than 1 percent of searches returned information that was clearly incorrect.

The company’s report also found that Google remains the clear leader in searches for online shopping or local information. Google has access to specialized data to answer such queries, such as business listings, hotel room availability or flight information. The kind of data needed to answer such queries can be expensive to obtain: Google in 2011 spent $676 million to acquire a flight data provider that helped enhance its travel search results.

Yu said he believes that despite SearchGPT’s limitations, it could still disrupt how people get information online, potentially akin to how the iPhone disrupted mobile computing.

Wood said OpenAI will use feedback on the SearchGPT prototype to “help guide our ongoing improvements.”