Arc browser’s new AI-powered ‘pinch-to-summarize’ feature is clever, but often misses the mark

The Browser company’s Arc, a browser focused on a less cluttered web experience, launched a new feature in its mobile app Arc Search that uses AI to summarize web pages. The feature involves a clever “pinching” gesture that shows a neatly formatted summary with main points.

The feature has gotten attention because it presents a new dynamic to interact with AI, but there is also concern that it could impact traffic to news publishers, like ourselves.

We have seen other efforts to look at AI-powered summaries such as Artifact, the shuttered app from Instagram’s co-founders, as well as from tech giants, like Google’s Search Generative Experience’s web page summarizing feature. However, because of its unique user experience and gesture design, Arc’s pinch-to-summarize got people on social media across platforms like Threads and X talking because of the good-looking transition animation.

If you have the new version of Arc Search on iOS, you can now pinch while you’re on a page, and the browser will show you an AI-powered summary with different points.

The gesture is cool to use and look at, because when you pinch a web page, it folds in an origami style while the browser generates a summary, and the transition effect is smooth. And this effect is more satisfying because of subtle haptic cues.

However, in our testing, the AI summaries themselves often miss the mark. For instance, we updated an older story to address the online hoax that Google is discontinuing Gmail rather than just noting that the company is discontinuing Gmail’s basic HTML view. Arc’s summary didn’t catch the important bit referencing the rumors about Gmail shutting down being false, which we had added on top of the story. 

There were a few other hiccups, too. When we tried to summarize a recipe page in Hindi, the function didn’t work. We just saw points like prep time, cook time and calorie count without any details about actually making the dish. Other users have also pointed out that the summary feature doesn’t work well with other languages. (We have asked Arc about language support for this feature, and we’ll update with additional information if we hear back.)

On an English-language page for baking chocolate cookies, we got a decent AI-powered summary including ingredients, recipe instructions and additional tips, but we had to scroll down to have those key points included. When we generated the summary without scrolling down, we just got ingredients and cook time as useful points in the summary.

When reading something about the upcoming Indian Premier League (IPL) cricket tournament’s schedule, the AI summary missed a point about a part of the schedule being released after the dates of the general elections in India were announced.

When Arc Search tried to summarize Bluesky’s blog about federation, the text felt more robotic, rather than explanatory, and missed some of the points such as moderation, which might be important for users.

The feature generated useful summaries for many articles and pages, but we felt like we had to double-check if there was something missing. As seen in the examples above, AI can miss critical information while summarizing at times, so it is hard to fully rely on these summaries unless the importance of information is trivial. Understandably, this is the first iteration of Arc’s feature, and it also has limited space to fit in all points of the summary.

That said, there is also a problem with AI-powered summary features in other places as well.

Both Perplexity and ChatGPT missed the update about the Gmail hoax in the above-mentioned article. And Gemini gave us a useless summary of the IPL schedule article.

Image Credits: Gemini

There are concerns that Arc’s approach could be harmful to journalism, too — an issue that was raised by several journalists this week, including The Platformer’s Casey Newton, who talked about how Arc’s approach might be harmful to journalism and the web overall. Ryan Broderick, who publishes The Garbage Day newsletter, wrote a Fast Company column that pointed out that companies building AI-powered search are not thinking about how their approach might affect websites and people’s motivation to contribute to the web.

These are valid concerns, not only for publishers but for news consumers, as well, because if AI misses important points in a summary it might not be reliable to trust the feature for accurate information.

At the moment, Arc Search’s summary feature can’t share these summaries along with embedding the source’s link; at least some people might click to read the whole article. (We asked Arc about the possibility of introducing such a feature and also how it plans to improve the quality of the summaries over time.)

Additionally, Arc updated its “Browse for me” AI-powered search to make the pages generated as search results sharable. Plus, the company made links within those pages clickable so people can visit links or read more. Arc also added incognito mode to the mobile browser, in its recent update.

There is a larger discussion about the value of data for AI and returning that value to content creators. A lot of people might skim over smaller mistakes or omit some details when AI features are fetching answers. However, for the price tag and valuations demanded by AI companies, it needs to be more accurate and reliable.