The web as we knew it, yesterday, looked to the minds of Larry Page and Sergey Brin as a series of connections. Much like research papers are considered more authoritative the more they’re referenced, it dawned on the two young Stanford researchers that web pages, too, were more likely to be authoritative if more people linked to them.

Yahoo!, Dogpile, Webcrawler, and other early search engines were directories, with pages of links that other humans thought you’d like to see. But there was bound to be wisdom in the crowds, they guessed. If more people linked to one page when talking about frogs than another, well, that page must be the best frog page on the internet.

“The citation (link) graph of the Web is an important resource that has largely gone unused in existing Web search engines,” their 1996 thesis surmised, and so they set about building what’d become the GoogleBot to read through the web, log and follow every link on every page, and rank search results based, largely, on what was linked to the most.

Great content would automatically float to the top, once enough people discovered and linked to it. That was, in many ways, the promise of the web (and, to a degree, of research papers in general).

Feeding the beast

Thus Google. And what we knew as search engine optimization (or SEO for short) for the past decades has, largely, worked along those lines. There was more to it, of course: A backlink from an educational website accrued more reputation, as did links from older websites, authoritative authors, and more. But largely, if you wrote something and put it on the internet, and more people linked to it than another similar piece, your page would rank higher on Google search results.

A (predictable in retrospect) cat-and-mouse game ensued, with the worst actors on the internet buying backlinks or hiding them spuriously in comments or invisible text, in hopes that Google would smile as favorably on those links as it did ones earned through word-of-mouth. Then Google would tweak the algorithm, block some old tricks while adding some new structured data that folks would rush to add to their site to feed the GoogleBot. Sites would rise and fall in rankings on a whim, only after a few months to settle down into a new equilibrium. Again and again, until you develop PTSD from looking at your site’s traffic go up and down, largely outside of your control.

It wasn’t like Google would come right out and say ok, here’s how search will work starting next month, prepare your sites accordingly. They’d tweak the algorithm, your traffic would drop, and you’d be left guessing. “For many years, being found was basically figuring out the Google algorithm,” Duda’s CEO Itai Sadan told me in a recent call while talking about shifts in search. “People really focused on trying to get the exact keywords, backlinks, and having authoritative sources pointing to your domain.” 

When Google said jump, the collective web said “How high?” We the writers of the internet needed Google far more than it needed any one of us individually. So if Google seemed to reward longer pages than shorter ones (were they counting bounce rates? time on page? actual number of words, or unique words? It never was exactly clear), we’d write the longest articles possible. Thus, among other reasons, why every recipe online seemingly has to start with the story of the author’s grandmother and a long-forgotten ingredient and more.

Yet it has always been a battle for increasingly scarce resources. What started as a competition to be one of the first ten blue links on a Google search results page turned into a losing battle as ads, answer previews, location details, questions, and more ate up increasingly larger swaths of the Google results page. Then when readers increasingly stop searching and turn to social media for answers instead, the entire value proposition of creating content in the hopes that Google would send you traffic increasingly wasn’t panning out.

Then Google itself upended the entire search paradigm it invented. “The goal of Search has always been simple: to help you ask anything on your mind,” Google’s VP of Search Elizabeth Reid wrote of what the search giant called the search box’s largest upgrade in over 25 years. AI mode, the new generative AI answers at the top of Google search, is now the default content at the top of Google searches; the blue links that we’ve been fighting over for decades are relegated to, often, under the fold.

People are “searching more than ever before,” said Reid. Only now, it’s likely they’re searching, getting their answer directly on Google, and never venturing beyond to your site.

Ranking in the age of AI

What stands in for Artificial Intelligence in 2026 is, largely, Large Language Models. What started as predictive text, if you vastly oversimplify, guessing that it’s highly likely the word “you” will appear after the words “I love,” turned with enough computing power into a simulacrum of humanity’s collective knowledge. 

You can ask Google’s Gemini (or OpenAI’s GPT, or Anthropic’s Claude, or any of the dozens of others competing for the AGI crown) the distance of the earth to the moon, or the meaning of weariness and insanity, and it’ll proffer reasonable responses to each. Ask it to code a webpage and it will, improbably, incredibly, do it with equal vigor, drawing from the vast examples of HTML code and website design on the internet to mashup something close to what you want. Ask it to count the number of r’s in “strawberry” and, as something that few people discussed prior to the dawn of AI, earlier AI models would confidently return a false response (they’re completing sentences to their most likely conclusion based on the weights of all words on the internet, remember, not specifically thinking through the problem—at least, not at first).

Which appears to lay out a model for SEO in the age of AI. The more your thing was mentioned alongside certain keywords, the more likely AI was to use your thing’s name when answering questions containing those keywords. It looked, if you squinted and were hopeful, for all the world like SEO—only this time, benefit would accrue to your site even if someone mentioned your product but forgot to include a link. AEO, it quickly came to be called, and when you’d ask an AI what the best CRM was and it mentioned Salesforce, Benioff’s team could feel that they’d done well at optimizing for AI.

Zoom out a bit more, though, and if you’re a glass-half-full kind of person, AI-driven answers offer the most likely answer distilled from the web, and thus in a similar concept to Brin and Page’s BackRub could let the best ideas float to the top if they’re repeated enough online. It could let your site, your articles, show up in AI responses if they’re focused on the exact combination of things someone asks, or if they’re popular enough that they’re often repeated and shared around online by others, for something close to both long-tail and mass search results in the Google-driven era.

It’s less predictable, and far more fractured than in when Google had a near-monopoly on delivering answers online. “It was difficult enough to crack the Google algorithm. Now you have to crack all these others,” as Sadan told me from the vantage point of a service used to build millions of websites, “and they are much more opaque.”

Yet maybe the solution is actually to go back to the pre-Google days, when we weren’t worrying about structuring things for the Googlebot, weren’t thinking about if this exact word would bring more people to our page than another.

This isn't quite the same as the oft-repeated advice to “write for humans,” which as often as not rounds to looking at the things people searched for then mechanically checking them off to, in turn, feed the Google bot. As Ryan put it, “Please don’t pick a CSV export before a thesis. That’s not the way to engage people’s thinking bags or touch their blood pushers.”

The early web was people writing pages for other people, for things they wanted to read. The best would float to the top when others recommended them, on personal sites and directories alike. Google piggybacked off that chain of references, and for a while everyone kept doing what they were doing.

It all went wrong when we started working directly for the Googlebot instead of for readers, for other humans. It’s a mistake that will be the death knell of the internet if we collectively make it again for AI—for this time, we have a chance to make a more human internet, if we want to.

Bots aren’t your customers. Humans are.

Google, it’s easy to forget, wasn’t your customer (nor your boss, if you’re not a Googler). It merely was an intermediary between you and your ideal customers. Working for the Googlebot never was quite worth it, directly, unless it was in service of getting more people to your site and, by extension, to what you’re offering.

People knew they wanted something, a dentist near their home or a flight out of town or a recipe for Teriyaki sauce or the CSS to make their emails work in dark mode. Google was just the best intermediary between them and you with the answer.

AI, it’ll soon be easy to forget, is not your customer. For all the talk of AI-intermediated commerce, for the OpenClaw talk of handing off your credit card details to AI and letting it build a business on its own, that for nearly every business on earth will not be your next best customer base. It’s people who will read your website, people who will be your customers. AI or search are merely the tools they’ll use to reach you. And AI, the Frankenstein-that-grew-out-of-predicting-words, actually reads words. It, for once, takes the details into consideration, in a way, not merely the keywords and backlinks but the context in which they’re shared.

The best websites for people, for human readers, in all likelihood are the best websites for AI. Things that people want to read—real, human stories told in your voice, with the truth-is-stranger-than-fiction details about things that went wrong and how your fixed them that no AI could make up on their own, things that share what you’ve learned in building your product and why you made the decisions you did—they’re what will turn visitors into fans, and at the same time are what has a better chance of being picked up and regurgitated by AI than yet another blog post that says the same things as everyone else.

Answer questions about everything—in yours, not AI’s words. The more pages the better, according to Duda, anyhow, seeing an increase in AI bot reads as sites add more content. Where previous SEO folk wisdom said to trim dead weight, a streamlined landing page—regardless of how many backlinks you gather—is far less likely to get suggested by AI than one with dozens of pages detailing everything people could possibly wonder about your business, and then more, each in your own words focused on your own audience, written for the actual humans who will walk in your door and show up time and again when you publish something new. It’s less building an optimized site and more building a publication around your brand.

If anything, you have more freedom to use your wording and write the way you want, when AI understands synonyms and is less sensitive to exact keywords than Google’s more brittle algorithms.

Sadan, with the millions of small business websites at Duda, has watched the changes unfold in realtime. “We do see that traffic has decreased in many cases to websites,” he said, an unavoidable consequence when AI can distill an average Teriyaki sauce recipe from the multitude online, turning that and millions of other queries into answers that never send a click to a site. That’s an unavoidable consequence of today’s answer machines.

But the people who do arrive, after skimming an AI answer? They’re primed to read, to dig in, to buy. “The traffic that is coming in is converting at a higher rate,” shared Sadan. “I think people are doing a lot of their research on these LLMs, asking ChatGPT and comparing between solutions. When they eventually arrive on your website, they’re much more ready to buy,” something Duda credits with a boost in both clicks and in real-world brick-and-mortar visits.

Which means it’s more important than ever to optimize your site experience for humans, with less annoying popovers and SEO-focused baggage and more of the human content that answers their questions, piques their curiosity, and endears them to what you’ve built. More important than ever to have a voice, and own it.

“Do the right thing by your customers,” says Sadan. “That’s what eventually will win.”

Image Credit: Header photo by Peter Thompson via Unsplash.