SEO in 2026: AI is talking nonsense, and I'm tweaking crawler configs

Honestly, doing SEO in 2026 feels weird. Not the “things are changing so fast I can’t keep up” weirdness—I’ve heard that since 2018. It’s a subtler awkwardness: you feel like you’re doing nothing right, yet the data slowly improves. Then you try to replicate that “success” and discover that last week’s configuration is completely ineffective this week.

I spent an entire three days last month trying to figure out why a page that had been consistently ranking in the top three suddenly fell to the seventh page. In the end, it wasn’t a content issue, a link issue, or any of the things I was trying to optimize. It was Google’s AI overview redefining the whole search intent. It no longer thinks users need “compare A and B,” but instead pushes the answer straight to the top of the SERP, and everyone stops clicking.

Zero‑click search is no longer a trend in 2026; it’s the norm. According to the Web Almanac, mobile zero‑click rates rose from 10.6 % in 2024 to 13 % in 2025. My take on that is: it’s okay, at least users aren’t going to competitors. But honestly, zero‑click search is a real loss of traffic for sites—especially for businesses that rely on informational content to attract visitors.

Some things are getting better, though. Take robots.txt management: it used to be a “set it and forget it” file, but now it’s a strategic layer. Should you block certain AI crawlers? How do you block them? Should you even use the newly introduced LLMs.txt file? I’ve seen people argue about this all night. Its principle is to let site owners tell large language models what content they may train on, but the problem is, how do you know they’ll actually obey?

After all, there’s no crawler police to enforce it.

AI Search: Traffic isn’t down, but the quality has changed

The unavoidable topic in 2026 is AI search. Some estimates say AI will handle about 25 % of global queries this year. I haven’t verified that number, but it feels about right. In my daily work I can clearly see two things: first, Google’s AI overview now covers the vast majority of the queries I’m responsible for; second, user behavior has shifted—people no longer click three results to compare, they just ask and read the summary, rarely scrolling down.

What does that mean for content strategy? At the end of 2024 I made a serious mistake: I churned out short, punchy paragraphs that were “well‑structured and easy to extract,” thinking that would make it easier for AI search to quote them. They were indeed quoted, but traffic didn’t increase because users left after the quote and never clicked through.

Later I realized that AI search isn’t stealing your traffic; it’s reallocating user attention and click intent. If you write content that users must read the original for—like comparative analyses, experiment reports, or industry insights with subjective judgments—AI can’t fully extract it because it lacks context. That kind of content actually drives more clicks because users find the summary insufficient.

That’s not a deep strategy; it’s common sense. The biggest challenge in 2026 isn’t knowing what to do, it’s having the time and tools to consistently produce that kind of content. I tried writing manually and gave up after two weeks. Then I started using SEONIB to automate the process. At least it can run the whole pipeline from topic selection to generation to publishing, instead of me spending two hours each day debating “what to write today.”

But honestly, tools are just tools. Depth of content still requires your own judgment about what’s worth writing and which angles will differentiate you. I just don’t want to waste too much time on a scheduling task.

Structured Data: Not for Google, but for AI

Structured data underwent an interesting shift in 2026. Previously we added schema with a single goal: get a rich media snippet from Google, like star ratings or FAQ collapses. Now the logic has changed.

The usage of FAQPage schema grew noticeably from 2025 to 2026, and the Web Almanac data confirms it. The growth isn’t due to a change in Google’s SERP display strategy; it’s because large language models, during training and inference, tend to cite highly structured content. The Q&A format of FAQs is naturally suited for AI crawlers to pull and stitch together answers.

In other words, the structured data you write may never appear on Google’s search interface, but it can show up in ChatGPT or other AI agent replies. If you’re in B2B or your customers prefer conversational search to learn about your product, the value of structured data shifts from “ranking optimization” to “being quotable.”

I have to admit, the payoff cycle for this strategy is long. I added FAQPage schema to a batch of pages last year and only saw noticeable changes after many months, with no clear attribution. Should you do it? If you already have content, consider spending a little extra time structuring it. But if your team can’t even maintain basic content production, solve that first.

LLMs.txt: Should You Dive In?

A new headache this year: LLMs.txt. The concept is simple—you write a text file telling AI crawlers which pages are okay to use for training and which should be ignored. It sounds reasonable, but the implementation is messy.

First, there’s no standard. Each large‑model company supports LLMs.txt to varying degrees—some ignore it completely, others partially adopt it. Second, how do you know whether a “bad” model actually accessed your content after it’s crawled? In one project I wrote strict exclusion rules, only two weeks later a new AI search engine still quoted the excluded page. It felt like hanging a “Do Not Disturb” sign that nobody sees.

My experience: you don’t need to be overly aggressive about this file right now. If you have truly secret commercial information that must never be used for AI training, don’t count on a text file to stop it. Prioritize a solid robots.txt and proper access controls on the pages themselves. LLMs.txt is more of a 2026 trend experiment—it may become a standard, but the timing is still premature.

Will you lose anything by not adding this file? So far, no. If you’re worried that someday it’ll be required for indexing, you can write a minimal version now and set it aside; it’s not a big effort.

E‑E‑A‑T: Google Hasn’t Forgotten, It’s Just More Hidden

Everyone’s talking about AI and structured data, but E‑E‑A‑T didn’t disappear in 2026; it’s showing up in more subtle places. Sites that previously relied on AI‑generated bulk content for rankings are now generally seeing declines. Google may not have updated many official guidelines, but the algorithm is already filtering them out.

I’ve noticed a trend: SEO tools and platforms in 2026 now automatically handle many basic optimizations. CMSs generate schema, internal linking suggestions, and meta descriptions at publish time. This lowers the barrier to entry for SEO, but it also makes “just passing” easier to fake. Web Almanac data shows that more SEO optimizations are achieved through default settings and tool behavior rather than deliberate effort. That sounds good, but the problem is: when everyone is just “passing,” how do you stand out?

The answer lies in the things “tools can’t do”: truly experience‑based content direction, nuanced understanding of user intent, and the willingness to invest several times the average resources into a single piece of content.

FAQ

1. Will AI search completely replace traditional search?
Not in the short term. By 2026 AI search already accounts for a substantial share of queries, especially informational and Q&A ones. For brand‑specific or transactional queries, traditional search remains dominant. Many people use both sides, not one or the other.

2. How big is the traffic impact of zero‑click search?
It depends on your industry. Information‑heavy sites feel the biggest hit; some categories have zero‑click rates above 20 %. If you run e‑commerce or B2B services, users have stronger purchase intent and clearer needs, so the impact is smaller.

3. Are there quick hacks for AI optimization (AEO/GEO)?
No. The most practical approach is to get your structured data right and ensure your content is clear, complete, and offers unique viewpoints. AI loves that kind of content. As for a shortcut to jump into AI search results—if you find one, please share it with me.

4. Should I start writing LLMs.txt now?
If you have many site resources, a simple version can be drafted and left idle without consuming much time. If you’re still focused on basic SEO, put it lower on the priority list. It’s not yet “essential.”

5. Is SEO still worth investing in 2026?
Absolutely, but the way you invest has changed. It used to be enough to produce a lot of content and build many links. Now you need finer judgment, more patience, and a willingness to embrace uncertainty. If you asked me whether I’d want to go back to doing SEO in 2020, I’d say: no, 2026 is annoying, but at least it’s not that boring.