Enterprise Site Intelligence Adds Crawler Profiles for AI Agents and Search Bots
New bot simulation capabilities let teams see their website exactly as AI models and search engines do. Without manual configuration.
March 31, 2026
Have you heard about the “subdomains vs. subfolders” controversy? With SEMrush, it doesn’t matter what website architecture your competitors prefer as you will be able to break down their SEO strategies anyway.
The Organic Research tool that provides information about a domain’s estimated traffic, keywords, rankings, SERP features, top pages, top subdomains, and main organic competitors has been recently enhanced with a “subfolder filter”. Previously, the data was only available for the root domain, subdomain or a separate URL. Now you are welcome to research specific subfolders too, which lets you analyze the performance of the website’s regional versions.
So if you are interested in how much traffic your competitor’s blog (or any other subfolder) is getting, what keywords it ranks for or what SERP features it has, just type in the subfolder link in the Organic Research search bar or choose the “as Subfolder” option in the drop down menu*.
The Organic Research tool that provides information about a domain’s estimated traffic, keywords, rankings, SERP features, top pages, top subdomains, and main organic competitors has been recently enhanced with a “subfolder filter”. Previously, the data was only available for the root domain, subdomain or a separate URL. Now you are welcome to research specific subfolders too, which lets you analyze the performance of the website’s regional versions.
So if you are interested in how much traffic your competitor’s blog (or any other subfolder) is getting, what keywords it ranks for or what SERP features it has, just type in the subfolder link in the Organic Research search bar or choose the “as Subfolder” option in the drop down menu*.
*Please note that filtering by subdomain, subfolder or a URL is only available to paid users.
After you press the “Search” button, several Organic Research reports (Overview, Positions, Position Changes, Pages) will be recalculated to reflect the data for the subfolder you entered.
After you press the “Search” button, several Organic Research reports (Overview, Positions, Position Changes, Pages) will be recalculated to reflect the data for the subfolder you entered.
This functionality comes in handy when you need to find out:
- how well the regional versions of the website rank compared to the main one
- which subfolders feed the authority of the root domain
- how the latent semantic indexing is arranged on your rival’s website
Enjoy the new functionality! And if you don’t, please feel free to send your comments and suggestions on how to make Organic Research better to product-beta@semrush.com.
Explore your competitors’ SEO subfolder by subfolder
About Semrush
Semrush is a leading online visibility management SaaS platform that enables businesses globally to run search engine optimization, pay-per-click, content, social media and competitive research campaigns and get measurable results from online marketing. Semrush offers insights and solutions for companies to build, manage, and measure campaigns across various marketing channels. Semrush, with over 91,000 paying customers, is headquartered in Boston and has offices in Philadelphia, Trevose, Austin, Dallas, Amsterdam, Barcelona, Belgrade, Berlin, Limassol, Prague, Warsaw, and Yerevan.
