Enterprise Site Intelligence Adds Crawler Profiles for AI Agents and Search Bots
New bot simulation capabilities let teams see their website exactly as AI models and search engines do. Without manual configuration.
March 31, 2026
Is your website affected by keyword cannibalization? Do you want to know how a competitor has taken a leading position and for how long? Which competitor’s pages were ranking by a specific keyword at any given time?
You can get answers to all these questions with the help of the new Organic Research feature. Now when you click on the arrow next to a keyword in the Positions or Position Changes tab, you will find the ranking history of your competitor for a particular keyword for all time!
You can get answers to all these questions with the help of the new Organic Research feature. Now when you click on the arrow next to a keyword in the Positions or Position Changes tab, you will find the ranking history of your competitor for a particular keyword for all time!
You can use the new function to monitor your competitor's strategy and discover:
- A month-by-month overview of the URLs that ranked in the top 100 for a specific keyword;
- Whether a competitor has been optimizing one page for this keyword for a long time or has recently been trying to show another page for this keyword;
- A new, powerful competitor’s page that has recently won a ranking for new keywords;
- What other pages of a competitor’s site may be relevant to such a request;
- Your competitor’s ranking trend for the keywords with SERP features that are important to you;
- Whether your competitor continuously improves any page, such as here:
But equally important, you can analyze your own site too. It will be useful:
- To identify and fix bouncing rankings caused by Google having difficulties choosing the right URL;
- To make sure that the migration is done correctly and that HTTPS pages rank well while HTTP URLs are no longer present;
- To check if your pages cannibalize for the same query, which is really important to avoid, as in the example below:
Try the new feature very soon and let us know what you think! There are multiple ways to use this data. Feel free to write your comments and suggestions to domain-analytics-feedback@semrush.com
About Semrush
Semrush is a leading online visibility management SaaS platform that enables businesses globally to run search engine optimization, pay-per-click, content, social media and competitive research campaigns and get measurable results from online marketing. Semrush offers insights and solutions for companies to build, manage, and measure campaigns across various marketing channels. Semrush, with over 91,000 paying customers, is headquartered in Boston and has offices in Philadelphia, Trevose, Austin, Dallas, Amsterdam, Barcelona, Belgrade, Berlin, Limassol, Prague, Warsaw, and Yerevan.
