An search engine marketing competitor evaluation helps you perceive the place and why your rivals are profitable in search.
On this article, I’ll present you the way to analyze your rivals quicker by combining Semrush information with the facility of AI instruments.
The best way to arrange the Semrush MCP
The Semrush MCP (Mannequin Context Protocol) connects your AI instrument to your Semrush account to supply entry to useful aggressive evaluation information. This allows you to analyze competitor domains, key phrases, and backlinks.
Join the Semrush MCP to Claude by clicking the “+” icon in any chat. Then click on “Connectors” > “Add connector” > “Browse connectors.”
Seek for Semrush and click on the “Join” button subsequent to the Semrush connector. Then, observe the on-screen directions to attach your Semrush account.
The Semrush connection makes use of API models while you immediate Claude to entry Semrush information by way of the MCP. The prompts beneath are designed to attenuate API unit utilization whereas nonetheless offering helpful aggressive evaluation information.
You possibly can see your API models in your Semrush account below “Subscription data” > “API Items.” I ran the prompts beneath and used round 16Ok API models, however it will range relying on how a lot information you request per run, the way you phrase the prompts, and what number of you ship.
I’ve prompt numbers to make use of in every immediate as a place to begin. I like to recommend protecting these numbers to start with and solely scaling up when it’s worthwhile to, as it will assist maintain your utilization low.
Subsequent, click on the “Initiatives” choice on the left after which click on the “New undertaking” button on the high right-hand nook.
Making a undertaking will allow you to add context information (see beneath) and maintain all of your competitor evaluation duties in a single place.
Give your undertaking a reputation and an outline, then click on “Create undertaking.”
Subsequent, create a enterprise context file to offer Claude details about your online business to assist enhance its outputs. Embrace details about your online business’s trade, any particular merchandise you promote, and the subjects you already cowl in your web site.
Use the immediate beneath to let Claude construct your context file for you, with the assistance of Semrush’s MCP so as to add your present natural search efficiency information:
Create a enterprise context file for [yourdomain.com] that I can add to this undertaking. Goal nation: [country].
Vital details about my enterprise: [What you sell, who you sell to, competitors you know of, topics you want to rank for, topics to avoid, and your content resources (team size, budget, whether you can do original research or build tools)]
Use Semrush information so as to add details about my enterprise’s natural site visitors, natural key phrases, Authority Rating, referring domains, high 10 pages by site visitors, and the principle subjects we already rank for.
Write it in Markdown. Add “Final up to date: [today’s date]” on the high. Label something you inferred with “(inferred)” so I can test it, and ask me about something you may’t fill in.
Right here’s an instance output of this context file:
Obtain this file, then navigate to your undertaking and click on the “Library” icon on the highest right-hand facet of your undertaking window, choose “+ Add” within the Library, click on “Add information,” and add the context file you simply downloaded.
Your Claude undertaking will use this enterprise context for every of the 4 use instances beneath to return extra related competitor, key phrase, matter, and backlink outreach steerage.
Four methods to make use of the Semrush MCP for search engine marketing competitor evaluation
Observe the 4 steps so as beneath to make use of the Semrush MCP for a complete search engine marketing competitor evaluation. Or, use any of the strategies individually at any time to investigate your rivals.
The prompts assume you’ve created a undertaking and added a enterprise context file from the earlier part. In the event you haven’t, modify the prompts accordingly.
I like to recommend working every step in the identical chat, or pasting in outcomes from earlier chats. This provides Claude the context from every step to supply higher outcomes.
1. Determine and prioritize rivals
Step one of any SEO competitor analysis is to establish and prioritize which rivals to evaluate. You doubtless have a few rivals in thoughts, however it’s essential to focus in your closest rivals to make sure you benefit from your assets.
Use the immediate beneath to establish and prioritize your rivals to investigate in steps two, three, and 4, based mostly on their search efficiency, key phrase overlap together with your area, and extra:
Utilizing Semrush information for [country], discover the highest 10 natural rivals for [yourdomain.com]. Add any enterprise rivals from my enterprise context file that are not already on the listing.
Solely embrace actual rivals. Exclude boards, social platforms, video platforms, and reference websites (e.g., Reddit, YouTube, Wikipedia), even when they share plenty of key phrases with us.
For [yourdomain.com] and every competitor, pull:
- Natural site visitors
- Natural key phrases
- Widespread key phrases with [yourdomain.com]
- Key phrase overlap % (widespread key phrases divided by [yourdomain.com]’s complete natural key phrases)
- Authority Rating
- Referring domains
- Natural site visitors development during the last 12 months (rising, flat, or declining)
If my enterprise context file already has these figures for [yourdomain.com], use these.
Return one desk with [yourdomain.com] as the primary row. Embrace these columns: Competitor, Sort (Direct competitor or Content material competitor), What they do (one line), and the metrics above.
Under the desk, advocate 3-Four rivals to investigate within the subsequent steps, with a one-sentence cause for every.
Right here’s an instance of the desk output for the beard care model Mad Viking Beard Co:
Claude then supplied some context, together with the highest 4 rivals based mostly on enterprise relevance, key phrase overlap, and website authority:
Use the listing of prioritized rivals for the subsequent three prompts.
2. Analyze competitor key phrases
Analyze your rivals’ key phrases utilizing AI to find content gaps you’re lacking, and to rapidly test whether or not it’s best to create a brand new piece of content material to fill that hole, or optimize an present web page.
Use the immediate beneath to search out key phrases your rivals rank for that you just don’t, and phrases they rank for greater than you, and to prepare them into topic clusters:
Utilizing Semrush Key phrase Hole information for [country], evaluate [yourdomain.com] with the rivals recognized in Step 1.
Half 1:
Discover key phrases that a number of rivals rank for within the high 20 however [yourdomain.com] does not rank for in any respect. Restrict to the highest 10 by search quantity.
Half 2:
Discover key phrases the place [yourdomain.com] ranks 11-50 and at the very least one competitor ranks within the high 10. Restrict to the highest 10 by search quantity.
Guidelines:
- Run every half as a single Key phrase Hole name masking all rivals directly. Do not run rivals one by one.
- Kind by search quantity and request 10 rows at a time. Solely request the subsequent 10 if fewer than 10 key phrases qualify after making use of the principles beneath.
- “High 10” means the highest 10 throughout all rivals mixed, not per competitor.
- To test for present pages, pull [yourdomain.com]’s listing of rating pages as soon as and use it for each cluster.
- Omit branded key phrases.
- Do not embrace key phrases that are not related to my enterprise, based mostly on my enterprise context file.
Group the key phrases into matter clusters. Every cluster ought to share a subject and search intent, so one web page might goal it.
For every cluster, test whether or not [yourdomain.com] already has a web page focusing on it.
Return one desk with these columns: Cluster, Major key phrase, Secondary key phrases, Complete search quantity, Common key phrase problem, Intent, Rivals rating, Our present web page (or “None”), Motion (Create new or Replace present), Precedence (Excessive, Medium, or Low).
Set precedence based mostly on relevance to my enterprise and potential to drive conversions. Inside every precedence stage, type by search quantity and key phrase problem.
This step is usually the heaviest by way of API utilization, so I like to recommend limiting the variety of key phrases that the instrument analyzes throughout all rivals to 10 as a place to begin.
As soon as you have examined the immediate and made certain you get respectable outcomes, you may then improve this quantity.
Under is an instance output desk with the subject clusters, key phrases, and related metrics.
You possibly can see within the present web page column that even the place there isn’t any present web page, Claude highlighted the closest various. For instance, within the first row, Claude flagged the collections web page for beard oil, despite the fact that there isn’t any web page particularly for “beard progress oil.”
Additional down, Claude supplied notes on the key phrase choice. The place a number of key phrases had the identical quantity, Claude seemed on the enterprise context file and opted for “beard oil vs balm” because it was most related to one of many merchandise laid out in that file.
That is an instance of how including context to your undertaking provides the AI instrument extra to work with than a easy seed key phrase. In flip, the AI instrument is ready to use its reasoning capabilities to make selections tailor-made to your model, reasonably than simply return an inventory of generic phrases.
3. Discover and analyze competitor content material that performs effectively
Analyze your rivals’ top-performing content material to know how one can enhance your personal content material’s search efficiency.
Use the immediate beneath, together with the key phrases recognized in step two, to search out the pages that drive rankings on your rivals:
For every of the highest Three precedence clusters from Step 2:
1. Use Semrush to search out the highest-ranking competitor URL for the first key phrase in [country]. If two of my rivals rank within the high 10, analyze each pages. Ignore discussion board outcomes.
2. Go to every web page utilizing your net fetch instrument. Observe:
- Content material kind (e.g., how-to information, listicle, comparability, template, free instrument, product or class web page)
- Angle and goal reader
- Important sections (H2s)
- Approximate size
- Standout components, comparable to authentic information, examples, visuals, templates, instruments, professional quotes, or video
- Final up to date date, if proven
3. Pull the highest 10 natural key phrases for every URL from Semrush. Flag any that weren’t within the Step 2 cluster.
4. If [yourdomain.com] already has a web page for this cluster, go to it and pull its high 10 key phrases too. Evaluate it facet by facet with the competitor web page(s) and establish: subtopics they cowl that we do not, key phrases they rank for that we do not, and variations in format, depth, and freshness.
For every cluster, give me:
- A abstract desk of the pages you analyzed
- What the top-ranking pages have in widespread
- If we’re creating a brand new web page: a brief transient with goal key phrase, secondary key phrases, advisable content material kind, prompt H2s, and 1-2 methods we might make our web page extra helpful than theirs, based mostly on my enterprise context file
- If we’re updating an present web page: a prioritized listing of modifications
This immediate will yield a number of outputs. The primary of those is an inventory of pages inside a particular cluster, together with details about the content material itself and why the AI instrument thinks it performs effectively.
For instance, for madvikingbeard.com, the primary cluster was for the time period “finest beard oil.” Claude advisable updating an present web page based mostly on its evaluation of the competitor web page that ranks effectively for this time period.
Claude’s evaluation included details about the angle of the piece of content material, what the principle sections seemed like, how lengthy it was, and even particular design selections on the web page. This supplies start line for you or your group to create or replace your personal piece of content material.
Additional down, Claude outlined the highest 10 key phrases for the competitor web page, together with place, quantity, and key phrase problem metrics. It then summarized the rankings and defined why focusing on the key phrase with a class web page is smart over a weblog put up.
Whereas the ultimate resolution nonetheless rests with you (the human), this evaluation, carried out robotically and at scale, makes it a lot simpler to rapidly categorize clusters into new content material and updates to present pages.
One factor I seen on this run is that Claude checked out two totally different pages on madvikingbeard.com’s web site: a weblog put up (which can at first seem to be candidate for the key phrase) and the collections web page.
On this case, for the reason that competitor’s rating web page is a set web page, that is the one Claude advisable updating. Claude then carried out a side-by-side evaluation of each assortment pages, outlining subtopics that the competitor targets that Mad Viking doesn’t, and even particular details about the product playing cards on the web page and which FAQs to make use of.
One of many particular suggestions was so as to add scent notes to each product card. I went to the collections web page on the competitor’s web site to confirm that the rival was truly doing this.
I counsel visiting competitor pages your self to confirm any particular suggestions the AI instrument makes. Keep in mind that that is only a instrument designed to flag potential actions you could wish to take. Don’t skip the human evaluation step earlier than making any selections.
4. Take a look at your rivals’ backlinks
Take a look at your competitors’ backlinks to know which of their pages earn hyperlinks from probably the most referring domains. These are the pages that could be contributing probably the most to your rivals’ authority and serving to them present up in search.
Use AI to take your backlink evaluation additional by automating the analysis and prioritization stage utilizing the immediate beneath:
Utilizing Semrush backlink information, discover the highest 20 pages on [competitor.com] by variety of referring domains. Solely embrace URLs in [e.g., /blog/], and skip the homepage.
For every web page, notice the variety of referring domains and backlinks. Then go to the web page utilizing your net fetch instrument and notice the content material kind (e.g., authentic analysis, statistics roundup, free instrument, template, information, definition, case examine) and what it covers in a single sentence.
Return one desk with these columns: Web page, Referring domains, Backlinks, Content material kind, What it covers.
Then discover 5 pages we might realistically create or enhance a model of, based mostly on my enterprise context file. Work by means of the desk, beginning with the most-linked, till you’ve got 5. For every one, test whether or not [yourdomain.com] has a web page on the identical matter and advocate:
- Create: we do not have a web page on this. Say what we would have to make a greater one (e.g., authentic information, design assist).
- Enhance: we have now a web page on this. Say what it could have to compete.
In the event you run out of pages earlier than discovering 5, inform me what number of you discovered.
Here is an instance output for certainly one of madvikingbeard.com’s rivals, Stay Bearded. You possibly can see it organizes the highest 20 pages by referring domains and notes the content material kind, together with a quick clarification of what it covers.
The output is helpful for getting at-a-glance details about your rivals’ pages that get probably the most hyperlinks, past simply numbers. When performing hyperlink evaluation at scale, that is a lot quicker than visiting each web page manually to be taught what they cowl.
Additional down, Claude supplies suggestions on pages to create or enhance, checking your web site’s rating pages and drawing on your online business context file. For instance, it recommends making a web page on beard errors, which earned hyperlinks from probably the most referring domains on the competitor’s web site.
Claude additionally supplies steerage to create a greater model of the web page by backing it with authentic information, together with before-and-after illustrations, and by utilizing the tone it recommends.
Madvikingbeard.com truly already had a put up on this matter, however Claude didn’t decide it up because it doesn’t have any rankings within the Semrush information. To account for this, I like to recommend manually importing an inventory of all of your web site’s pages to your Claude undertaking.
Because the web page doesn’t rank, the suggestions would nonetheless apply to bettering the present web page, reasonably than creating a brand new one.
I would not make any modifications to your web site simply based mostly on the steerage of the AI instrument. However this can be a very fast technique to flag pages you might wish to create or enhance, with out having to manually search by means of your rivals’ pages and match them up with your personal pages.
You possibly can carry out this evaluation repeatedly and maintain your online business context file updated with your personal new pages as you add them.
Flip these steps into an interactive dashboard
A remaining optionally available step is to show the competitor analyses above right into a single interactive dashboard. That is useful on your personal group, however it additionally makes it simple to share your plans and why you are going to take these actions with key stakeholders and different members of your group.
Use the immediate beneath to show the evaluation into an interactive dashboard inside Claude:
Flip the outcomes from this [conversation/project] into an interactive dashboard with 5 tabs: Rivals, Key phrase clusters, Content material evaluation, Linkable belongings, and Motion plan.
Make each desk sortable. Within the Motion plan tab, mix the suggestions from every step into one listing with these columns: motion, web page or matter, effort (Low, Medium, or Excessive), affect (Low, Medium, or Excessive), and the step it got here from. Kind by affect, then effort.
Present the date the info was pulled within the header. Solely use numbers from this dialog.
Under is an instance output for madvikingbeard.com. You possibly can see a abstract of metrics like natural site visitors and the way that modified yr over yr, together with details about the highest 10 rivals beneath that. Retaining the dashboard updated is so simple as prompting Claude every month to replace the numbers, because it makes use of Semrush information.
The “Key phrase clusters,” “Content material evaluation,” and “Linkable belongings” tabs all present useful summaries of the findings of every related step. Nonetheless, the “Motion plan” tab is a very powerful, because it supplies a prioritized listing of actions that you just or your group might take based mostly in your AI search engine marketing competitor evaluation.
It estimates the trouble and affect stage that can assist you prioritize duties. And since it is interactive, you may type by whichever column is most essential to you.
Carry out search engine marketing competitor evaluation at scale with the Semrush MCP
Utilizing AI for aggressive evaluation is all about performing analyses extra effectively. You continue to want a human to make the selections based mostly on the info you discover, however gathering insights is a lot quicker while you join your favourite AI instrument to a dependable information supply like Semrush.
Begin performing your personal AI aggressive evaluation utilizing the Semrush MCP right now.
