Google’s Advice on Fixing Unwanted Indexed URLs
An SEO expert did a site audit and shared their findings. They pointed out problems with using rel=canonical to control indexed pages. Instead, they suggested using noindex to remove pages from Google’s index and then blocking them with robots.txt. However, Google’s John Mueller had a different idea.
Site Audit Finds Indexed Add-to-Cart URLs
The audit found that over half of the client’s 1,430 indexed pages were paginated or “add to cart” URLs with extra query parameters. Google ignored the rel=canonical tag and indexed these pages. This shows that rel=canonical is only a hint, not a rule. Google looks at many factors before deciding which page to index. If other signals are stronger, it might ignore rel=canonical.
In this case, the indexed URLs were created dynamically based on filters like brand or size, also known as faceted navigation.
Example of an indexed URL:
example.com/product/page-5/?add-to-cart=example
The SEO expert suggested:
“I will noindex all these pages and, after that, block them in robots.txt.”
Related: Google Explains How to Identify Indexing Issues Linked to JavaScript
SEO Fixes Depend on Context
SEO strategies depend on the situation. The rel=canonical tag suggests which URL Google should index but doesn’t force Google to follow it. Stronger tools, like meta noindex, give more control over what gets indexed.
A LinkedIn discussion on this topic got over 83 responses, highlighting the difficulty of stopping unwanted URLs from being indexed and why rel=canonical may not always work.
Related: Your Ultimate Resource for Understanding Cornerstone Content
John Mueller’s Advice
Mueller suggested looking at URL patterns before choosing a solution. For example, if unwanted URLs all have “?add-to-cart=” in them, blocking those patterns in robots.txt can help. His main tips were:
- Check URL Patterns: Look for common patterns in unwanted URLs before deciding on a fix.
- Block Add-to-Cart URLs with Robots.txt: These URLs should not be crawled because they can affect website data.
- Handle Pagination and Filters Properly: Google has guidelines on managing filtered pages.
Learn More: He suggested listening to Google’s “Search Off the Record” podcast for advice on handling duplicate content.
Why Did Google Index These URLs?
Several LinkedIn users asked why Google indexed shopping cart URLs. There was no clear answer, but the issue might be related to how the eCommerce platform works. Following Mueller’s advice—using robots.txt and managing crawling better—can help avoid similar problems.
Sarosh Khan has been part of CyberX Studio since 2024 as a Content Writer and Strategist. With a degree in Media & Communication Studies, Sarosh is passionate about creating content that is both informative and engaging. She specializes in researching topics and crafting content strategies that help boost engagement and support the studio’s marketing goals.