COMMON ADSENSE CRAWLER ISSUES AND HOW TO FIX THEM - ░▒▓█ Face Insider⌥ █▓▒░ | Social media marketing

Post Top Ad

Saturday, April 7, 2018



It’s not easy being a publisher – it demands time and focus, not to mention some necessary technical  knowledge to be able to manage your websites.
For publishers who focus on monetizing their ad inventory, it is a huge plus to know what AdSense crawlers are and how they can help you to fix known issues. Fixing these crawler errors will help Google’s crawlers better access your site so it can display more relevant ads based on content — which we know can drastically improve your ad revenue. Fixing these issues is like building an Ikea bookcase: some assembly, no Ph.D required.
So before we ramble on, let’s just clarify that AdSense crawlers are different from other Google bots.  AdSense crawlers specifically index website content for the purpose of sending appropriate ads. More often than not, this type of crawler will access site URLs where the AdSense tags are already implemented, including pages that redirect. This is why regular website maintenance is imperative to avoid any issues with Google.

Now, Google crawls automatically at any time but reports are usually updated on a weekly basis. You may find a variety of errors when checking the crawler report in your AdSense account, and we can help you figure out exactly what to do.


1) Page Not Found
Also know as Error 404, this is one of the most common errors seen by webmasters and page visitors alike. It simply means that the crawler attempted to crawl a page that may have been previously deleted or repositioned elsewhere on your website. A temporary URL can cause this as well. There is a fantastic tool that can help you find these pages to avoid this type of error: Webmaster URL parameter tool.
2) Robot Denied
This means that the AdSense crawler tried to crawl a page in the domain or subdomain level. Check the reports to find out where exactly. If this happens, Google will not be able to index that specific page and they won’t be able to serve ads on that page.
To resolve this, simply remove these 2 lines from your robots.txt file to grant Google crawler access:
User-agent: Mediapartners-Google
Disallow: /
3) Content Behind a Login
There are a lot of websites that require login information to get premium access to the site’s main content, and this usually means that a crawler login hasn’t been set up for that premium content. The same as the error #2 above, Google will not be able to access this page and Google ads will not be served. Imagine if you have 3,000 premium users — how many lost impressions do you think will that be?
This is thankfully very simple to resolve. Just log in to your AdSense account; under Access and Authorization in Settings, go to the crawler access and provide login details for the crawler to use to access your site. You may want to check out the Google’s step-by-step guide about displaying ads on login-protected pages.
4) Crawler Error for a Site I Don’t Manage
This error means that someone is using your ad code in a different site without your permission. The impressions and clicks will still be counted but will not be paid. Therefore, you will not be earning as this is not authorized.
If this ever happens, set your AdSense account to “Only allow certain sites to display ads for my account.” This is an option available under Settings > Accounts and Authorization. Expect changes to happen in 48 hours.
There you have it. Crawler issues are remarkably straightforward and easy to fix, and you don’t even need a Swedish assembly guide.
MonetizeMore helps publishers with website and ad optimization issues. If you’re having problems with your Adsense inventory, contact us to discuss your need.You may also sign-up for FREE to the exclusive Google AdExchange – the best alternative for Adsense.

No comments:

Post a Comment

Post Bottom Ad