Analytics Referrer/Ghost Spam Blocker

What is Referrer or Ghost Spam?

Referrer or Ghost Spam are domains in your Analytics referral traffic which are added there by spam bots and not by people visting your site via a click on a third party website. Though this traffic is not harmful, it does pollute your Analytics data and your statistics. Below you see an example of those domains.

Example of Spam Referrers in Google Analytics.

There are number of ways to fight Google Analytics spam. We decided to use Google Analytics filters to exclude spammy domains with our tool Referrer Spam Blocker.

Traffic Spam statistics

The analysis of 292 Google Analytics accounts which had al least 25 referrer visits per day to find out how big the problem actually is. The referrer spam traffic is about 11%. Also more than half of all websites had spam traffic.

More than 500 domains were spam. Here the bad guys were Semalt and AliExpress. These domains got most redirects from the spam domains.

Analytics/Ghost Spam statistics 2015.

Two types of Analytics spam

There are two types of referral spam: ghost spam and crawler spam:

1. Ghost spam directly visits Google Analytics, it never actually visits your website. It uses the Measurement Protocol to send data directly to Google Analytics’ servers. By using a fake hostname and randomly generated tracking codes, the websites leaves fake data on your server without knowing who they are targeting.
2. Crawler spam does visit your website. They are harder to identify since they use real data and know who they are hitting. When they visit your website it looks like a legitimate visit. This can be used as a marketing tool since the domain of the website will show in your analytics data and you might then click on it.

How to prevent referrer/ghost and analytics spam?

There are different solutions for the two types since they both infect your data in different ways.

Block/filter Ghost Spam

Ghost spam can be filtered/blocked from your data with a hostname filter since ghost spam uses a fake or no hostname. You will need a list of the fake hostnames that have been found in order to block them.

Block/filter Crawler Spam

There are multiple ways to remove crawler spam from your data. With a Campaign Source Filter in Google Analytics you can filter the crawlers from your data, they will not be removed though. With a filter in the .htaccess file the crawlers will be removed before visiting the website. However, blocking them in the .htaccess file is tricky and can crash your website if you make mistakes. For both solutions you will need a list of all spam domains that have been found so far.

The following solutions work too, but not perfectly.

1. A screen resolution filter can help since a lot of the spammy domains have not set a resolution.
2. A new tracking ID can be used but only if your data is not affected yet. This does not prevent spam from visiting your website but it makes it a bit harder.
3. Bot Filtering is a function in Google Analytics that can be used against referral spam. However, it does not remove everything. It is the easiest solution though.
4. You can use custom segments in the Advanced Segments setting in Google Analytics to filter some of the spam.

The best way to get rid of the spam in your data is by using a combination of all these solutions. This costs a lot of time so fortunately there are tools that will do the work for you.

What is Analytics Spam Blocker?

Google Analytics Referrer Spam Blocker is a free tool created by AdWords Robot to help webmaster and site owners to keep their Analytics data and statistics as clean as possible.

How does Referrer Spam Blocker work?

Filters to block spam

There are 35+ filters which will applied to each view and these filters will exclude traffic coming from spam bots. There are 2 types of filters:

1. Hostname filter: A filter which excludes traffic where hostname is empty i.e. hostname == '(not set)'. The name for this filter is AR-Spam-Killer-Hostname.

2. Screen resolution filter: A filter which excludes traffic where screen resolution is empty i.e. screenresolution == '(not set)'. The name for this filter is AR-Spam-Killer-Screen-resolution.

3. Domain filter: A filters which excludes known domains (see below) i.e. CampaignSource MATCHES 'domain1\.com|domain2\.com|...'. Because we cannot put all domains one filter (there is a limit of 255 characters per filter match criteria), we need to create 35+ of them. Domain filters are called AR-Spam-Killer-Domains-X, where is X is a number 01..35.

Using these two filters lots of spam domains will be filtered out, but probably there are some unknown domains out there. If you have any new additions which are not in the list (see below), please send an e-mail to blog@adwordsrobot.com.

What happens?

After you select a view and click on Apply filters, 3 things happen:

1. We remove our old filters (starting with AR-Spam-Killer-*) from selected accounts. With this all account views will lose their filters. This means if you want to apply filters to an account you previously applied filters to, you must select ALL views you wish to update.

2. We add new filters to selected accounts.

3. We link new filters to selected views. All other views will be cleared of our filters.

If everything goes well, all views will have the right filters connected to them. If something goes wrong, for example you have no 'edit' permission to one of the account, you will get a warning and the tool will continue with other accounts.

How long does it take?

Each operation takes around 0.5 seconds. The number of operations is calculated as follows: 2 * (#accounts * 30) + (#views * 30) operation. So if you select 6 views in 3 accounts, this will generate 2 * (3 * 30) + (6 * 30) = 360 operations.

Most of these operation are write operations, therefore we need edit access to the Analytics accounts. If there is a permission error, the account will not be updated. We are still working on a good status messaging to show you accounts which were skipped.

Which domains are excluded?

The following domains are supported. If you have any new additions which are not in the list submit them.

FAQ

Will this remove spam from my Wordpress, Blogger, Drupal, Joomla, etc?

This tool will stop and prevent traffic spam from being added to your Google Analytics. Also when you use Wordpress or Joomla site. If you are using other analytics software, like webalizer or PiWik, this tool will not work. You can download our open source spam domain blacklist from GitHub.

Can I download this spam script and use it on my own?

Our tool is not open source, but the whole script can be found in the source code of this page. If you copy it, please let us know and we would be glad if you would link to us.

How do I remove my given permissions?

We are not storing your access token information after you login, but you can deauthorize the tokens by going to Google Account Settings for the account you used and remove AdWords Robot GA Referrer Spam Blocker.

How much does it cost to use the tool?

The tool can be used for free.

Current features

This tool is still work in progress, so new featres might be added in near future.

• Apply filters to multiple accounts and views in one go.
• Remove filters we created before (filters starting with AR-Spam-Killer).
• Search through account with a handy tree view of accounts/properties/views.
• Extended Google Analytics limits.
• Up-to-date spam domain list, containing 500+ domains.

Limitations

Google Analytics has a limit on number of operation per second and number of operation per day. There is a chance we hit this limit. You will get a message when you do and you will have to wait couple of hours before you can run the tool again.

In any case, please don't abuse our limit :)

Disclaimer

The software is provided “as is”. To the fullest extent permitted by local law, AdWords Robot does not warrant the software, either epressed or implied, for any purpose. AdWords Robot expressly disclaims any implied warranties including, but not limited to, the warranties of merchantability and fitness for a particular purpose.

AdWords Robot does not guarantee that the software will perform without interruption or error-free and under no circumstances shall AdWords Robot be liable, whether under contract or tort, for any direct, indirect, incidental, special or consequential damages arising from, or in connection with, or as a result of the use of the software by you or any third party, even if AdWords Robot has been advised of the possibility of such damages.

To the extent some jurisdictions do not allow the exclusion of implied warranties or the limitation of legal rights of the consumer, it is possible that these exclusions and limitations may not apply in such jurisdictions.