One of our clients wanted go beyond general web site statistics and traffic, Client wanted a more of a combination audit tool to provide detailed information about:
- Who is using their site? (tracking usernames/roles)
- Click paths and search terms on the site
- What information they are looking at or downloading as paid subscriber?
- What they are using to access the site?information they want to look at.
- And even how many times users have to scroll down the page to get the page to get the information they want to look at.
We started out with researching for tools available out there to log this data and give analytical reports. Here are some of the options we looked at:
- Software Offered by Your Web Host
- Site Counters
- Google Analytics
- Advance web analytics tools
a. Lyris HQ Agency Edition
b. WebTrends
c. SiteCatalyst
Google Analytics proved to be the best pick for our purpose as we were looking for an effective and low cost solution but also with powerful means to extend and customize the analytics. Google Analytics service offered basic out of the box reports for:
- Visits by Browser
- Visits
- Real Time report on Location, Traffic sources
This gave us a bunch of statistics and reports for free. Then we used Google Analytics API to capture detailed user actions like:
- Products added to favorites
- Product view based on filter and search result
- Page view
- Report download
You can set it up in 3 simple steps:
1. First register at the Google Analytics site using your google ID.
2. At the end of registration process, Google Analytics will provide you with JavaScript which you need to copy on every page you wish to track
3. Once you have tracking JavaScript code added into your web pages, your site start gathering data. You can view basic Site usage reports by log-in into Google Analytics site.
Design approach (Image 1) we took to accomplish the task which had 3 parts:
- Generate summary reports within Google Analytics and leverage out of the box reports.
- For detailed user level action reports we captured custom action and events in JSON format and logged data in Google Analytics. Wrote a nightly service to download the data through Google Web API. Data was downloaded and kept in a NoSQL database (RavenDB). The JSON originally generated had all the information captured such as unique GUID to identify user and link all the activity, log in time, logout time, values on which user searched on, filters used by user, tab clicked etc. All the information was stored in JSON format as key value pairs.
JSON data also had a header which contained basic common info across pages and events. E.g.
This design gave the flexibility of adding more parameters if we wanted to capture more information for “Login” user action. This JSON was created on client browser and then tracked using Google Analytics event tracking call.
Google provide very nice WebAPI interface to download data, windows service was written to download the data on daily basis using this WebAPI. This Service downloaded data using Google Analytics webAPI interface and inserted data into NoSQL database. From this database we generated detailed reports which “Client” requested. With all the user event logged in JSON we were able to come up with all kinds of reports:
- Over all Company usage
- Users behavior within companies
- Detailed user usage with a session
- All the user action performed by user with values they wanted to Filter/searched on
Thus we were able to user Google Analytics as Audit tool and provide a flexible and detailed reports at low cost.
Check back for our upcoming article on our use of RavenDB and why it was a perfect fit for storing /querying JSON.
References: 1. https://developers.google.com/analytics/resources/concepts/gaConceptsAccounts
