Real Browser Performance Testing with Jmeter

In case you want to have more realistic performance test results , use of browser based performance testing is the way to go. As we know that Selenium has capability to perform browser based testing using WebDriver.  This blog describes step by step procedure to configure Selenium WebDriver with Jmeter and how to use it for performance testing activities.

Configuring WebDriver Plugin:   

1. Download Selenium web driver plugin from jmeter-plugins.org
2. Copy extracted jar files into /lib and lib/Ext
3. Delete Older / Duplicate Jars files from /Lib  

 

Once you have completed above steps , you are done with configuration. 

 

Creating Browser based Test Script in Jmeter - WebDriver Plugin: 

 

• Open Jmeter and Add Thread Group
• Add Firefox Driver config' from config element

 

• Add WebDriver Sampler
• Add WebDriver code

 

 

• Add Listener for Debugging
• Run the Test

 

Understanding Google Analytics Metrics

Before we start reading about Google Analytic measures, it is worth to have a little background about basic concept of dimension  and metrics.

In Google Analytics we have two types of data :

1. Dimensions :  It describes characteristics of users , their sessions and actions. The dimension city describes a characteristics of sessions and indicates the city for example "Paris" , from which sessions was originated. The dimension page describes a characteristics of page view actions and indicates the URL of each page that was viewed.
2. Metrics : These are simply quantitative measurement of users, sessions and actions. Metrics are numerical data , basically they are numbers.

When extracting data for City as a primary dimension and Browser as a secondary dimension , we  will have following view :

 Google Analytics Metrics :

1. Visitors or Users : This metrics measures number of unique users that visit your site during certain period of time. This is most commonly used metrics used to measure overall size of audience. This can further be categorized as New Visitors and Returning visitors. This number is more accurate in telling you how many individual people visited your web site. In order to perform work load modelling for your performance test , always ask this metrics as part your requirement gathering.
2. Visits or Sessions :  Visits are also known as sessions , are defined as a period of consecutive activity by the same user. By default in Google Analytics , a session persists until a user stops interacting the sites for 30 minutes. 
3. Pageviews : Within each visit or session , your users will engage in one or more interaction with your web pages. Google Analytics will automatically track these interactions as "pageviews". Pageviews metrics counts every time a page is viewed on your site.
4. Pages per Session : This is average number of pages viewed during a session on your web site. More pages per session indicates user is quite engaged with your web site. 
5. Average Session Duration : This is average length of user's sessions. Higher number again indicates users are more engaged with your web site.
6. Bounce Rate : This is percentage of visits that are single page only i.e users who visits single page and leave. 
7. % New Sessions :  This is an average percentage of first time visitors on your web site. 

We will get all above mentioned metrics under Audience Overview :

Performance Testing of Micro Service Based Architecture

 

Background:

Micro services are a style of software architecture in which system is delivered using a small set of services which are granular, independent and collaborating services. It is a technique of applying single responsibility principle at architectural level.

Micro services are often integrated using REST over HTTP. 

Layered Architecture of Micro services :

Resources handle incoming requests. They validate request format, delegate to services and package response. For RESTFUL services this includes deserialization of requests, authentication, serialization of response and mapping exceptions to HTTP status codes.

Services represents core business logic. They may collaborate with other services, adapters or repositories to retrieve required data to fulfill a request. Services only consumes and produces domain objects. They don't interact with DTOs from persistence layer and transport layer objects.

Adapters handles outgoing requests to external services. They are responsible to marshal requests, unmarshal responses, and map them to domain objects. Object mappers are widely used at this layer.

Repositories handles transactions with persistence layer.

A lightweight micro service may combine one or more of the above layer in a single component.

Performance Testing challenges:

1. Whole application is not available from starting instead we have set of fully functional modules which later plug into end product.
2. Use of different technologies in micro service development.
3. Interaction among micro services is not readily available unless there is not single point of contact who has complete view of entire solution.
4. Performance monitoring is a big challenge considering different technological aspect in micro services development ( Message Brokers, NOSQL, DataBase, N number of independent running JVMs etc).
5. We need different benchmarks for capacity planning considering benchmarking activities specific to technologies (e.g  Heap sizing needs different benchmarks for JVM based services and different for allied services like Node.js apps and .NET CLR will also needs its separate benchmarks)

Performance Testing of Micro Services - Approach

While designing performance testing approach for micro service based architecture, we should consider solutions to above mentioned challenges.

1. Consider shortening your selection from lot of services bunch to have focus on those services which represent critical business activities model.
2. Try to build a service interaction diagram for all of your performance scenarios.
3. Always start with performance testing of services in isolation manner rather than replaying end to end business scenario in an integrated environment. Once you have separate performance benchmarks for each services in isolation then focus on integrated test / business scenarios.
4. Performance monitoring is critical during testing activities , and use of commercial agent based monitoring solution can increase cost of testing to great extent. Although SaaS based monitoring solutions is also a solution but use of open source tools and in house developed API profiling solution will reduce testing cost to great extent.

                                                         Testing Phase Pyramid

          

Disk I/O in Amazon RDS and EC2 instances

Disk I/O in Amazon RDS/EC2 instances

 

Amazon RDS/EC2 uses EBS volume for database and log storage. Depending on the size of storage requested , Amazon RDS automatically stripes across multiple EBS volumes to enhance IOPS performance.

Amazon EBS provides 3 volume types :

1. General Purpose (SSD) volumes

2. Provisioned IOPS (SSD) Volumes

3. Magnetic Volumes

Before we further discuss characteristics of each of these EBS volume, let us first take a look what IOPS are and how measurement take places for these.

What are IOPS:

IOPS are input and output operations per second. Amazon EBS measures each IO operation per second ( that is 256 kb or smaller) as one IOPS. I/O operations larger than 256KB are counted in 256KB units. For example 1024 KB I/O operation will be counted as 4 IOPS.

General Purpose (SSD) volumes:

General purpose (SSD) volumes offers cost effective storage that is ideal for a broad range of workloads. These volume can offer single digit millisecond latency , the ability to burst to 3000 IOPS for extended period of time. These volume can range from 1 GB to 16 TB.

General purpose (SSD) volume provide baseline performance of 3 IOPS/ GB and maximum of 10000 IOPS  (3334 GB). Throughput of these volumes range in between of 128 MB/s to 160 MB/s.

Performance of general purpose (SSD) volume is governed by volume size.  Higher the size , higher accumulation of I/O credits where I/O credits represent available bandwidth that General Purpose (SSD) volume can use to burst large I/O when more than baseline performance is required.

Provisioned IOPS (SSD)  Volumes:

Provisioned IOPS (SSD) volumes are designed to meet the need of I/O intensive workloads , mainly database workloads that are very much sensitive to storage performance.  Here while configuring (creating volume) provisioned IOPS (SSD) volume, we have to specify desired IOPS rate.  These volume can range from 4 GB to 16 TB with a maximum throughput 320 MB/s. We can achieve maximum of 20000 IOPS with these volumes.

Magnetic Volume:

Magnetic volumes provide the lowest cost per gigabyte of all EBS volume types. Magnetic volumes are backed by magnetic drives and are ideal for workloads performing sequential reads, workloads where data is accessed infrequently, and scenarios where the lowest storage cost is important. These volumes deliver approximately 100 IOPS on average and they can range in size from 1 GB to 1 TB. 


 

APM - Real User Monitoring Vs Synthetic Monitoring

As part of setting up application performance monitoring framework , this is very critical to measure end user experience. End user experience monitoring sits on the top of APM framework. Further business judge their web application performance in terms of responsiveness of application. There are two approach by which we can monitor end user experience :

1. Real User Monitoring ( RUM)
2. Synthetic Monitoring

Real User Monitoring ( RUM):

While server side performance can be measured by looking HTTP requests in data center, full page load experience which includes downloading of static contents, rendering page, executing java script - can not be seen from server monitoring aspects. Real User Monitoring is the practice of using java script agent embedded in web page to gather performance data about end user's browsing experience.

Synthetic Monitoring:

Synthetic performance monitoring involves having external agents which run scripted transaction against web application. These scripts are meant to follow steps which is expected from typical user behavior. This might be searching product , log in activity, hotel booking etc.  Synthetic monitoring don't track real user's session.

We have to careful while setting up monitoring , which one best suites to our needs. While Synthetic monitoring provides details about reliability, availability whereas RUM provides real user browing experience.

Happy Monitoring !!!!!

String Deduplication - New Java 8 update 20 feature

Duplicate Strings in JVM heap ..Worried ??


During JVM heap dump parsing, it is quite common to visualize that Strings are consuming lots of heap space. Especially char[] are always ( most of the cases) present as biggest object in histogram.

If we perform a deeper analysis on String objects it is quite common to encounter thousands of duplicate String instances. Controlling these duplicate Strings may provide improvement in overall JVM memory footprints.

With Java 8 update 20 we now have a new features called String Deduplication which require G1 garbage collector in place and this feature is turned off by default.

Here G1 GC identifies Strings which are duplicated more than once and correct them to point to same internal char[], to avoid multiple copies of the same string.

Parameter :  -XX:+UseStringDeduplication

How to capture Connection Time in Jmeter 2.13

Jmeter 2.13 has introduced a new key performance indicator that is connection time.  Why is this important  ? It takes time to connect to a server before making a HTTP request and this can have impact on response time , specially for HTTPS traffic.

Starting from 2.13 version , new metric ConnectTime has been added. It represent the time to establish connection. By default it is not saved to CSV or XML. In order to save it we need to add following line in user.properties :

jmeter.save.saveservice.connect_time=true