Performance management and SLAs

Whether you are procuring an on-premise product or a SaaS application, software procurement is often expensive, time consuming and involves disruption to your business. Because of this, it is important that once you’ve procured a piece of software, you’re able to continue using it with as few problems as possible. This makes managing the performance of your suppliers (whether software suppliers, hosting suppliers or support and maintenance suppliers) an important part of the procurement process.

Performance management

Ongoing performance management often includes review meetings between the supplier and the customer, performance reports issued against defined service levels (SLAs) and key performance indicators (KPIs) and third party input, such as user satisfaction surveys.

There is no single prescribed structure for managing the ongoing performance of a contract. The appropriate structure will depend largely on the size and scope of the deal, the nature of the services being provided and the organisational structures of the customer and supplier.

When deciding on what performance management measures to put in place, you should consider what you are trying to achieve from the contract and what factors will impact on whether you consider the contractual relationship to be operating successfully. From a customer perspective, the issues which you will likely want to monitor will include:

  • Whether the software / service is being provided on time and to the required quality
  • Whether the software / service is available when you need it
  • User satisfaction
  • Accurate invoicing
  • Regular and accurate reporting

 

Your suppliers will likely be concerned with:

  • Receiving the necessary input from you within the required timescales and in a clear and accurate manner
  • Prompt payment

Dealing with under-performance

A key element of performance management is dealing with under performance. It is important that the contract managers feel equipped to address any problems promptly as they arise in accordance with agreed procedures set out in the contract. Under performance can be minimised by having a performance management structure that allows prompt and ongoing feedback. This can avoid a problem worsening and can avoid a supplier being confronted by a disgruntled customer raising a problem for the first time which the customer has known about (and been seething about) for some time.

Service Levels (SLAs) and KPIs

SaaS contracts and contracts for the provision of hosting services and / or support and maintenance services often contain service levels (SLAs) and key performance indicators (KPIs) against which the services can be measured.  Measures can include successfully reaching milestones, meeting response times, service availability and other objective measures. Having clear service levels will enable you to monitor the supplier’s performance and will give you the opportunity to raise any issues with the supplier at an early stage if the performance is dropping below the levels detailed in the contract.

If you are procuring a SaaS application or a hosted service, it will be important for you to be aware of what the SLAs and KPIs will actually mean in practice. For example, it is very common for the providers of SaaS and hosted services to offer availability SLAs but, as can be seen from the table below, whilst a 99% availability SLA might sound good, it actually enables the service to be unavailable for over seven hours each month without the supplier having failed to meet the SLA.

 

Availability SLA

Downtime
(per day)
hrs:min:sec

Downtime
(per month)

Downtime
(per year)

99.999%

00:00:00.4

00:00:26

00:05:15

99.99%

00:00:08

00:04:22

00:52:35

99.9%

00:01:26

00:43:49

08:45:56

99%

00:14:23

07:18:17

87:39:29

 

It is also important that you are aware of any exceptions to the SLAs. Using the cloud service example again, it is usual for SaaS and hosting suppliers to exclude certain downtime from the SLA measurement, such as downtime due to scheduled or emergency maintenance. You should ensure that you are aware of this excluded downtime and appreciate the effect that this will have on the SLA as, for example, a 99.999% availability SLA will be essentially worthless if the contract allows the supplier to take the system down for an hour each day to perform maintenance.

Business Hours, Support Hours or 24/7

 Another important issue to address in the contract is whether SLAs are calculated by reference to business hours, support hours or 24/7. The example below shows how a failure to appreciate the difference between these can have a huge impact on the effectiveness of the SLA.

 

The customer identifies a major fault at 4.30pm on a Thursday and the contract contains a response time SLA of 1 hour:

 

 

SLA calculated on a 24/7 basis

SLA calculated on a Support Hours basis defined as 9am – 5pm 7 days a week

SLA calculated on a Business Hours basis – 9am to 5pm Monday – Friday excluding bank and public holidays in England

Maximum timescale for response

5.30pm Thursday

9.30am Friday

9.30am Friday

Unless the Friday is Good Friday in which case 9.30am the following Tuesday!

Assessing severity

SLAs are often dependent on the nature and severity of the problem. It is important that the contract clearly differentiates between the severity levels and sets out which party will ultimately determine the severity of any problems reported by the customer. It should also set out how any dispute regarding the assigned severity level will be escalated (making it clear that the SLA timescales will continue to run while the dispute is being escalated).

How to make service levels work for you

With the above in mind, when purchasing a new SaaS product or negotiating the contract with your hosting supplier, you should give careful thought to the service levels and how they can be tailored around your business in a way that works for you.

Flexible and creative service levels

To get the service levels working for you it is important to be creative and flexible, as long as the service levels agreed upon are objective and measurable.  For example, a 99% availability service level may work for the majority of the year, but in busy periods, for example during your financial year end, or, if you are a retailer, during week leading to Christmas or around the Black Friday sales, even 10 minutes of downtime or more could result in a noticeable fall in online sales and ultimately on your revenue.  Therefore, rather than purchasing all-year-round ‘platinum support’ with a corresponding price tag, you should consider alternative options. For example:

  • Explore the possibility with your supplier of varying the service levels across the year in line with your peak usage periods.  In exchange for 99.99% service availability during peak periods, you may be comfortable accepting lower service levels during periods which tend to be significantly quieter.
  • Place a freeze on non-essential maintenance work during peak periods (which would otherwise result in the service (or part of it) being down).
  • In addition to agreeing different response and fix times depending on the severity of the fault or issue – the more critical the fault, the faster the response and fix times. Consider the value of negotiating quicker response and fix times for the busier times of the year. 

Consequences of SLA failures

 It is also important to consider the consequences of failing to meet the service level, rather than the service level itself.   If the service levels are not achieved, SaaS and hosting contracts often require the supplier to provide the customer with a service credit.  However, as noted above, the consequences for you if the supplier fails to achieve the service levels may be vastly different depending on the point at which the failure occurred.  Therefore, you could consider attaching a multiplier to the service credits for failures which occurs during peak usage periods.    

Incentivised / motivational service level models

If you are procuring high demand or business critical software, in addition to seeking meaningful remedies for service level failures, you might also seek to incentivise your supplier in achieving higher service levels by providing the supplier with the ability to earn modest bonus payments if they can achieve higher service levels during peak usage periods.

Escalation procedures when things go wrong

Most software procurement contracts contain an overview of how disputes and concerns can be raised and to whom they can be escalated if they are not resolved within a specified time.  However, the importance of these escalation procedures is sometimes overlooked and it is important to consider whether the timeframes in which disputes or concerns are escalated work for you and you are comfortable that the persons to whom the disputes are to be escalated have sufficient authority to deal with the issue promptly and efficiently. 

For further information please contact Kathryn Rogers on +44(0)1892 506 147 or at kathryn.rogers@crippspg.co.uk.