When it comes to scalability and performance, challenge your vendors. They may not be telling you the entire story.
Introduction
In the context of software, what does scalability mean? This seemingly simple question has been plagued with misinformation, misuse, and unclear marketing, leading to confusion on a critically important topic. I'd like to help dispel some of this confusion with a thoughtful analysis of scalability.
What scalability is and is not
The first thing we need to do is to understand what we mean when we say a piece of software is scalable.
What it is
Definitions of scalability vary widely. For example, the Wikipedia version states, "Scalability is the property of a system to handle a growing amount of work by adding resources to the system." Gartner defines scalability as "[t]he measure of a system's ability to increase or decrease in performance and cost in response to changes in application and system processing demands." And there are plenty of others.
Despite the many ways to define scalability, some elements are similar across definitions. For example, scalability focuses on two aspects of a system: hardware and software.
Any platform will have different performance limits based on the hardware it's running on. And any given machine or network of machines operate with a limited number of CPUs, amount of RAM, disk space, and so on. For hardware, scalability is a measure of how many resources you have.
With software, scalability involves the necessary libraries and code needed to perform its job. Each piece of software requires a certain amount of server resources each piece of software needs in order to function. Software scalability is a measure of how much it costs to run.
Furthermore, scalability is a matter of how a system responds to current and future requirements. While many scenarios relate to this characteristic, many organizations want to ensure that what they've built today will continue to operate with tomorrow's demands.
What it is not
In short, anything not related to a system's ability to handle requests given a set amount of resources, is unrelated to scalability. Surveying the internet, here are some of the pieces of misinformation I found regarding scalability:
- Scalability includes security. Although security is absolutely critical to any solution, you should follow proper security policies and practices regardless of the rest of the implementation. Therefore, you can leave security considerations out of a scalability calculation.
- High availability influences scalability. This could not be further from the truth. High availability reflects the reliability of your platform, not how it performs given particular workloads.
- Auto-scaling results in scalability. This is a tricky one. The ability to auto-scale is beneficial, but it doesn't imply efficiency. In fact, auto-scaling architecture can operate without efficiency in mind.
- Scalability is about the features on your platform. You should only evaluate a platform's features compared to those of other platforms; you should not factor platform features into scalability calculations. While each set of features will have performance implications, it doesn't have implications on the system's holistic performance.
Any time you hear language like this, you should question that vendor. Look for vendors that are willing to help you understand how your particular solution will scale.
Considerations When Determining Scalability
When determining the scalability of a software platform, you should consider four variables:
- Number of concurrent users. How many users will need access to the system at once? Be sure to include transactions requested by service accounts or other automated means.
- The average number of requests per user. This includes things like the number of calls to a particular service or internal communication between platform services.
- The average size of each request. How large is the particular request? Rarely are requests the same size. Improper estimation of actual size requirements can skew scalability estimations greatly.
- The size of the machine the platform is running on. Different hardware configurations will perform differently. Test on a system that closely resembles what you'll be using. Consider the number of processors, size of RAM, and disk space. Getting this size right, whether you're upscaling or downscaling, is important.
These variables help you evaluate a system's scalability. For example, let's say I have 10 concurrent users running 100 operations a day, and each operation averages 1MB of data. Let's also assume this amount of service consumes 10% of the hardware's resources. If I add 10 additional users, will it only consume another 10% of system resources? If yes, this platform is linearly scalable.
Software can go beyond linear scalability through parallelization, but this is a topic for another day. It's also important to note: Just because a solution follows linearity at one point doesn't mean it will always follow this trend. There's a limit to linearity, at which point adding additional resources will yield less return.
It's All About the Cost
The reason we check a platform's scalability comes down to money. How much work can you get done as efficiently as possible? Even if a solution shows good performance, it has to do it in a cost effective manner.
I recently looked at a different vendor's published scalability numbers. While they were showing some really impressive figures, the vendor was running those tests on a $26 million mainframe computer. Consider that you can get a top-of-the-line cloud solution on Amazon for about $4.50 per hour, which you could operate for 660 years before the $26 million mainframe becomes cost effective. For your purposes, it doesn't matter if the vendor's system can host 10,000 or 100,000 people. It's still a bad deal.
Another "gotcha" happens when dealing with vendors that create artificial scalability challenges. This comes in the form of fees per transaction or fees per unit of solutions. Options with this type of pricing may seem like a good deal to start, but you, the customer, will be forced to make bad design decisions.
Imagine that you are being charged per API call. Wouldn't you try to increase the payload size of each call to maximize efficiency? Sure, but what if your use case requires many calls over a long period of time? Now you must decide whether to create a solution that becomes bloated because it has to carry around a single large payload, or pay the extra fees to build your solution right.
Object-based fees pose a similar scenario. If you're paying per object, you'll try to condense your design into as few steps as possible. This creates an artificial need to overdesign elements in your solution, which can lead to suboptimal performance.
Of the variables described above, call sizes and transaction numbers are the easiest places to obfuscate performance results. Vendors are savvy. They know by charging fees in these areas is an effective way to hide the actual costs of a solution.
How to Solve this Problem
Let's tie this all together. Assuming the information above is true, here are some fairly universal best practices for evaluating a platform's scalability.
Establish a Baseline
It's critical to know where to start. For any use case, know the user count and what you need to do. From there, estimate the resources each person needs. Finally, choose a hardware setup the you think will run this solution.
Test Test Test
Test your baseline. How did it perform? Once you've tested that, modify your variable and test again. How does the solution run with double the users? Users times 10? Users times 50? See what happens when you introduce a new operation to the mix by increasing the number of calls and/or size of the call. When does the hardware you're using no longer perform well? If you increase the size of the hardware, either by scaling up or out, what happens?
Know the Hidden Costs
Remember, scalability is about more than test results. While a test will tell you whether a solution is scalable, it won't tell you whether that solution is cost-effective. Watch out for hidden costs like transaction charges or extra fees per object or unit. While the system may be able to handle the work, your checkbook might feel differently.
Who's Responsible for Ensuring a Scalable Solution?
Here something you never thought you'd hear from a software vendor: Scalability is your problem. Whether you build the solution yourself or pay a third party to do it, you'll have to live with the consequences of the overall design.
Any software platform out there can scale poorly. Conversely, with an unlimited budget on hardware, you can make any of those solutions work.
If a vendor tells you their solution scales and you don't need to worry, challenge them. I believe Appian has one of the better-performing platforms in the marketplace. Our services are written on a more performant framework compared to our competitors. In addition, Appian evaluations can be performed in parallel, something other vendors don't support. Even with these advantages, I don't want to mislead you in thinking that's the end. Beyond the back-end power of our platform, we want to work with you to ensure you're building the most scalable applications possible.
How Appian Can Help
Appian wants to help you create the most scalable, cost-effective solution available. We can tell you we're the best, but we'd rather show you.
If you want to get the hard numbers for Appian's performance, check out our scalability white paper. In it, you will find out how well our platform performs on three separate hardware instances.
Once you've read the whitepaper, come talk to our sales team. I think you'll be surprised to find out how interested we are in earning your trust and becoming your enterprise vendor of choice. We're more than willing to walk you through our platform and help you figure out the most effective way to use our product.
Scaling enterprise applications on mission critical systems is not easy. Tell us your story and we'll help you fully understand how to effectively scale with Appian.
