How To Scale Your High Traffic WordPress Site

WordPress is hands-down the most widely used Content Management System (CMS) on the planet and with that, scalable WordPress hosting is concept that you’ll see often across different WordPress hosting company websites.

But what does scalable WordPress hosting really mean and are there things I can do to ensure that my WordPress site is scalable?

Lets dive into some of the challenges we must overcome when scaling your WordPress site. Many hosting providers say they offer scalable hosting for high traffic websites but do they really? If you’ve purchased software or done any work with IT vendors previously, you likely know that you have to be able to verify these claims yourself or you could be risking serious downtime for your WordPress website and business.

Feel free to use our table of contents below to jump to an appropriate section for your convenience.

The Basics of Website Scaling

As with any successful website or business, like Ollie, scaling is an eventuality every admin must one day consider. There are many challenges with running a platform that reaches millions of visitors throughout the world. Scaling a project to handle spikes in demand is no simple task in any industry and the IT world is no different.

WordPress websites in particular can very quickly hit several bottlenecks with server resources if not planned for appropriately. Scaling upward is a multi-faceted complex problem with very few easy answers. However, it can be done effectively if you know what caveats that you need to look out for.

We’ll touch on a the basic plot points of website scaling briefly before tackling the WordPress site specific concerns.

A rudimentary understanding of these factors will help you better grasp scalable WordPress hosting so that you can make the right decision when selecting a hosting provider. You should be confident that the web hosting provider that you select can handle the traffic spikes or constraint on server resources that you are experiencing or could be looming.

Vertical Scaling vs. Horizontal Scaling

The natural progression with site growth over time typically goes vertical scaling before horizonal scaling. This means the server housing your website starts small, on a single server. If very small, a shared server along with dozens of neighboring sites of similar size. This is typically referred to as “shared hosting” on most hosting provider websites.

Then you’d normally progress to a single server that houses your all of your WordPress sites or domains. This one server grows in size adding more and more server resources until eventually you hit the zenith.

Our Monolithic server then splits off horizontally, first adding a dedicated database server, then more web nodes with a load balancer to handle the growing userbase of your site.

Vertical Scaling – Monolithic Servers

The practice of using a monolithic server to overcome system resource bottlenecks is called Vertical Scaling. Monolithic servers are a single physical server with high-end multi-socket/multi-core processors, massive banks of memory, and servers that host all or most of a website’s required services. Your typical growth path for a website’s growth looks like:

Horizontal Scaling – Server Clusters

Horizontal scaling uses load balancing and data replication techniques to effectively distribute requests proportionally among a group of server clusters. Server Clusters are a set of like-purpose physical systems, called nodes, working in unison to emulate a single hosting service that is distributed among all cluster members evenly or proportionally. Superior performance, reliability, and scalability are the hallmarks of clustered servers and the backbone of horizontal scaling solutions the world over. You will be hard pressed to find a single large company who does not employ this type of scaling.

Vertical Scaling – Monolithic Servers

The practice of using a monolithic server to overcome system resource bottlenecks is called Vertical Scaling. Monolithic servers are a single physical server with high-end multi-socket/multi-core processors, massive banks of memory, and servers that host all or most of a website’s required services. Your typical growth path for a website’s growth looks like:

• Shared Hosting Plan (SHP)
  • Virtual Private Server (VPS)
    • Monolithic VPS (M-VPS)
      • Dedicated Server (DS)
        • Monolithic Dedicated Server (M-DS)

Horizontal Scaling – Server Clusters

Horizontal scaling uses load balancing and data replication techniques to effectively distribute requests proportionally among a group of server clusters. Server Clusters are a set of like-purpose physical systems, called nodes, working in unison to emulate a single hosting service that is distributed among all cluster members evenly or proportionally. Superior performance, reliability, and scalability are the hallmarks of clustered servers and the backbone of horizontal scaling solutions the world over. You will be hard pressed to find a single large company who does not employ this type of scaling.

• Monolithic DS + Database Server
  • Load Balancer + Web Cluster + Database Server + File Server
    • LB Cluster + Web Cluster + DB Replication Cluster + File Server
      • LB Clusters + Web Clusters + DBR Clusters + FS Cluster

SSD vs HDD

Depending on the current stage of your infrastructure you may be using standard Hard Disk Drives or the faster Solid-State Drives. You will want to make sure you are employing drive types effectively.

Our recommendation: SSD drives for the operation system, NVME-SSD for database and other high I/O file systems.

Standard HDDs have little place in hosting active services. They can be employed for non-mission critical services like backup or long-term file storage. Trying to run a large scale WordPress website on standard HDDs will certainly fail almost immediately upon any significant surge in concurrent users.

Disk Caching & Reverse Proxy

Server performance for any WordPress site must effectively apply caching technologies to be successful. The higher the traffic demand, the more unrealistic it becomes to have the webserver actively processing site code for each incoming request.

Caching stores prerendered pages and code snippets as static content that requires active processing or database interaction to serve up to visitors. Store your most important content caches in memory with services like Memcached, Redis, and Varnish to eliminate any disk speed bottlenecks with accessing frequently used objects.

CDN (Content Delivery Network)

Front loading static website content (i.e.: CSS, HTML, images, JavaScript, PDF, and text) onto a geographically distributed network is a proven strategy leveraged by all high traffic websites on the web.

You will need to have adequate geographical diversity in the regions your content is expected to have the largest demand. Providers like Amazon CloudFront, CloudFlare, Google Cloud CDN, StackPath/MaxCDN, Rackspace, Sucuri, and many more… are massive networks of interconnected content caching servers throughout the world.

Their CDN services make it easy to have reach even beyond your principle locality by caching your content on their infrastructure before the request ever makes it to your web servers.

Database Optimization Milestones

There are several database performance milestones we want to address when planning to scale-up your website(s), be they Joomla, Magento, WordPress, or any other content management platform.

The following items are known database bottlenecks that you must address before a large-scale audience can be effectively reached:

Table Engines – InnoDB vs MyISAM

Make certain you are using proper table engines that is tailored to each of your database tables unique needs. A major factor in scaling websites is table structure. Use the right tool for the job else your performance will fall short, turning your once promising spike in interest into a detrimental drop in reputation due to inaccessible or slow moving websites.

InnoDB tables are for high-performance massive tables stretching from the hundreds of thousands into the millions of rows. InnoDB makes these immense data structures manageable through row-level locking to limit the effect a single locking event has on the entire table.

MyISAM tables have their own niche as well. They are the superior choice for small datasets, sub-100k rows, that require a lot of write activity. Speeding up processing time by locking the entire table is much faster and less resource intensive than the overhead of tracking table-row locking like with InnoDB.

While this particular article does not go beyond the confines of InnoDB and MyISAM, these are not the only two table space engines available. Feel free to explore the other engines thoroughly and consult with a DBA to assist with any assessment.

Indexing

Database indexing is a critical milestone to get right. Consulting a developer or Database Administrator (DBA) to design an effective schema for your software’s unique data structure is a crucial milestone to get right. Large datasets cannot function without effective indexes speeding up the most frequently read data points. Depending on the usage case, some tables may require multiple indexes for different query formats. Making these decisions is exactly why the DBA role exists in the field.

Memory & Buffers

Adequately sized buffers fine-tuned by an experienced DBA or developers that gets the maximum performance out of the available physical resources is a must. Front loading the important datasets into memory buffers should cover all the most commonly accessed and/or changed data tables. Ideally, all of your important indexes should reside directly in memory and not require swapping data to a cumbersome disk that will tank the performance of such queries.

Periodic Maintenance

Maintaining tables sizes under the tens of millions is a critical performance metric for mission critical service performance. Periodic maintenance should syphon off data from the primary database cluster onto a secondary backup archival cluster to keep the database servers responding quickly and efficiently with their active loads. Use reasonable judgement for how much data to keep directly in your production database cluster so that you’re not undermining your own performance due to data hoarding habits.

WordPress Scaling Challenges

There are a number of concerns that come up when specifically dealing with WordPress sites. The same holds true for every other piece of software. They each have their own major performance bottlenecks that need to be addressed.

The following sections are the unique pain points I’ve personally witnessed with clients attempts to scale-up their WordPress driven sites.

WordPress Themes

Choosing a scalable theme is more important than you might think. WordPress themes are not just a pretty face to the platform. They are an integral part of the site’s performance, site speed and scalability. I have seen entire sites go down due to traffic spikes and enormous losses in revenue simply because the chosen theme was never written with performance in mind.

This becomes more prevalent when moving from the stage of Vertical Scaling into Horizontal Scaling mentioned above.

For example, the vast majority of themes on the marketplace were not written with server clusters in mind. Something as simple as storing cached data internally within their directory structure and not allowing for this important location to be moved to a more suitable file system can cripple certain types of file replication solutions, ultimately tanking WordPress site performance when it’s needed most.

Engage with your theme developers to make sure they can assist with any corrections needed for scaling the site upward. Turn off, do not just hide, unnecessary elements, widgets and features. When it comes to performance, you want only those necessary components to be included in the server and browser processing stacks.

WordPress Plugins

WordPress lives and breathes plugins. These mostly third-party expansions to the core functionality of WordPress are what makes WP such a lucrative platform. When it comes to plugins, there are certain ones that are required for high-scale WordPress sites. What’s equally important is knowing what types of plugins to avoid. Since plugins cover such a broad set of features, the remaining sections will each talk about plugins

WordPress Caching

Having a proper caching tool that understands the quirks of the WordPress ecosystem is one of the most critical components to get right.

Luckily there are a large number of equally proficient solutions already available on the WordPress plugin marketplace. There are both free and premium options. Deciding which caching plugins is best for your needs is a topic for another time as it is a quite a lengthy one. However, having any one of these types of plugins installed and caching your site is a basic necessity. You can use the button below to visit the WordPress Plugins marketplace and view a list of caching centric plugins to evaluate and choose.

WordPress Caching Plugins

WordPress Cron & Scheduled Tasks

The way WordPress approaches scheduled tasks has long been a common scaling problem. One of my first technical publications was a deep dive into this specific subject over on Medium. It’s still just as relevant today in 2023 – The nightmare that Is WP-Cron.

My personal opinions of wp-cron aside, the key take away is making sure that you disable the default wp-cron.php behavior. Then setup a server side cron job to execute wp-cron.php on a fixed interval. This ensure that the scheduled posts and other cron related features work properly while not simultaneously taking down your web server.

When scaling horizontally, care must be taken so that scheduled tasks are ran from a single node and not once per node. This can cause strange behavior like double posts, double payments, and other such repetitive behavior that should not be conducted more than once.

WordPress Security

The web is chock-full of ever-present dangers that should not be taken lightly. Running hardened service configurations on your web server is an absolute must. ModSecurity is compatible with all web servers. It is the unparalleled industry giant Web Application Firewall (WAF). It can detect and evict badly behaved inbound requests at the web server level. Any sites running too laxed by failing to source a proper repository of ModSecurity rules is doomed to become a victim of cybercrime the moment a general surge in demand hits the site.

Along that same vane, WordPress has its own set of WAF plugins. These take a similar approach by inspecting connections to detect malicious patterns and stop it immediately, before further data from the request is passed onto the remainder of your WordPress environment. You can easily find and implement any number of application firewall plugin via the WordPress plugin marketplace. The following link will take you to the market page listing for firewall solutions. Just remember, you only need one.

WordPress Application Firewall Plugins

DDoS Protection & Mitigation

You won’t make it far in the hosting world with a successful WordPress site, especially an enterprise site, without some form of edge-network protection.

Stopping the multitudes of requests that make up a DDoS attack can only be mitigated at the network level. Long before the traffic has a chance to damage or cripple the server. This is as true for WordPress as it is for every website. The more familiar site code is to the general public, the more unique attack vectors are visible as well. The upside is the list of known attack vectors stays trimmed by running only the most recent stable versions of your software. This is not limited to WordPress, but affects all the underlying pieces of the software stack.

From the Operating System, to the web servers, to the CMS platform, right down to its plugins and themes. All of these are potential problem zones inherent in your WordPress backed websites. Avoid letting these components get behind from their most recent versions to protect your WordPress site.

Intrusion Detection & Prevention

There are many security products available on the web. Some are more well rounded than others, but they all serve their functions well. You should invest time and consideration in finding a means of protecting yourself from would be intruders. The reality of WordPress being a titan in the industry means every would-be hacker knows about its pain points. So make sure you have your login interfaces locked down with brute force protection. This can be done a number of ways both within and outside of WordPress.

I personally find the options to both rename the WP login page to a unique name as well as using an edge-level Access Control List (ACL) to limit login and other sensitive URLs to a specific list of authorized addresses. These solutions stop unauthorized traffic from being able to attempt penetration tests, but are less feasible for sites that require or allow user-base logins.

A newer approach are full-on Intrusion Detection Systems (IDS) services through third-parties. These tools use a monitored and managed event detection with human-beings responding to and investigating events to sift out false positives. These kinds of services are make sure that any suspicious behavior gets a set of human eye-balls vetting them and not just some trickable programmatic logic.

There are many solutions to this sort of problem within WordPress as well. The below button will take you to the intrusion related plugins on the WordPress marketplace. Use it to shop around for a product that catches your eyes.

WordPress Intrusion Detection Plugins

Conclusion

Thanks for taking this journey with us through the scope of understanding how to scale your WordPress website. I hope this publication helps demonstrates SirsteveHQ’s expertise with scalable WordPress hosting. Regardless if you are are open to switching web hosting companies or not, this article should help you identify at least a few areas to explore to ensure you don’t experience any downtime.