Review Signal has become over the years the reference in finding unbiased figures of the hosting providers for decision-makers and developers in the vast WordPress - WooCommerce world. Response times for cached - static and PHP - dynamic requests, uptime, SSL certificate grading stay as solid proofs for prospective customers when navigating hosting services and making a decision based on hard evidence, not on affiliate-driven blog posts. Furthermore, the evolution of these specific numbers for a certain provider show the amount of care and interest the they are investing to improve their service quality.
The undeniable authority of Review Signal is confirmed by the social media coverage that the WordPress hosting performance benchmarks receive when released every year. Kevin Ohashi’s conclusions and feedback surface on the marketing materials that hosting providers are proud to display: landing pages, sales pitches and service presentation slides. We’ve been there too, while we’ve been working in the managed WordPress hosting biz, eagerly expecting the yearly testing results, to see how we stack up against competitors and hopefully include the positive results as high on our homepage as possible.
Review Signal has been using constantly over the last years K6 - an open source-based load testing platform that has evolved into an enterprise offering, making it unaffordable for an indie project. What can be so complex to run a load test for a PHP-based CMS, one could ask. Well, accurate testing that’s taking into consideration geographical distribution requires a consistent budget. Since there was no plan that would fit the yearly testing use-case at Grafana, the parent company of K6, the need of a custom solution has arised at Review Signal.
Here, at Bitpoke, after stepping aside from the managed WordPress hosting path, we’ve set course on developing automated solutions for running WordPress on Kubernetes and complement them with custom development. We were presented with the quite unique challenge to contribute to an open solution based on K6 that would leverage container orchestration to deploy, distribute, run, collect and display results for Review Signal’s WordPress and WooCommerce benchmarks.
Our initial take was to suggest Locust and make the most out of a solution brilliantly designed for load testing, which we’ve extensively used for making sure the infrastructure we’ve written for our previous managed hosting endeavour would hold and scale properly. Unfortunately, this was not a viable alternative, since previous test results depended on the K6 test-runs and backward compatibility of existing test scripts is important for the Review Signal following.
The next step was to try out the K6 Kubernetes operator readily available, generously contributed by Grafana. Unfortunately, this has not worked due to missing features such as a way to concurrently run tests from multiple geographic locations.
Now we are at the moment of writing a custom Django webapp along with a Golang Kubernetes controller - which will very soon be open-sourced on the Review Signal’s Github - that can manage test jobs across different Kubernetes clusters.
What we realised now while looking back at how we worked so far was to retreat in our “garage” and design, code and implement products, open source them and write about them only at a later time, when we consider them polished enough and production-grade - preferrably already including them in our live systems. But this time, we are thinking of reversing a bit the way - working AND writing, looking forward to collect feedback as we go and, of course, help others avoiding to reinvent the wheels (in this case of load testing WordPress with K6).
