I have been thinking a while about writing this story, but the busy life of a CTO in a bootstrapped startup hasn’t given me pause. Now, in a long haul flight to San Francisco, it seems the right time to do it! 🙂
How did it begin?
A few months ago, I spoke to Coen, CTO at Radio + TV North-Holland (RTV NH). Being one of the most popular public broadcast organizations in the Netherlands, their website serves endless amounts of content. However, they had a problem in doing so.
They were running their portal from in-house servers connected to internet via an expensive uplink. These servers, limited in number and resources, were not able to cope with large spikes of traffic to their site and additionally they were having problems to renew the uplink with the telecom provider. They needed to find a solution and they needed it fast, as the uplink lease was ending in less than two months.
Cloud seemed the obvious answer and he reached us for a proposal.
And, here you have a situation where Cloudways’ skills, flexibility and nimbleness perfectly fitted. RTV NH setup is not terribly standard. They run out of a not-so well-known CMS and have a huge shared dataset between web servers. Auto-scaling was an obvious answer to their needs, but there would be some challenges ahead.
How did we solve it?
After an initial analysis, we proposed a cloud-based solution on Amazon Web Services with a load balancing, auto-scaling farm, a redundant caching layer with Varnish and Memcached, and a multi-AZ MySQL RDS backend. As shared storage for the web farm, we decided to initially try with an S3 file system.
So, we got all this up and running quickly and deployed RTV NH site on it. Alas, the initial tests didn’t succeed. The CMS (which lacked any S3 plugin) didn’t play well with S3. It was simply too slow and unreliable.
So, we moved to Plan B, that implied setting up an NFS server with shared web farm data. Again, given the volume of data involved, the performance was not optimal. Consequently, we went back to the drawing board.
Finally, given the time constraints and the fact that the CMS provider was already working on S3 shared storage support, we decided to implement a fix and it worked wonders. All editors were told to update the content on one specific web server in the farm. (Let’s name it “Master”.) We keep all data that has to be shared between web farm servers in a separate EBS volume.
Then, when a new auto-scaled server is started, this is what happens:
- We take a snapshot of the shared EBS in Master.
- We create a volume from it and mount it to the auto-scaled server.
- We periodically run a sync from slave to master (via rsync and for the shared EBS).
This has worked fantastically as a fix while the CMS provider rolls S3 support and could be done within the limited time we had to experiment.
I wanted to highlight this project because it perfectly summarizes our open-minded approach to customers’ problems and the close engagement with them and their providers to find solutions to their problems. Through all this process, we worked hand-in-hand with RTV NH developers and people from the CMS company to come up with a solution.
So, did we impress RTV NH?
Well, let the last words from this experience be from Coen and not from me:
“Due to the extreme differences in concurrent visitors on our site, the auto-scaling setup on the Amazon cloud provides us with a flexible setup and a cost efficient solution. This was unachieveable on the dedicated solution we had before.
“It’s good to know there is a specialist team working in the background and we do not have to worry about the setup.”
There you go! We are happy to serve RTV NH and we are waiting for you to join. Give us a chance and start a free trial from the banner below.
Pere Hospital
Pere Hospital (CISSP & OSCP) is the CTO and co-founder of Cloudways Ltd. He has over two decades of experience in IT Security, Risk Analysis and Virtualization Technologies. You can follow Pere on Twitter at @phospital and read his blog at www.perehospital.cat
