Bill Karwin talks about Database Designing, Softwares and How to Work Around Them

by Ahmed Khan  December 15, 2015

With more than 20 years of experience under his arsenal, Bill karwin is one of the senior most guys to go to when you want to know about Databases. Bill Karwin is a Senior Database architect and has authored many different books related to SQL, including the SQL Antipatterns: Avoiding the Pitfalls of Database Programming. Before working as a Senior Database Architect, he was involved with multiple projects of software engineering and has also worked in zend for as a project manager.

Bill Karwin Interview

Bill Karwin, in his interview to Cloudways, talks about the different approaches of database designing and what affect database has on technology along with sharing his personal opinions with regards to different database softwares.

Cloudways: When did you switch from a role as a software engineer to a full time database developer? What attracted you toward databases rather than software?

Bill:  I’ve always used databases in my software engineer jobs, but if there was a specific time I became a specialist with databases, it was when I worked for InterBase. I realized that no matter what language, framework, or operating system you use, and no matter what vertical market you develop software for, the database is the foundation of every software application. I don’t see database work as separate from software development — rather, databases are central part of software.

Cloudways: What was the most difficult database architecture you have designed?

Bill: The most difficult assignment, which actually happens frequently, is to design a database architecture that is arbitrarily flexible. That is, design a schema that never needs to be altered as the project evolves. This happens when the project requirements are constantly in flux. My view is that this is an impractical constraint for any system (not just at the database layer). I was once asked by a client to produce an application so that it could be infinitely re-configurable for future requirements not yet known. I joked that this application already exists, it’s called “gcc.” But to use this application, we are required to write complex “configuration files” (they must be configuration files, because they have a “.c” extension).

Cloudways: What are the key points to keep in mind while designing the database?

Bill: First, the schema for the database has to support the most complex case, so you have to define requirements before you can design the database. Second, optimizing the database is done for the specific queries your applications run against the data, so you have to implement a lot of your application code before you can optimize. Third, it’s just as important to design the database operations including security, monitoring, failover, backup and restore, and archiving.

Cloudways: What was your key job as a Project Manager at Zend?

Bill: My key job was to release code and get the project to 1.0 as quickly as possible. That’s what I did, releasing 12 betas in 10 months. I made sure each developer had clear objectives, checked their progress, and gave them feedback. I did a lot of code development too, not only on the framework, but also project pieces that fell between the cracks, like test frameworks, integrating community contributions, developing release tools, and generating progress reports.

Cloudways: What was your first thought when you joined Santa Cruz Operation as a Software Engineer, since it was your first job?

Bill: I transitioned from an intern to a regular employee at the same company, so it wasn’t too different on a day-to-day basis. I felt like I was suddenly expected to know everything once I started as a regular employee, and I was too shy to ask for help or guidance. But junior engineers still need training and supervision, especially for working with a team and a big project. As you become more senior in your career, be sure to help the younger developers. It’s not a favor, it’s the job of senior developers to do that.

Cloudways: Do you think NoSQL can ever compete with SQL in terms of designing a database system or managing relational databases?

Bill: NoSQL already does compete with SQL. There are use cases that some NoSQL technologies solve elegantly, that SQL doesn’t do well. But the reverse is also true, SQL still has advantages for some tasks. It’s like comparing a hammer and a drill, and asking which is the better choice for carpentry? You might drive a nail by tapping it with a drill carefully, but it will be awkward and take too long. You can certainly smash a hole through a wall using a hammer, but not a clean hole. You need to use the right tool for each task, and that means you have to know how to use both tools well.

Cloudways: Is it easy to manage work and family simultaneously, since you are a Senior Database Architect as well at School Messenger? Architecting a database is a difficult task. Has there been a time when you have chosen your family over work?

Bill: I’ve worked at some demanding jobs that took 70 hours per week, but I try to avoid doing that for more than a couple of weeks at a time. To be healthy and sustainable, no job can demand that level of work on an ongoing basis. I’m just as productive as someone who works long hours, and I do better work, when I’m not exhausted, and when I have been recharged by my personal life. I have declined jobs that said from the start that they work crazy hours.

Cloudways: What is your advice to beginners and students to build a career as Database Developer? How do you motivate them?

Bill: Anyone who wants a career as a database developer, or any other type of developer, must have a deep love and curiosity for their work, and for learning constantly. Don’t just take a class and finish the assignments. You have to learn more outside of classes by doing. Challenge yourself to explore new technology and methods. Try new things for the delight of seeing them work. Then share your knowledge. Your employability comes from a combination of hard work, competence, confidence, and enthusiasm.

Cloudways: Have you ever come to create a Distributed Database Design? What are the difficulties that occur in them and how can we resolve them easily?

Bill: The primary challenge with distributed databases is keeping each part in sync with the others, while also maintaining high performance. I don’t think there’s a single solution to resolve that difficulty “easily.” There are many techniques, but they all sacrifice a little bit of flexibility in different ways. It’s up to you to know the various tricks, and to make a judgement call in each project for the thing you can sacrifice with minimal impact. Anyone who says differently is selling something.

Cloudways: In your view, what’s the most exciting thing going on now in the world of MySQL?

Bill: The recent release of MySQL 5.7 is much anticipated. The performance of MySQL has improved with many concurrent clients. Interesting features such as JSON support, generated columns, security enhancements, sys schema, multi-master replication, and more. Like all major releases, we should test carefully before using this for production.

Cloudways: How was your experience when you first attended MySQL conference by Percona as a speaker?

Bill: I’ve been speaking at the MySQL Conference for many years. One of my best experiences was doing a tutorial on SQL Antipatterns in 2008. The room was large and filled to standing room only. I realized I had found a great topic, so I developed the content into my book “SQL Antipatterns: Avoiding the Pitfalls of Database Programming” from Pragmatic Bookshelf. That book has been very popular.

Cloudways: Large databases tend to become a resource hog on the servers. Do you have any suggestions to large websites and enterprise level products to improve the performance of the database?

Bill: As databases grow, of course they require more resources. So don’t let them grow in an unbounded way. For example, separate high-traffic data such as persistent web sessions into a separate database instance. Establish policies for archiving or pruning old data. Also use tools like Percona Toolkit or New Relic to monitor which queries account for the most time. Re-measure this regularly. Most databases that seem slow are really just neglected.

Cloudways: Hosting large databases can be tricky. Do you think hosting them on cloud servers like AWS EC2 instances will give a boost to database performance?

Bill: Technology doesn’t give you performance. Architecture is what gives you performance. What I mean is that you might get 1.5x or 2x or 3x performance from switching to another platform, but those linear multipliers will never be enough to keep up with the growth in your data or the growth in your traffic. To gain orders of magnitude improvement, you need to change the way you use data, and that could mean major changes to your application. At Percona we observed that to support each 10x growth (approximately) in data volume or traffic volume, you should reassess your application architecture. But you probably can’t anticipate what you need to change until you can measure the bottlenecks that are created by the growth.
Cloudways: There are many MySQL forks available, like Percona, MariaDB and others. What are your suggestions for choosing one over the other?

Bill: Percona Server is a branch, not a fork. It maintains compatibility with MySQL Community Edition and integrates changes from every point-release of MySQL. When choosing one vendor over the other, one should consider the features, and see if those features are something you need for your project. Some of the enhancements are subtle and pretty specific to a specialized workload, so you may not see any benefit. Another factor is how well you like the company for support and other services.
Cloudways: People say that MariaDB is faster and more secure than MySQL. What are your thoughts on it?

Bill: MariaDB forked from MySQL 5.5, so they are bound to the internal architecture circa 2010. They have back-ported selected changes from MySQL 5.6, but as time goes on, they will diverge from mainstream MySQL, and eventually they won’t be able to take advantage of newer code changes. They have developed some interesting experimental changes specifically for MariaDB, but with a smaller development team they can’t help but fall behind.
Cloudways: How would you compare MongoDB, Firebase databases with MySQL? What do you think mobile app developers should use as their database storage?

Bill: MongoDB still has more maturing to do before I’ll be confident with it. Its lack of true durability is a deal-breaker for me. I’m also skeptical about schemaless databases. Your application still needs the data to be well-formed, even if the database doesn’t enforce a schema. But then the schema enforcement must be coded by you, the developer, in your application code. How does this make less work for developers?

I haven’t worked with database-as-a-service platforms like Firebase or Parse, but I have misgivings about platforms where you don’t own your data. Also, they are both non-relational JSON stores, which means you have no schema or data integrity. This is fine for some applications, but it’s not general-purpose.

Start Creating Web Apps on Managed Cloud Servers Now!

Easy Web App Deployment for Agencies, Developers and E-Commerce Industry

About Ahmed Khan

Ahmed was a PHP community expert at Cloudways - A Managed PHP Hosting Cloud Platform. He is a software engineer with extensive knowledge in PHP and SEO. He loves watching Game of Thrones is his free time. Follow Ahmed on Twitter to stay updated with his works. You can email him at ahmed.khan@cloudways.com

Stay Connected:

You Might Also Like...