Golang - The database/sql package

This post is part of a series where I do my best to organize my thoughts around Go: its paradigms and usability as a programming language. I write this as a Java programmer that respects the principles of Elegant Objects.

I am studying the Go Code Review mantra Accept Interfaces, Return Structs and was inspired to write this post after coming across Eli Bendersky’s post Design patterns in Go’s database/sql package. This is the first instance where I feel I can endorse the mantra with confidence. Eli does a good job analysing the architecture of database/sql - I’m just here to provide a little nuance and some of my own notes.

Problem statement

Application programmers need a Database abstraction layer over a variety of SQL or SQL-like datasources for the most common use cases.

Designing DALs is hard for two primary reasons:

• The large variety of database implementations and drivers
• The large variety of common use cases
  • CRUD
  • Transactions
  • Connection pooling
  • Prepared Statements
  • Mapping of data types
  • Stored procedures and functions

Any solution is bound to end up as fat APIs ala sql.DB or JDBC.

database/sql.DB

Interestingly, sql.DB is a concrete type, not an interface. Why?

sql.DB exposes a fat interface for the reasons laid out in Problem statement. There are several strategies to limit an API’s obesity¹ - both database/sql and JDBC opt for the Interface Segregation Principle. database/sql muddles the waters a bit by doing More Than One Thing, while JDBC offers a cleaner separation of concerns.

Because sql.DB is necessarily fat^2,3, making it an interface will only hinder code that depends on it: it’s painful and wasteful to have to implement all those methods in your production code and in your mocks when you only need 3 or 4. For these reasons, programmers in general prefer to design facades or adapters and place them in front⁴ of fat APIs. Both in Java and in Go this extra component can be either a concrete type or an abstract type.

YAGNI: whether you use a concrete or an abstract type depends on whether you’ll actually need the extra level of abstraction.

Decoupling the user interface from driver interface

Regarding why database/sql split the user interface sql.DB from driver.Driver, Eli notes:

1. Adding user-facing capabilities is difficult because they may require adding methods to the interface. This breaks all the interface implementations and requires multiple standalone projects to update their code.
2. Encapsulating functionality that is common to all database backends is difficult, because there is no natural place to add it if the user interacts directly with the DB interface. It has to be implemented separately for each backend, which is wasteful and logistically complicated.
3. If backends want to add optional capabilities, this is challenging with a single interface without resorting to type-casts for specific backends.

These points are true, but I think the overarching theme behind this design decision is simplicity and ease of use. Proper interface segregation and separation of concerns⁵ took the back seat and it all led to a mix of several orthogonal requirements in a single interface:

• Execute queries
• Connection pooling
• Thread safety

They decided to implement Connection Pooling and Thread Safety themselves while drivers need only provide connections (and statements, and everything else derived from connections).

The upside of this clear violation of SRP⁶ is a simpler learning curve for the user (they are exposed to a single, simple interface) and a simpler driver interface for vendors to implement.

The downside is that the maintainers are burdened with the maintenance of code that doesn’t necessarily meet all user’s needs and may be stifling innovation in this area for Golang⁷.

Conclusion

database/sql eases the learning curve for users by providing a simple API that nevertheless breaks the single responsibility principle and incorporates orthogonal yet useful functionality into this package, potentially discouraging innovation.

sql.DB is presented best as a concrete type and not an interface because its requirements necessarily inflate it into a fat API, greatly diminishing any returns an interface has in a structurally-typed language like Go.

---

¹ aka “surface area”, but hey - since we’re talking about “Fat APIs” we might as well run with it :)

² justifiably breaking the Go Proverb The bigger the interface, the weaker the abstraction

³ see section 2.9 of Elegant Objects vol 1 Keep interfaces short; use smarts

⁴ aka “wrap”, but I dislike the term because its meaning has been diluted and may refer to any one of several distinct patterns

⁵ see design parameters for database/sqlhere

⁶ Single Responsibility Principle

⁷ consider the wide variety of database-connection-pooling libraries in the Java ecosystem and how they each emphasize different aspects like ease of use, performance, features, etc.

⁸ see “slow builds” and “uncontrolled dependencies” in section 4 Pain Points of Rob Pike’s Go at Google: Language Design in the Service of Software Engineering