Big Ball of Mud, Refactoring and Bottom-Up Programming

Black-box Abstraction

Abstraction

Procedures as Black-Box Abstractions from _Structure and Interpretation of Computer Programs (SICP) by Abelson and Sussman.

The importance of this decomposition strategy is not simply that one is dividing the program into parts. After all, we could take any large program and divide it into parts--the first ten lines, the next ten lines, the next ten lines, and so on. Rather, it is crucial that each procedure accomplishes an identifiable task that can be used as a module in defining other procedures.

a procedure definition should be able to suppress detail.

Refactoring

Big ball of mud (Foote and Yoder)

Refactoring

Big ball of mud (Foote and Yoder)

Cost: Writing-off a system can be traumatic, both to those who have worked on it, and to those who have paid for it. Software is often treated as an asset by accountants, and can be an expensive asset at that. Rewriting a system, of course, does not discard its conceptual design, or its staff's experience. If it is truly the case that the value of these assets is in the design experience they embody, then accounting practices must recognize this.

Refactoring

Big ball of mud (Foote and Yoder)

When designers are faced with a choice between building something elegant from the ground up, or undermining the architecture of the existing system to quickly address a problem, architecture usually loses. Indeed, this is a natural phase in a system's evolution. This might be thought of as messy kitchen phase, during which pieces of the system are scattered across the counter, awaiting an eventual cleanup. The danger is that the clean up is never done. With real kitchens, the board of health will eventually intervene. With software, alas, there is seldom any corresponding agency to police such squalor.

Refactoring

Big ball of mud (Foote and Yoder)

an examination of software development costs showed that problems were many times more expensive to fix during maintenance than during design. Surely it was best to mobilize resources and talent up-front, so as to avoid maintenance expenses down the road. It's surely wiser to route the plumbing correctly now, before the walls are up, than to tear holes in them later. Measure twice, cut once.

Refactoring

Big ball of mud (Foote and Yoder)

One of the reasons that the waterfall approach was able to flourish a generation ago was that computers and business requirements changed at a more leisurely pace. Hardware was very expensive, often dwarfing the salaries of the programmers hired to tend it. User interfaces were primitive by today's standards. You could have any user interface you wanted, as long as it was an alphanumeric "green screen". Another reason for the popularity of the waterfall approach was that it exhibited a comfortable similarity to practices in more mature engineering and manufacturing disciplines.

Refactoring

Big ball of mud (Foote and Yoder)

Today's designers are confronted with a broad onslaught of changing requirements. It arises in part from the rapid growth of technology itself, and partially from rapid changes in the business climate (some of which is driven by technology). Customers are used to more sophisticated software these days, and demand more choice and flexibility. Products that were once built from the ground up by in-house programmers must now be integrated with third-party code and applications. User interfaces are complex, both externally and internally. Indeed, we often dedicate an entire tier of our system to their care and feeding. Change threatens to outpace our ability to cope with it.

Refactoring

Big ball of mud (Foote and Yoder)

The incentives that drive the evolution of such systems can, at times, operate perversely. Just as it is easier to be verbose than concise, it is easier to build complex systems than it is to build simple ones. Skilled programmers may be able to create complexity more quickly than their peers, and more quickly than they can document and explain it. Like an army outrunning its logistics train, complexity increases until it reaches the point where such programmers can no longer reliably cope with it.

Refactoring

Big ball of mud (Foote and Yoder)

This is akin to a phenonmenon dubbed the PeterPrinciple of Programming by authors on the Wiki-Wiki web. Complexity increases rapidly until the it reaches a level of complexity just beyond that with which programmers can comfortably cope. At this point, complexity and our abilities to contain it reach an uneasy equilibrium. The blitzkrieg bogs down into a siege. We built the most complicated system that can possible work.

Refactoring

Big ball of mud (Foote and Yoder)

When it comes to software architecture, form follows function. Here we mean "follows" not in the traditional sense of dictating function. Instead, we mean that the distinct identities of the system's architectural elements often don't start to emerge until after the code is working.

Refactoring

Spaghetti Code

Variable and function names might be uninformative, or even misleading. Functions themselves may make extensive use of globalvariables, as well as long lists of poorly defined parameters. The function themselves are lengthy and convoluted, and perform several unrelated tasks. Code is duplicated. The flow of control is hard to understand, and difficult to follow. The programmer's intent is next to impossible to discern. The code is simply unreadable, and borders on indecipherable. The code exhibits the unmistakable signs of patch after patch at the hands of multiple maintainers, each of whom barely understood the consequences of what he or she was doing. Did we mention documentation? What documentation?

Refactoring

Spaghetti Code

BIG BALL OF MUD might be thought of as an anti-pattern, since our intention is to show how passivity in the face of forces that undermine architecture can lead to a quagmire. However, its undeniable popularity leads to the inexorable conclusion that it is a pattern in its own right. It is certainly a pervasive, recurring solution to the problem of producing a working system in the context of software development. It would seem to be the path of least resistance when one confronts the sorts of forces discussed above. Only by understanding the logic of its appeal can we channel or counteract the forces that lead to a BIG BALL OF MUD.

Refactoring

Spaghetti Code

Frequently, the organization of the system reflects the sprawl and history of the organization that built it (as per CONWAY'S LAW) and the compromises that were made along the way.

Refactoring

Spaghetti Code

At first glance, a BIG BALL OF MUD can inspire terror and despair in the hearts of those who would try to tame it. The first step on the road to architectural integrity can be to identify the disordered parts of the system, and isolate them from the rest of it. Once the problem areas are identified and hemmed in, they can be gentrified using a divide and conquer strategy.

Refactoring

Keep it working! (Foote and Yoder)

One always has, at every stage, in the process, a working system. I find that teams can grow much more complex entities in four months than they can build. – From "No Silver Bullet"

Microsoft mandates that a DAILY BUILD of each product be performed at the end of each working day. Nortel adheres to the slightly less demanding requirement that a working build be generated at the end of each week. Indeed, this approach, and keeping the last working version around, are nearly universal practices among successful maintenance programmers.

Refactoring

Keep it working! (Foote and Yoder)

Another vital factor in ensuring a system's continued vitality is a commitment to rigorous testing. It's hard to keep a system working if you don't have a way of making sure it works. Testing is one of pillars of Extreme Programming. XP practices call for the development of unit tests before a single line of code is written.

Refactoring

Keep it working! (Foote and Yoder)

Always beginning with a working system helps to encourage PIECEMEAL GROWTH. Refactoring is the primary means by which programmers maintain order from inside the systems in which they are working. The goal of refactoring is to leave a system working as well after a refactoring as it was before the refactoring. Aggressive unit and integration testing can help to guarantee that this goal is met.

The Selfish Class (Foote & Yoder)

A design, and the objects that embody it, survive and evolve only if they are used. To be reused, they must attract programmers. To do this, they must make a good first impression. ...an initial, painless presentation of impressive functionality can enhance an object's appeal. Objects which do this can flourish, and those which cannot, even if they are technically superior in other ways, may not.

A particularly powerful force in the realm of reuse is Comprehensibility. If an artifact is easy to understand, programmers are more likely to use it than if it is inscrutable. Code that is easy to read is easier to modify. Comprehensibility is determined by the quality of the code itself, as well as any available examples and documentation.

The Selfish Class (Foote & Yoder)

A code-level artifact must be able to attract programmers in order to survive and flourish. The paper addresses the question of what an object might do to encourage programmers to (re-)use it, as opposed to using some other object, or building new ones.

Piecemeal Growth (Foote and Yoder)

If a programmer needs a thorough understanding of an object's internal workings, the programmer will often find it easier to write his or her own code.

In the Oregon Experiment Alexander noted:

Large-lump development is based on the idea of replacement. Piecemeal Growth is based on the idea of repair. ... Large-lump development is based on the fallacy that it is possible to build perfect buildings. Piecemeal growth is based on the healthier and more realistic view that mistakes are inevitable. ... Unless money is available for repairing these mistakes, every building, once built, is condemned to be, to some extent unworkable. ... Piecemeal growth is based on the assumption that adaptation between buildings and their users is necessarily a slow and continuous business which cannot, under any circumstances, be achieve in a single leap.

Alexander has noted that our mortgage and capital expenditure policies make large sums of money available up front, but do nothing to provide resources for maintenance, improvement, and evolution. In the software world, we deploy our most skilled, experienced people early in the lifecycle. Later on, maintenance is relegated to junior staff, when resources can be scarce. The so-called maintenance phase is the part of the lifecycle in which the price of the fiction of master planning is really paid.

incrementally address forces that encourage change and growth. Allow opportunities for growth to be exploited locally, as they occur. Refactor unrelentingly.

Questions? ^_^

Future Topics

Artifacts

This chapter is not finished. In fact it's barely been started. In fact, you shouldn't even be reading this slide.

Artifacts

Characterizing people as non-linear, first-order components in software development by Cockburn

Being good at communicating and looking around counter inconsistency, leading to the prediction that methodologies can make good use of low-precision artifacts whose gaps are covered by personal communication.

  1. Problem 1. The people on the projects were not interested in learning our system.
  2. Problem 2. They were successfully able to ignore us, and were still delivering software, anyway.