Michael Feathers believes many believe a flawed theory of unit testing. This theory is that unit testing improves quality by catching errors. This seems to make sense but is does not explain the real value in unit testing.

To write effective tests, you need to make sure each unit of code only does one thing, and it does it clearly without unexpected side effects. This gives you a chance to reflect on the intentions of the code. Very often, while trying to understand how to write a goode test, you can discover a way to improve the code. Almost every time I write a test I find a way to improve the code while I am at it.

Catching bugs is an added value of unit testing. The real value comes from thinking and reflecting on your code.

Here is some code I borrowed to make emacs work nicely with scheme.

First I installed some stuff
apt-get install mzscheme
apt-get install quack-el

Then I added this to .emacs

;; Start up Scheme

(add-to-list 'load-path "/usr/share/emacs/site-lisp/quack-el")

(setq scheme-program "mzscheme")

(global-set-key [(f5)]

		'(lambda ()

		   (interactive)

		   (require 'quack)

		   (run-scheme scheme-program)))

The code fragment borrowed from http://jeffcjensen.net/scheme/_emacs

That link has quite a bit for scheme customization for emacs but I am not sure I will need it that much yet.

I picked mzscheme because I can apt-get install it and the .emacs I borrowed from seems to have some support for it.

Paul Graham once again documents a law of programming that most people understood but never could define.

To be productive a programmer needs to have the program in their head. This is why distractions of any length, require a 30 minute reload before a progrmamer can be productive again.

This is not exactly 100% true, but programmers like to exaggerate like it was. It is often true, and its certainly true a mind numbing meeting will cause a distruption for sure. Of course, email, news feeds, and looking out the window can do that too if you are not disciplined.

There are ways to work around this. I always take notes if I have to stop someting. Suspend to disk or "sleep" mode on your computer is also a wonderful thing. Keep all the tabs in firefox open, emacs, etc and your code is there waiting for you when you get back. I usually type in some comments to get me started when I get back. I also write everything down in a notebook. This has two purposes. I can always read the actual notebook if I need two, but more importantly, writing someting down stores it in a memory location that is easier to retrieve.

Of course, with most modern systems, its impossible to have the entire system in your head. Mostly you'll have a high level view of the major subsystems, plus a more detailed view of the particular code you are working on. This happene to me recently when I needed to explore the security code for OpenACS. It works fine but there are subtle things that can affect the user experience. It took about a day to poke around in the code, read it, read it again, add some debugging statements, change someting, break it, fix it, notice the fix actually broke someting else, and fix it again. Eventually these activities can help build a model of the code in your head.

Another more systematic way to do this is to write tests. If you write tests as you design an build a system, it will help you get a better, clearer picture of what you are building while you are builiding it. Of course, nothing is better than a test to gain understanding of a bug or unfamiliar code.

I am reading iWoz and he discusses what he thinks it means to be an engineer.

What I am talking about is what it means to be an engineer's engineer. A serious engineer. I so clearly remember him telling me hat engineering was the highest level of importance you could reach in the world, that someone could make electrical evices that do someting good for people takes society to a new level. He told me that as an engineer, you can change your world and change the was of life for lots and lots of people.

Paul Graham's The Roots of Lisp should be required reading for anyone attempting to understand programming in a functional language, or anyone interested in the amazing, simple ideas that make programming languages powerful.

I am quite surpised I have never seen a link to this article. I must have missed it. It really explains the power and the simplicity of the math behind Lisp. Basically the entire language of Lisp is written in itself using a few primitives and a few more functions.

I found this article poking around Paul Graham's site looking for the online version of On Lisp, which I thought I would read to expand my learning. I think this short paper gave me enough to think about for a while.

Ramon Leon comments on a post called Smalltalk - Language of the gods by saying this
"Languages like Smalltalk and Lisp (always got to give Lisp its props) are amplifiers. They make you more of whatever it is that you already are. If you're a bad programmer, they're going to make you worse, they'll let you shoot your foot off in more interesting ways than you can imagine."

This ties into the maze of twisty abstractions idea. It is definitely the case that some of the historical design issues of ACS 4 (the predecessor to OpenACS 4 and 5) were due to a dangerous amount of a little knowledge. Powerful tools can be used effectively or ineffectively, just like any tools. I guess the more power, the more effective or ineffectively they can be used.

I just wanted to share my favortie phrase to describe over-engineered systems. The title says it all, of course. The term "architecture astronauts" is usually used to describe the folks who build such things. I think my phrase describes the feeling you get when you get lost in that kind of code.

TODO: Find some real-world code to talk about?

A few links to Behavior Driven Development, which focues on specifying the behavior of code, in a precise language, and taking Test-Driven Development to the next level.

Blog Post http://blog.daveastels.com/articles/2005/07/05/a-new-look-at-test-driven-development
( related PDF http://blog.daveastels.com/files/BDD_Intro.pdf ) and a video from Google http://video.google.com/videoplay?docid=8135690990081075324

and the Behavior Driven Development Wiki http://behaviour-driven.org/

Functional Programming for the Rest of Us is an very clear exlanation of what functional programming is, and how it is different from imperative programming.

This arcticle is the first time I have understood what a function is. That's pretty important in doing functional programming. Without understanding the nature of functions (as defined in functional languages) you will never "get it". Now I have it.

The basic idea is this, a function always returns the same return value for the same arguments. It can't do anything else, or its not a function. Simple, and elegant. The article goes on to explain how you can possibly accomplish anything built on this simple foundation. Its good. Go read it.

I remember now that the Berkeley computer science lectures I linked to also explain a function in this way. That is, its always returns the same thing when given the same arguments with no side effects. This was buried inside the rest of a lecture, and I did not understand it in the whole context of what was going on. I am glad I found this article. I found it in a thread asking about the explanation of continuations on a Squeak mailing list.

I was happy to see, the other day, that my setup of a testing framework for a new set of features was very effective. I saved time, and gained understanding of what the code was doing, and thnat made it much easier to get the code to do what it was supposed to do, always the goal of testing!

I really need to write down my process of writing tests during development of a new API to share with the rest of the OpenACS community. Like writing the tests themselves, I am sure explaining the process will lead to better understanding of the process, and most likely show room for improvement.

Sometimes, the itention of code is so clear, you don't even need a test, just reading it, you can intuitively understand weakness in the API and work around it. That probably doesn't justify writing a test, because its much easier to reproduce conditions with an automated test than by hand and I haven't met a test I have only run once yet.

Of course, really using the code to get a job done is the best way to learn about it. Writing a test is a quick way to give your code a job to do.

My good friend and fellow OpenACS hacker Vinod pointed me to a set of videos/podcasts of a recent course following Structure and Interpretation of Computer Programs.

Signifigantly, they approach the material in a little different way, and order than the classic lectures available online http://www.swiss.ai.mit.edu/classes/6.001/abelson-sussman-lectures/

It seems like the lectures given by the authors in 1986 follow the development of the book itself pretty closely. I was getting stuck on one part of the book, and not surprisingly they skip that part and come back to it, in the Berkeley lectures.

The push up first-order functions much earlier in the course, and I was able to "get" it right away. I am sure there is much more to understand, and I will keep listening and learning.

I had a conversation the other day when another developer mentioned using a REST interface instead of HTTP POST. Despite reading quite a bit about REST, the idea had not sunk in. My colleague explained that the basic idea is that all operations on a "resource" use the same URL. So if you have a file/data structure/whatever you PUT to create it, GET to retreive it, at the same URL. Where, with a POST, in general, the URL stays the same, and the paramters, change to refer to the resource. GET would usually occur at yet another URL, with the same parameters. I'll have to keep thinking about it.

I found an amazingly, simple, and greate explanation of the limits of automatic, scaffolding code based on models. MVC in Ruby on Rails explains that the scaffolding creates the most basic CRUD controller based on a single model. This is usually not quite what you need, and you end up gluing together several models into one controller that gives your users exactly the interface they need.

I am certainly not singling out any one toolkit, I think that most descriptions of MVC describe the single model per controller example, and it really doesn't show you how to really use MVC.

David Heinemeier Hansson has said before that scaffolding is fun, but you will need to customize whatever automatically generated code comes out of it.

I am reading Iterating Infusion, by Greg Anthony. It tries to show a clearer way of design object-oriented systems. It's full of theory and throws in a new way to describe object-oriented design. I'll skip over all that for now. I am in the section on designing a system now, and there is one bit that is sticking out as a read it.

He says designing a toolkit, where the outcome is uncertain, that is, you don't know what software will be built with it, is risky. He says done well, a toolkit is more flexible is building multiple system, but done poorly makes it diffifult to build anything.

This makes alot of sense, and is pretty obvious. I hope he has advice on how to design a toolkit well. I am feeling OpenACS suffers from the problem of poor toolkit design. It sometimes gets in the way of what you want to do with it. A good toolkit point the way to your solution when you know what result you want.

Comparing SICP to HTDP we find one book focused on learning how to solve problems, and think in new ways, while the other focuses on building computer software systems.

SICP is for training computer scientists, while HTDP is for training good programmers.

So if you just want to implmenet new versions of the software we already have, in a new lanaguage HTDP will help you produce better, easier to maintain code. If you want to change the world, and create something new, you need SICP.

I found a post on Lambda, The Ultimate that links to a few articles discussing what should go into a first programming course, and how it should be taught. I originally looked at SICP and I think it teaches some really interesting fundamentals of how programs work. This critique by Felleisen, et al,[pdf] auhtor's of How to Design Programs advocates a different approach. The critique says that SICP shows how to do the main parts of programming, but does not teach when or why to choose a certain technique. It seems that SICP expects 1) the instructor to add this value, or 2) the student will learn this by applying the priciples to example programs, and be able to transfer this to their future programming projects.

I have just started SICP and I agree with this critique. Probably most of all what I need to learn is when to apply a certain technique and how to anaylyze a program structure to know what techniques are being used and how to improve it.

Another critique of SICP is the specific domain knowledge required to understand the programs, especially mathematical concepts. One of the first examples, and exercises focuses on testing is a number is prime. The forumla and algorithm for this is introduced, but it seems it is assumed the reader understands this from prior math courses. This can make the learning frustrating as you end up focusing on understanding the math instead of the program.

It probably is important for a programmer to understand this math, but its hard to aquire it in this context. The critique shows a table of the concepts taught and the examples used to teach them comparing SICP to HTDP and its striking how the examples in SICP are of interest only to the most dedicated computer scientist who wishes to understand the deepest concepts of the art, where HTDP examples seem much more fun, and approachable.

I still will stick with SICP, but I am also going to look at HTDP in parallel. I am intersted in the funadmentals of programming langauges and how they are built, even if it isn't directly applicable to my programming work. I also am interested in learning about learning, and the different ways the programming fundamentals can be learned.

Here are some references toward the concepts I am interested in learning.

• aDUni - ArsDigita University program
• ACM guidelines for computer education, in particular the 2005 Final Report. This includes several different types of computer programs. I am focused on a combination of Computer Science and Software Engineering based on their definitions.
• MFA in Software Master of Fine Arts in Software, a new way of looking at software as a creative endevour.

I have decided that even though I don't have time to formally pursue a degree in Computer Science, I am going to attempt to learn the content of a Computer science/Software Engineering program on my own.

I am using this weblog category to document this process in case I ever decide to take what I have learned and apply it toward a degree. There are schools like Empire State College that grant credit for documented learning. Documenting the learning process should also help me reflect and learn what I have learned.

I feel that I have quite a bit of computer science type knowledge already through my open source volunteer programming, and consulting projects. The idea is to figure out what parts I have learned and where I can learn more.

John Hiler at Microcontent News gives a great overview of the evolution of hierarchy vs. search for finding information. I have been thinking about this since I read The Inmates are Running the Asylum.

Well, really I have been thinking about it since I have had to organize large amounts of email. When I started filing things in folders, it was a pain to find something. Once I saw gmail, I adapted that model to my IMAP email with mutt. Now I have everything in my inbox, over 3000 items now, and search. It is still working fine. Maybe its not the best solution but it works for me.

Adapting these ideas to content management, organziation, and findability of web site data, I have been using search, combined with categorization, and other metadata filtering of search results. I think this is a good way to go, but the trick is figuring out what metadata is useful, and how to present it in the user interface.

I realized the main difference in the approch of Interaction Design and Agile Development. ID assumes the cost to change software once it is written in very high. Agile processes are defined precisely to reduce the cost of change, and that is how it can work closely with direct customers, and do more, and smaller designs. Basically it takes the 1 year design, 3 months coding schedule of the ideal interaction design project, and shortens it to an extreme. The other big difference is that Agile allows coding during design. Agile also says that each interation must be able to stand along as functional software. With such short design and programming phases, it is possible to do a good job, just on a much smaller pice of software.

I still haven't figured out how Agile can support integration of a huge application from all the pieces. It seems risky to design one feature in the first iteration, and expect somehow that the interaction for the users will integrate with the whole application, and not change dramatically. I think changing interfaces is a huge training and probably morale problem, even if the changes are improvements. So this is a place where I think ID really has the advantage. I know there is still more to learn.

I was reading and had an idea about user entered tags, and other user-derived metadata. I am referring to data such as "users who viewed this object also looked at these objects" or "users who purchased this also purchased that" sort of information. Or "user X says this list of items go together". It is another way to tagging, I supposed, figuring out what items are related by passive means (at least with the first two examples.) Basically just poking around in your database for interesting correlations. Amazon.com is obviously the leader in this space. I can imagine a system that can take advantage of freeform text tags, and user-derived metadata would be very powerful. The focus would be on intelligent analysis of the data. Or maybe not. Perhaps just making the data available is enough to inpsire creative use of it.

I am reading Alan Cooper's "The Inmates are Running the Asylum" and it tries to build a business case for interaction design. Interaction Design seems to be deciding how the program will behave, and actually what it will do. Too often this is really not decided except during programming. Some people think it is realted to user stories and other techniques used in various agile programming methods. I agree, but Alan Cooper himself does not.

One bit that is short and really is important is

Will the user understand it? Can we present this information in a way that makes sense? Is the sequence of instructions appropriate for what the user wants? What information does the user need most?

These questions are incredibly important and too often the are not considered fully before deciding what software to build. Cooper says all interaction design and all decisions must be made before one line of code is written. I think the interaction design ideas can be applied to shorter iterations and smaller pieces of software, but prioritization and deciding what is most important is even more critical in that case.

Exploring the Mnemonic user interface(pdf) looks into the future of computer user interfaces.

I found an interesting article describing StarLogo: "StarLogo is a fun (and free!) present to give your kids. The true gift is one of learning and sharing the learning experience with them." Wow. StarLogo sounds like a great way to teach programming to kids. StarLogo is free to download.

In addition I just learned about Computer Science Logo Style, a three volume set of computer science fundamentals using Logo as the foundation.

The StarLogo folks also have a new book called Adventures in Modeling.

Thanks to Lambda The Ultimate.

There is also TinyLogo for PalmOS.

A program to offer a Master of Fine Arts in Software is being developed at the University of Illinois. This is a great idea, studying the craft of software by examining great works of software and doing research through your own creation of software. Here are some of the ways it would work:

"...

• You read the work of masters, take them apart to see what makes them tick.
• You do a lot of work of your own, which is critically discussed and frequently revised.
• You write and teach about the craft.
• It's for people who already have experience.

..."