I have often thought whenever I meet a programmer who is obviously grossly incompetent that there needs to be some sort of certification board for software engineering that can help enforce some minimal skills to help weed out the riff-raff.

Obviously, this is completely impossible. The tools and community change so fast that individuals are barely able to stay on top; good luck getting a standards committee to completely revise every few months. Who would be on the committee? And what would they agree on to enforce? Regex? Recursion, OO, testability, and functional programming?  Proof of understanding in the half-dozen "mind expanding" languages? Not going to happen. Not while my Joomla-site maintaining friend and my Android 3D game authoring cohort are both lumped under the title "software developer". The field is too vast, the content too broad, and the jobs too diverse.

So, we are doomed to wallow. Or are we? What is the one thing we all have to do? Get hired.

I used to be a big sissy about interviewing for jobs. Especially interviews full of the nit-picky details I can never remember, yeah, I really hate those. I would moan and groan about how the interviewer could only answer those questions because they had to look them up the other day. "MySQL index implementations, what am I, a DBA?!", I would screech to anyone who would listen, "They must have misunderstood, I am a developer, when would I ever need to know that?!"

But.

Interview prep gets me to cram stuff into my head I would otherwise skim or avoid if just sitting at home programming for fun. I get a great refresher on all the stuff I am fully able to do at work, but maybe I haven't needed in a while. Like linked-lists. In four years of fairly diverse programming jobs, I have yet to need a linked list for anything, but man, when that day comes, I have linked-lists in a quick-draw holster. I used to say the same thing about trees, then last year, I had a problem that surely needed a tree, and hear you me, I had that data being in-order traversed before you could say: "tree". Would I have even remembered what trees are good for without my semi-annual refresher?

My wife is a commercial pilot, and every so often she is required to skim over all those little details about being a pilot, refreshing everything, double-checking her knowledge so she can keep her licenses, even if she hasn't needed a certain technique since the last test. Huh. Sounds like me, every so often, cramming for an interview.

We don't need certifications for being a programmer. We already have the tools in hand, and just need to learn to use them better. Interviews are our certifications.

This is a double-edged sword, because this means that, "what you sow, you will also reap". The responsibility to keep standards high falls on the interviewer. If you interview a candidate for tricky puzzle questions, or in-depth knowledge of C++ libraries, you will find someone who knows those things, and possibly ONLY knows those things. I have seen developers hired who couldn't handle VCS merging or shipping actual software, but were absolute geniuses when it came to programming language trivia and puzzle questions.

A few months ago, I read of a guy with no programming experience who did like 100 interviews in a few weeks, till he was a master interviewee. By the end, he was able to pick his choice of companies, and yet he couldn't ship software. He gamed a system intended to sort out good developers by asking questions that are ancillary to development, without actually having the candidate prove any real ability. And now that company has a developer who is great at gaming systems, but unskilled at development: I hope they have a good training program.

No offense intended to the man who did this, I really admire his tenacity and clever thinking, and I think he will do well regardless of the task. But it speaks volumes to the brokenness of the interview system for those companies that would hire a completely unskilled developer based on the correct answers to trivia questions about programming.

Parting question to consider for interviewers: if interviews are our certifications, what should a candidate be able to do to be "certified" to be able to do the job in question?

Lately, I have been deliberating on quality vs. speed. Throughout my career, I have worked with developers who basically consider quality to be a programmer's vice, an extravagance only for the navel-gazing idealists. I have also worked with developers who consider all time spent on quality to be of the utmost importance, and that programming for speed is for those lazy and slapdash realists. I recently read Yegge's post Done, and Get's Things Smart, and I think I agree with Yegge on this point, that it is possible to perform fast and with high quality, and like a wandering samurai, that is the path I seek.

There are many facets to this, of course, but I have been playing with this mental model. Specifically, my insight here is a new definition of quality: quality is future speed. In my mind, quality that is not for future speed is an extravagance, something programmers do to entertain themselves, and it is worse than cutting corners in the present, because it sacrifices current work for no future gain.

Lately, my personal projects have taken a certain theme: I want more practice tuning my programming to be as fast as possible both for today and for the future. And to be perfectly honest, usually there is a lot more "future" in most systems than just "today".

So, how have I been doing this? I set arbitrarily short deadlines for myself, so I am forced to practice dropping anything that is not directly related to speed. I then try to rotate projects every month, so I am forced to return to old work, to learn better where I made bad choices that cost me future speed.

Doing this with the right mindset is very humbling, and very insightful. Honestly, this may happen at work accidentally from time to time as well, but rarely at work am I able to carefully reflect on the mistakes of the past. At home, the pain I feel is my own, no one suffers at home other than me, and there is no one to blame but myself. If an architecture is brittle, or the test coverage is low, or the design is resistant to change, (all of which have happened) I am able to take those lessons to heart because I directly caused those problems.

I encourage anyone who, like me, seeks to be a more excellent developer, to give this a try: start a couple of side projects (and don't worry, it took me a year to get the ones I have), and set yourself some crazy tight "delivery dates". Heck, make real products and try to make some money off them, maybe you will be able to get some side-cash! But then, after you have "delivered", take some time to rest, maybe read a book or two, tinker with some other projects, and after a break, return to analyze your past work. Set yourself another strict deadline, try to make a second version, or add some more features. You will find some incredibly valuable lessons there.

December 1st, I made myself a New Year's resolution to completely create an Android game that I would not be ashamed to show people from (mostly) scratch before the end of December. What made it the most fun was that I started with practically no experience with the Android platform at all, and so I got to pick it up by the seat of my pants.

The biggest challenge was probably the last three days, where I was faced with some hard calls, when the deadline loomed, and my still mile-long list of wants had to be pruned down just about every few hours. I basically had a "buggy beta" by Wednesday morning, and all the problems remaining were ones I didn't know how to solve or hide, and while all the graphics where there, I could barely stand to look at them. I felt like I was spinning my wheels with no progress. My friend Dan tried to talk me down, pointing out with some clarity that there was no gun to my head, and I was doing this for myself, so I could stop any time. Amusingly enough, that was exactly what I needed to hear. I was doing this for myself, to better myself, to learn more discipline. I wanted to stretch my abilities to the limits. So, I pushed through. I found a book that showed me how to fix my problems, and I didn't stop until every bug I could find was fixed, and everything my poor family pointed out (I made them each play it like a dozen different times) was smoothed.

And last night, after about a hundred hours of total work this month, I slid under the wire with roughly two hours to spare. The game is not in the market YET, because at the end of it all, I concluded I would rather spend the few dozen more hours to convert it to OpenGL ES, and add in a few neat things I was unable to do using the Android canvas, but for a v1, I am immensely proud of the whole experience.

I can say, unequivocally; WHEW.

This challenge was a fantastic learning experience, and I had several great reminders of what my dad always told me growing up: just work smarter, son, not harder.

I can't get focused today. My mind is a torrent of activity completely unrelated to my task at hand, so I am writing this to get these thoughts out so I can get back to work.

I got a call from a recruiter the other day trying to get me to come interview at this job. Right now I am working in VB.NET and Actionscript. Until this job, I was always working in open-source languages: Perl, PHP, and some Java. This is my first real divergence from that pattern, and of course it is in two languages that are, to say the least, not very popular with the open source crowd. This gets me wondering, "what if when I decide to move on, all the fun jobs look down their noses at me for working in unpopular languages?".

To be fair, I still program quite a bit outside work in my favorite languages, and that is all well and good.

For some reason though, that recruiter really got under my skin. I keep asking myself questions like, "is spending time right now making a Java game for the Android a good choice long term?" or, "should I spend any more time learning Python with Django, or just get more invested in Javascript and node.js?".

These are not questions I want to be asking myself.

So, I sit down to think about it and get it out of my system. What kind of programmer do I want to be in 5 years? I know without a moment's hesitation.

I want to be the kind of programmer who is not tied to any one language or platform, but is instantly effective and rapidly excellent in every language and platform.

Right now, I am learning Common Lisp. Why? To learn better functional programming. Why do I play with Javascript? To be better at client-side web programming. PHP/Python? Server-side web programming. *.NET? Compiled, versioned programming. Android games? For fun! Why do I all this? Because the programmer I want to be is not constrained to his language/platform at all. The programmer I want to be programs at a level over the language. 

In ancient times, plenty of builders did their work without math, and I would guess that a good portion of them likely did it faster then their math-doing counterparts.

I am certain that when builders first started using math to prove that buildings would stand up safely, they were mocked by their math-less counterparts. I can hear them now, "Oh, Stephos, it takes you 5 times longer to prove your building will stand up, real men can just see that it will work". Even in modern times, many people build houses and small buildings that probably don't even need the math to really "prove" they work, but they perform the due diligence anyway. On the other hand, some people today still build houses, sheds, tree-houses, lofts, and fire pits without any drawings or math at at all, and why should they? They are responsible for the risk.

This is all fine, but, builders who shy away from math also don't build airliners, space shuttles, and skyscrapers. An aircraft designer who doesn't feel like using math to prove their aircraft will fly is handed an "Experimental" sticker and gravely patted on the back.

Today, programming is at the same place as both ancient and modern engineering: many projects can be done "by eye", and some of the most experienced programmers are able to build immense projects alone without any sort of automated testing at all. When you go to a forum and ask for help, you are just as likely to hear from a "space shuttle" programmer as you are to hear from a "tin shack" programmer. The "space shuttle" programmer would be horrified to hear of working without automated testing and rigorous review practices, whereas the "tin shack" programmer would be horrified to think of the time wasted to even learn the procedures to automatically prove his code works as it should. The "space shuttle" programmer might work on the same internal financial project for months or years, finally releasing with zero defects; the "tin shack" programmer might churn out several quick web sites a day. Neither project is better then the other, they both might be perfect for what their business needs.

I have heard several times from very good lone wolf programmers that testing just slows them down, and in some ways, it does appear to be a slowdown for them. Unfortunately, the cost to build even medium-sized projects with several workers without automated testing is only felt when it is too late, when the deadlines (and costs) are spiraling out of control.

The disciplined application of mathematics transformed and solidified the field of engineering into a field far more vast and capable of some of the most incredible constructions imaginable. Would we have smart-phones, GPS, modern cars, and airliners if engineering had just avoided mathematics? Just imagine what the field of programming could grow into if we continue to learn and grow our testing, designing, and writing of software.

I propose that automated testing is to programming what math was to ancient engineering.

The last three and a half years for me have been terrible. Why? Well, three and half years ago, I discovered the endless debate, Emacs vs. Vim. Being an efficiency nut, I could not fathom the horror of learning one, and by that action, not learning the other. What if the one I learned was not the "best" one? I could see that both were excellent and worth learning, but I wanted to learn both concurrently. Unfortunately, my body has a hard time learning two completely different keyboard systems at the same time. Heck, three and a half years ago, I was still learning to touch-type.

So, three years pass, and I am now no farther along.

Additionally, I read on an almost weekly basis, articles that explain how great Emacs and vim are, you just need to learn one, and learn it well. I wanted to stick with Emacs, for the built-in lisp, and several of my favorite programmers highly advocate it. But, I wanted to pick up vim, because it is obviously the far superior text editor, and the default on Linux, and probably will help me not get Emacs RSI but, perversely, it is not nearly as easy to extend as Emacs.

So, the debate raged in my head for three years, and I stupidly got better at neither. Then, the other day, it clicked. Vim is the greatest text editor of them all, with text objects, highly combine-able commands, and modal editing. Emacs is the greatest IDE of all, with an awesome programming language built-in and thousands of classes and libraries to build on. Vim is built in the tradition of a small, sharp UNIX tool; Emacs in the tradition of HAVING thousands of small sharp tools.

Finally, in a rush, it clicked into place with earth-shattering force: I can make Vim inside Emacs, with lisp, but I really could not remake Emacs inside Vim.

Suddenly, in one glorious moment, the world all made sense. Vim, the greatest text editor, should be a part of Emacs, the greatest IDE! My searching had been in vain, there was no way to compare them, they are totally different tools!

In my typical child-like innocence, I rolled up my sleeves, opened up an instance of Emacs, and started making Vim. A few hours later, Google pointed out the obvious: Emacs already has a vi mode built-in (viper). Better yet, with a simple file in the right place, I had vimpulse (the practically complete Vim mode) installed and ready to go. Suddenly, a huge weight lifted off my chest. I am free to use the best tool for the job, and, like Huck Finn, float off down the river, leaving silly feuds behind me. If my actions here seem unnatural, an abomination, an unholy union, I encourage you, leave behind the fan-boyism, pick up your tools, and come build something with men who see their tools for what they truly are: not a good hill to die on.

At work last month, I was assigned to fix some bugs in an asynchronous legacy code base. My supervisor gave me a month to see what I could do, and to see if I could fix a few tricky bugs. The code is divided into two sections, a client application and a web server. Both the client application and the server have their own persistence, and they communicate through a soap web service.

I wanted to try and write some sort of end-to-end tests around this, but I could not figure out how to get around that web service. I don't want my tests to rely on a server having to always be up, so I initially was planning on writing tests from the client database to the web service, where I would just stub some responses. I got about an hour into that plan, before I realized I would then have to do that same thing on the server, and that sort of testing will not really tell me what the code does. I needed to cover this whole process end-to-end.

I walked around for a bit, thinking. Then it hit me, both the server and client code are written in vb.net, and, except for the web service, work just like plain old highly-coupled code. Hmmm.

For the first pass, I wrote an interface to wrap the web service, then used dependency injection to inject a fake web service that (and here is the cool part) simply calls around the web service entirely to the server code directly and injects a different database connection string, so the server talks to a second database I have prepared.

Just like that, I can run, all on my machine, 99 percent of the code in a single end-to-end test. I can even setup several client databases, and rotate them out, to "swap" between several clients in the middle of the tests.

I added a few "integration" tests that do talk to a real published testing web service, just to make sure everything is still in place correctly.

Now, I said this was asynchronous, and it is, the executed system-under-test kicks off a thread to talk to the web service, and so to get it to test, I needed to wait for the thread to finish, then do my assertions.

I used a system where the spun off thread will set a boolean when it has finished running, to let the main thread know it has finished, and the main thread just keeps checking that boolean until it is true. While hackish, that is my current working solution, as no more then one thread is ever spun off at a time, I can just have my test wait for the thread to finish, then complete the required assertions.

So, working through the Land of Lisp book, getting the hang of it.

Chapter Six, the author starts off by talking a little about the Common Lisp REPL, or Read-Eval-Print-Loop that you use as sort of the "command line" for interacting with lisp. It is pretty cool, you type in code, and it executes. And then he opens the hood a little:

(defun repl ()
    (loop (print (eval (read)))))

WHAAAAAAAAAAAAAAA??!?!?!

I just about spit my drink all over the monitor laughing. The function eval() evaluates whatever text is passed in as code, in this case, from the command line via the obviously named read and after executing it, print() prints the result, and it just does that forever till you quit.

That means, you could make one of these for any language that can evaluate strings into code. I know python has one built-in. Php can evaluate code with eval, and a quick search shows a highly nifty (and robust from the looks of it) REPL for php complements of facebook. Sweet. Or, you can use php -a for the interactive mode.

(They have thin skin)

I have been reading G.O.O.S. and honestly, much of it has been just a reiteration. Until chapter 20, that is. I get to the section Don't Mock Concrete Classes, and it clicks.

Mocking concrete classes used to be my bread and butter. Almost every unit test I wrote for the last year had several mocked concrete implementations, to pass in as dependencies of the system under test.

What do I mean by that? Imagine a class that takes one dependency through the constructor like this:

class Velocity {
  public function __construct (PayoffCalc $PayoffCalc)
  {
      $this->_PayoffCalc  = $PayoffCalc;
  }

  public function toFloat()
  {
      $paymentPerDay = $this->_PayoffCalc->getPaymentPerDay();
       return $this->_calculate($paymentPerDay);
  }
}

Would have a test driving it, looking like this:

public function test_velocity_validDailyPayment()
 {
    $paymentPerDay = 3;
    $expectedVelocity = 1392;
    
    //Mock PayoffCalc->getPaymentPerDay() to return 3
    $PayoffCalc = $this->getMock('PayoffCalc', array('getPaymentPerDay'));
     $PayoffCalc->expects($this->once())
        ->method('getPaymentPerDay')
        ->will($this->returnValue($paymentPerDay));
    
    $Velocity = new Velocity($PayoffCalc);
    $this->assertEquals($expectedVelocity, $Velocity->toFloat());
 }

For several months now, writing unit tests this way has become increasingly irritating to me, for several reasons. First off, notice how phpunit does a mock? With a string for the classname and the method to mock. Obviously, in small scale this duplication is manageable, but imagine for a second thousands of unit tests all mocking this way, and you will realize it gets out of hand fast. More than once in the last year I have set out to refactor a classname or method signature, to have dozens of failing unit tests, because they all mock the string of the old name.

Secondly, and more importantly, I find this to be brittle because you are locking yourself into an implementation of the class you need mocked. What if I wanted to move that getPaymentPerDay to a different class? Or I changed that version of getPaymentPerDay to getPaymentPerMonth? I would have to traipse through every one of those brittle mocks and rewrite their duplicated, crystalline structure.

Stupid.

My inner frustration was so great as to cause me to start thinking up all sorts of mad ways to get around this issue. Store the code needed to build that mock as text on the class itself, to be parsed and executed on the fly? Check. Create massive hierarchical testing libraries to build the mocks? Check. And all these did was mask the real issue. I was mocking the concrete implementation.

So, what would mocking the interface be? Well, PayoffCalc actually has two public methods: getPayoffDate() and getPaymentPerDay(). The Velocity class only needs getPaymentPerDay(), but by mocking the concrete class, we are forcing it to depend on the whole (undefined) interface of PayoffCalc. Velocity does not need getPayoffDate(), but it is in the interface we give it (in the form of the concrete PayoffCalc).

Whoa. Sounds like ISP. Written about by Bob Martin. Fifteen years ago. *Forehead smack*

I make an interface for this concept of PaymentPerDayRetriever, for lack of a better Role.

interface PaymentPerDayRetriever {
    public function getPaymentPerDay();
}

Then use the interface in the test:

public function test_velocity_validDailyPayment()
 {
    $paymentPerDay = 3;
    $expectedVelocity = 1392;
     
    //Mock PayoffCalc->getPaymentPerDay() to return 3
    $PayoffCalc = $this->getMock('PaymentPerDayRetriever');
    $PayoffCalc->expects($this->any())
        ->method('getPaymentPerDay')
         ->will($this->returnValue($paymentPerDay));
    
    $Velocity = new Velocity($PayoffCalc);
    $this->assertEquals($expectedVelocity, $Velocity->toFloat());
}

By mocking the interface, I have freed my Velocity class from knowing about the rest of the old PayoffCalc interface, and when I realize later that my PayoffCalc really is doing two totally separate things, I can just move the PaymentPerDayRetriever interface to getPaymentPerDay's new home (probably on the PaymentRepository, not shown here).

While this example only used one method per interface, in reality, it would have all the methods necessary to complete that aspect of the class.

Fascinating.

I have been playing around with lisp off and on for some time now. I started out in Emacs Lisp, making Emacs play as nice (nicer in some ways) as any available php ide, but then dropped off after discovering PhpStorm from JetBrains. When I heard about Land of Lisp, I was intrigued and picked up a copy to work through over Christmas, got a few chapters in, but as Christmas tends to go for me, I spent much of the time playing board games with the family.

And through it all, I never had even the slightest inkling of why everyone says you should learn lisp. These guys too.

What is the deal? Surely, I am using it wrong. Everyone who learns it raves, and yet after using it off and on for several years, all I see is just another language, with a steeper learning curve from the non-C, non-VB syntax.

Then, last week, this article came across my browser.

I read just the first quarter, and there was a riot in my brain.