It seems every time I think of an idea, someone has already thought of it. More so, its typically Google. Previously, I mentioned that it would be great to have a highly collaborative environment for coding. Now, they’ve just unveiled Google Wave which is going to be the definitive online collaboration environment. It will redefine communication on the Internet and how you interact with people. No more trying to organize trips, weeding through endless emails, managing workflows. Google has the answer to everything.
Google has quickly become a depot of innovation and an incubator of ideas. The challenge for the rest of us is to be able to co-exist and if possible, even succeed without being destroyed by Google.
Being an entrepreneur is difficult enough without having this looming cloud of Google sitting on top of you. Once you do find your idea and fall in love with it enough to accept that even if Google is doing it already, that you will still ford ahead and do it, then the fun starts. For the most part, the technology part is the easy part, its everything else that’s a pain. Marketing and sales can be tough and shouldn’t be taken for granted. Of course there are some who say that a product markets itself, and that usually just takes patience and hard work.
Check out the video, the intro is a little creepy and cultish, but the demo is pretty neat. You can skip ahead to about 8:02 of the video.
“Great software, likewise, requires a fanatical devotion to beauty. If you look inside good software, you find that parts no one is ever supposed to see are beautiful too.” - Paul Graham, Hackers and Painters
I am often brought back to this essay by Paul Graham. He often grounds us in our pursuit of perfect software by putting in perspective the nature of our work. We may not be as artistic as our tradesmen counterparts, but if we strive for a more balanced reform between technical work and artistry, we can deliver high quality and more original work, leading to innovation and greater aspirations.
Creativity is key in software development as much as it is in any other profession.
In Uncle Bob’s book on Agile Development Principles, he mentions 7 design smells. These are all very good articulations of what it means to have clean, maintainable and extensible code.
What he does not mention is that there is another smell: Fragile Unit Tests (FUTS). Perhaps the reason no one thinks about this, is because not many people actually do unit tests at all. If you code has fragile unit tests, you’ve surely got even more fragile code. Having a solid foundation of tests acts as a safety net, but when your net has holes, it will do no more good than just adding to the maintenance of your program.
After having read Kent Beck’s article on Just Ship, Baby, I have become more wary of unit testing for the sake of unit testing. I have since then come up with my own rule of thumb for TDD. If it hurts to do something, then you’re probably doing it wrong (or don’t do it at all).
Here are 3 other testing smells that I think are good indications of your code having FUTS and in turn, lead you to take notice of more subtle design smells lurking in your code.
Zombie Unit Tests
With unit testing, it can quickly become a giant blob of ugly code. You’re constantly maneuvering an obstacle course of external dependencies and as you do, you’re not really paying attention to the mess of prior tests you’ve written. You’ve started in earnest with very solid TDD principles, and promised yourself to go back and refactor those duplicate lines of code. As you’re getting further, you fall behind in your refactoring, but hey, they’re just unit tests, not production code!
You’ve finished enough to get 80% code coverage, and breath a huge sigh of relief, never to embark back on that class again. Like Attack of the Killer Tomatoes, you worry that one day, your giant class of four or five unit tests will come storming down 5th Ave heading directly for your head! So our rule is, If you are dreading your unit tests coming back to haunt you, then your tests are already unmaintainable.
Making a Mockery of Your Code
Legacy systems are hard to test. You’ve joined a company and congratulations, you’ve just been awarded the worst inheritance ever: 600K of legacy code. You try your best to add unit tests, slowly unweaving the external dependencies, and realize, there are still more external dependencies. This is something you should recognize as bad design according to the Rigidity design smell. If you can’t inject dependencies then you can’t test without the dependencies. So what is your only recourse? Mock objects. There are several drawbacks to mocks. I consider mocking to be the last resort or last line of defense against legacy code.
The danger with mocking is if you end up mocking out too much at the beginning. You know its going to be hard when your code is considered “legacy software” (probably in 3 months), so why put yourself through it now? Heavily relying on mock objects is a good indication that your design has some serious flaws. Even with JMock 2.x, you’ve got several sequences or orders that need to be mocked out exactly in order for the test to fail. So our rule is, If your test relies on a lot of mocking, you are likely violating OCP.
Impersonating is Fragile
Along with heavily using mock objects, the other key insight is that mocking objects and not roles creates fragile unit tests. Roles are defined by interfaces, which can be implemented in a variety of ways. If you mock out concrete classes, you are constrained to the internals of that particular implementation, and now you’ve violating the The Interface Segregation Principle. Thus forcing a tighter coupling between your code, and your tests are not proving anything except for this design flaw. The only time you should mock out a concrete class is if you have no way of changing that class (e.g. EJB2.x). This was also mentioned by Steve Freeman, co-author of jMock library. jMock provides a class to do so, called Imposteriser but it has been moved into the legacy module. So our rule is, Prefer mocking roles and interfaces, than objects and concrete classes.
Conclusion
These a few that I have taken note of and have made an effort to be wary of when I’m practicing TDD. Recently, I’ve been more inclined to define contracts first, rather than tests, in order to maintain a more aligned adherence to the domain problems. Setting up your contracts as though you were the client has helped a great deal in deciding what to test, how much to test and how to focus on solving the business problem at hand.
Does doing TDD make you a professional? Software debates can be dry at most times, but this debate between Robert C. Martin and Jim Coplien (”Curiously Recurring Template Patterns”) gives rest to the notion that TDD can be substituted for good (up-front) architecture. Though friendly and jovial, I found the talk to still be enlightening, particularly hearing Cope’s opinions on XP and the benefits of doing CDD (contract-driven development).
“On the other hand I don’t believe architecture is formed out of whole cloth. I believe that you assemble it one bit at a time, by using good design skills, by using good architectural skills, over the weeks and months of many iterations. And I think that some of the architectural elements that you create, you will destroy; you will experiment in a few iterations with different forms of architecture. Within 2 or 3 iterations you will have settled into the architecture you think is right and then be entering into a phase of tuning. So my view of that is that the architecture evolves, it is informed by code that executes, and it is informed by the tests that you write.” – http://www.infoq.com/interviews/coplien-martin-tdd
It is frustrating how so often projects treat unit testing as an afterthought, scheduled somewhere between development completing and QA testing beginning. Project plans seldom meet their deadlines, so unit testing is often the one to suffer. Or in some cases, testing in general.
Frameworks that can assist in unit testing have been a major time saver for the amount of work that would otherwise need to be put in. I’ve recently re-discovered the joys of development, and in particular test-first development. Using the Spring TestContext Framework it has become even easier to not just do unit testing quickly, but also integration testing.
The importance of both is often understated, as integration testing is more crucial when there are external dependencies. With a few annotations, one can create an integration test simply by specifying the database connection settings and focusing on the critical test paths through the business logic.
The test-context-data.xml contains your usual datasource, transaction manager and other bean configurations.
With this class, you can test logic with rollbacks, test without rollbacks, or you can set up transactions before entering them. If so desired, you can also extend AbstractTransactionalJUnit4SpringContextTests to have access to convenient underlying low-level classes.
This is the way to develop, and its unfortunate that developers will most likely miss out on this, as they are rushing to meet deadlines and pressured to push out code. Using annotations with the Spring TestContext Framework gets us one step closer to having a real internal DSL that allows us to develop in the context of the domain. We can start shedding all the baby fat of traditional Java development, and worry less about all the plumbing.
If Java is to compete with the young, fresh new blood on the horizon, it needs to continue moving in this direction. Java is a powerful language, but it must become more lightweight and faster to develop to stand up against languages like Ruby or even Python.
I’ve recently begun to pay more attention to celebrities. Celebrities like David Deutsch, Dave Eggers, Brian Greene and Nobel Laureate James Watson. Celebrities that are, indeed, far less likely to appear at a Hollywood party than a pink chihuahua.
Though these celebrities are not mainstays of tabloid newspapers, they are inspired purveyors of hope and vision. Listening to these talks have eased my transition from startups to big name corporation and have given me the motivation I need to get off the couch and start building on these ideas.
Prior to leaving my last company, I had spent hours weeding through Monster.ca, Workopolis, and various other avenues for employment seekers. Through weeks of reviewing job postings, looking at companies and hoping to match them to my own ideals, I found there were very few that stood out as fresh or very agile at all. I was looking for a company that believed in what they did, not to become rich, but to be able to sustain themselves so that they could continue to do what they loved, indefinitely. To me, that is the ultimate success. Now, where would I find something like this?
It use to be that you could go onto craigslist.org and see very unique job postings. Since only the “cool” people used craigslist, it was evident that only “cool” companies would post job ads there. No longer is the case, as word has spread and the enemy is onto us. Now, everyone is trying to be cool. It seems, nowadays, everyone wants to be the cool, hip, company. Unfortunately, in reality, these types of companies are very rare — and that is exactly what makes them cool.
Some of the postings on craigslist are actually written in a very affected language, hoping to entice fresh minds with their somewhat obvious marketing ploy. Imagine your dad, wishing to fit in with your generation, trying to make jokes about Kanye West and Britney Spears when your friends are over hanging out. His attempts to fit in, readily fall out as awkward, sad and ultimately, pretty lame.
It seems this kind of marketing is becoming more prevalent, with the onslaught of Koodoo ads. Everywhere I go, there are pictures of leotards and cellphones. These ads hope to win you over with its slightly obvious attempts to connect with your hipster/scenester/trendster sensibilities. How does this kind of advertising do anything but annoy people? Similarly, in the delicate art of recruiting, these gimmicks not only fail to attract the right talent, but drive them away or worse, attract the wrong type of talent.
It is entirely possible that I have myself become out of touch with what the kids think is cool these days. Perhaps I am the Dad (who is not a dad) without a sense of humour. Could it simply be a matter of my own conceit or rather a deplorable closure of ingenuity and authenticity?
I have reached a point in my career where I am slightly behind in experience, yet slightly ahead in ambition. While working at my last company, I’ve condensed in 2 years more experience and exposure to technology and practices than most people would in 5 years at a large corporate company. In fact, during interviews, I’ve often been asked, how is it possible that I was able to work on all these different technologies, as though it were slightly incredulous for this to be true.
Most other companies, the opportunity to experiment or adjust if something is not working, is harder and more costly. When requirements change, as they often do, large companies cannot adapt quick enough because of all the processes that were put in place to protect the company against the whole trial-and-error affliction which most startups are generally known for.
In the software industry, change is often the only constant, but change in large companies is often difficult and slow.
These companies have adopted as a matter of practice certain methodologies from 30 years ago that at the time seemed fine and worked for many projects. Those software processes that large corporations adhere to, cannot handle today’s fast-paced environment, yet many of them are trapped because they’ve grown so big that they can’t risk changing to a whole new process. As the common phrase goes, “Better the devil we know, then the devil we don’t.”
Unfortunately, the software practice is still immature, and what was agreed as the only way to develop software in 1980, has now many other, and in many ways, better, options. How do we ween corporations off the old processes that are costing money? And how do we get them to adopt a more agile approach? It must start of small, and with a project whose success or failure is equally less important. It must grow organically as a movement from within the groups. This will then lead to bridging the gap between management and product development.
There is a disconnect between those who understand how to drive a business, and those that understand how to get the most from technology. Then there is also the disconnect between those who are managers, and those who are developing the product. I believe those that manage software developers should be as or more passionate about software than the developers themselves. There is nothing more disheartening than being lead by someone who cannot connect with you at the most basic level. Leaders who do not lead from within the group itself, risk being seen as an outsider or worse, as the enemy.
In some instances, I have noticed that developers are often viewed upon as factory workers; resources who are tasked and are accountable for completing those tasks. This type of thinking discounts the ability of software developers to provide any more value than what they’ve been tasked with.
I subscribe to the more unpopular belief that software developers are capable of being more creative than most other professions. If we were to learn anything from Google’s experiments, it is to say that a company lead by technologists (Brin and Page) in equal cooperation with a strong business mind (Schmidt) will undoubtedly yield ground-breaking and innovative products with high utility and soaring profits. Google has become unparalleled in brain-power and proven that business growth can go hand-in-hand with technology growth. If we could only spend some time investing in change, and preparing ourselves for change, perhaps we’d understand better, how to be as successful.
In an attempt to be more versatile and perhaps less Java-centric, I’ve taken a look at learning a new programming language. The choices were narrowed down to classic Objective C, Ruby or Python. Given that Objective C is a bit more arcane and less likely to make much headway on the Internet, I’ve now got to choose between Ruby and Python.
Looking at both of these language, I can immediately see the similarities. Neither requires compiled binary class files, and both are compact languages. Clearly, Ruby is the more trendy language as everyone and their roommate’s girlfriend’s dog seems to be doing it. So I chose Python.
Python is a fairly young language, currently in its 2.x incarnation, and 3.0 under development. Invented in late-80’s by Guido van Rossum, it is heavily influenced by Modula-3 and Haskell (!). It had gained a large cult following of hardcore developers and early praise from the likes of Martin Fowler. Fowler once wrote in his Patterns of Enterprise Applications book that he’d love to see Python give Java a run for its money. All we need is a huge corporation to back it, and a guerrilla marketing gambit.
In my brief excursion so far, a few programming principles have come into question. Revelations of the clumsiness of the Java language have surfaced and I’ve gladly enjoyed the experience so far. Certain things that only nerdy computer scientists would fine remotely intriguing, such as comparisons between statically-typed vs. dynamically-typed lanugages, checked vs. unchecked exception and verbosity vs. compactness of a language.
As architects and designers we are resolute in attacking problems creatively and without prejudice. As implementors and builders, we must understand all the tools available to us and appreciate the differences between each. Knowing the advantages and likely uses of one over the other will afford us a greater breadth of knowledge and allow us to make better choices.
I have recently discovered Google Docs, and decided that I love it, like many things Google. I was collaborating on a few documents with a friend, and watched in real-time as he edited and fixed my typos. He was somewhere in Vancouver, BC and I was sitting my gitches on my couch in Mississauga, ON. It dawned on me that this is collaboration could be the key to distributed development.
Suppose there was Google IDE, you could have developers pair-programming remotely and watching each other code. It would be revolutionary! Incorporated with Subversion and revision history, your development environment now lives on a remote server. For fresh, new startups and open-source projects you could house your project and code on Google. For corporations and proprietary source code, you could install the web application internally.
In some ways, this would be similar to the “screen” program in Unix, where you could trade control back and forth to each other and see each other type. Of course, it would need to have all the features of eclipse, minus the bloat and memory consumption.
When the Internet goes down, you could still code and work, since its just a text editor. Perhaps give the option of storing a cache locally for offline development. Perhaps even add a merge functionality for when you reconnect to the repository. It would need to be responsive, so obviously the UI has to be responsive like everything else Web 2.0.
No longer would you need to download overly bloated IDEs, or wait for minutes as the Mac color wheel spins, eating up memory. No longer would you need to undergo a learning transfer as you upgrade to the next popular IDE of the Month/Year/Decade.
How would we build this, short of getting hired by Google? If we built this first, there’s a slim chance that Google will buy us. Slowly developing more and more applications running solely on the Internet, Google has solved many problems and done the hard part for us. Soon “Google” will become synonymous with “Internet” and that would be just fine with me.
We need to take our cues from Google, and create wealth by solving the hard problems. Build something people will want to use, because our measure of success will not be how technically perfect our solution is, but by how many people find our product useful. Otherwise, it is just adding to the already cluttered closet of the Internet.
I’ve always had a penchant for all things miraculous: God speaking to me through a homeless person, large structures randomly falling out of the sky, estimating a development effort accurately 8-months in advance. I’ve recently been tasked with the last one, given that we are subscribing to a SDLC that is slowly falling down a giant waterfall. We’re currently in Week 13 of requirements gathering.
The field of software engineering has evolved through years of theoretical research. Computer Scientists have come up with great ideas that should work in theory, but are seldom applicable in real world situations. I have learned more about software development out in the field than university could ever have prepared me for. Most of the things I did learn in university, I would need to look up on the Internet anyway, say if the rare occasion ever arose that I would need to know how hash maps are implemented using Red-Black Trees (I only apply at Google once ever 18 months).
Software is an immature discipline and software itself rarely lasts much longer than 4-5 years before it becomes legacy. As such, it becomes increasingly hard to predict, as complexity increases. Only thirty years ago, computers were the size of refrigerators and Irene, the 70 year old grandmother you see today coding in COBOL while staring at a 50lb CRT monitor running at 640×480, was in the prime of her life. And in fact, there’s an army of Irenes who don’t like it when you show up in Lacoste shoes, and tight pants. Sometimes I wear pin-striped dress pants with a checkered shirt and that completely throws her off.
Irene probably went to a very good school where they taught her everything to know about documentation and software specification. In fact, she probably received accolades for her work in the field of requirements engineering. But now, the business’ needs have changed drastically, as have the times, and that mainframe system backed by a flat file-based database, can’t support what the business needs to do.
Large corporations reinvent themselves all the time to accommodate climate changes in the economy, business direction changes and continually respond to the market’s whims. There is no reason software development shouldn’t be any different. Software processes need to be adaptive to these changes just as much as individuals do. Clinging onto traditional methods that are known to be heavyweight and slow will only demean what we do as professionals and continue to lose the trust and confidence of our customers. Arguably, some of the time, developers are happy to engage in lightweight processes, but its the archaic bastions of old-school software engineering who claim authority over this subject matter, that refuse to allow it.
Agile development is so radically different than most of what they’ve been teaching in school that traditionalists are worried that they will need to adjust their thinking. Meanwhile, there are those of us who have been dying, waiting for something like this to come along and revitalize our craft. In time, agile processes will mature and evolve, and things like distributed agile development will come into focus. In time, agile processes will become obsolete and perhaps a successor will emerge as a better way to build software and provide business value. Those are the things we should look forward to and embrace, rather than reject and undermine.
How do television evangelists do it? Day after day, they convince millions of Americans to follow them and give them money, and yet I can’t convince one person in this organisation to adopt a lean software development methodology. Perhaps, I’d have better luck converting them to mormonism. But that’d be too easy. Given the option of marrying multiple 16-year olds, I’m sure they’d choose that over lent any day. I’d be happy with just one.
Christians need to work on their marketing. The best that Christians have come up with in the last 100 years is a fish. Lame! I recently saw a bumper sticker that said, “Try Jesus” as though Jesus was a spicy Mexican dish that you’d never seen or heard of before advertised on a giant poster on the side of a dingy roadside restaurant that read, “Try our new Mole con Pollo!” And frankly, I don’t doubt that Jesus loves mole con pollo.
I realized on the weekend that I really don’t like to iron my own clothes. In fact, I don’t like ironing other people’s clothes either. How blissful it was, to just throw on a pair of jeans and tee-shirt and head to work. These days, I have to have a routine in order to get ready for my day in consultinghood: underpants, dress shirt, dress pants, belt, security badge, cell phone and keys all to the iTune of Tom Jones. Sometimes, I throw on a sweater vest.
Sometimes, I consider everything in perspective. Yes, I do enjoy looking like Carlton Banks, just not everyday. It makes me wonder if perhaps the only way I can achieve job satisfaction is to run my own company.
I have an idea to start up a website that incubates business ideas in partnership with technology. Something like what open-source software is, except for entrepreneurs. A common meeting place online where people can post blogs about business ideas and have a community of contributors help build that idea. Open-source ideas where everyone is free to use and incorporate it into their business model. People who are successful contribute back to the community with lessons learned, and help future entrepreneurs.
An Idea Farm of sorts.
I would love to start something like that, even in real life. I suppose its more akin to what Google has done with the 80/20 rule, where 80% of the business funds the 20% of the entrepreneurs. Wouldn’t it be rad if we had a 20/80 rule where 20% of the business funds 80% of the cool ideas.
For now, I’m commuting in my Civic 46.8 kms to the client site (which I’m not allowed to say who) where I sit in the basement elbow to elbow with other consultants and try to convince someone to buy more of my time. That is what I do; I sell my brain through Word documents and Power Point presentations (they’re called “decks” in the biz). Maybe someday, through RFPs as well.
The consulting I want to get into is the kind that ThoughtWorks does or what SpringSource does. Agile Software enablement, or training. Developing skills through SOLID methodologies. But of course, these types of engagements are harder to sell because there’s no visible bottom-line. You can’t tell the VP to pay $250K for two hippy consultants and all they’re going to do is make the employees happier. What a waste of money! We need to deliver business value through better software practices.
So that’s why there’s so many shitty projects out there. Because if the project was fun and exciting to work on, there would be no reason to bring in consultants. Most companies only bring in consultants when they are desperate.
Currently, I’m working as a Business Analyst. My official title is Senior Consultant, which translates to “A Much Older Business Analyst.” I write requirements documentation and do analysis. I take those documents and get sign-off from the business. Then I take that document and go to the Systems Programmers and translate it to tech terms. Then they say its impossible to do that in COBOL and I say, What the fuck are you still doing supporting COBOL, and they say, Its bringing us in $280 million a year so we’re not touching it. And I think, That’s fair.
And it goes back and forth a few more times, and in fact, a few more months until someone in a pair of expensive running shoes jumps in the room and yells out, Just Do It!™ And to your disappointment and mine, its not Tommy Vu!
Perhaps the hardest part of consulting is getting things to start moving. Analysis paralysis is an easy trap to fall into, and while most people think they’re being iterative and agile, rather, they are just not moving at all; only everything else around them, particularly time, is moving along quite splendidly.
If everyone by some miracle ends up happy with the requirements documents, then we start the design. The design is perhaps both the most painful and most fun part. Partially, because I come in knowing nothing about the existing system, but I’m required to come up with a working and viable solution and design document. Sweet! What about proof of concept or spiking on a task you say? What! You have just violated Waterfall process. Go back 10 paces. In this game, if you can somehow get in some coding, you probably haven’t been at it for long enough for your technical skills to become stagnant.
Perhaps I’m too jaded already for this job. Perhaps I haven’t given it a fair chance yet. Perhaps tomorrow, I will be reassigned to an engagement where I can actually use what I’ve learned the last three years and maybe even improve on it. Perhaps tomorrow, I’ll win the lottery and never have to work again. Personally, I hope I win the lottery.
