Let’s get one thing out of the way. If you are a programmer, chances are you would love SCRUM. Love with a capital L.  I mean really…. Think about it… if you come from the typical top down process where “Moses delivers the ten commandments to the tribes”  -  in other words the Dir of Development sits the team down and explains XYZ has to be done in 4 weeks – if you come from that sort of place, and find yourself in an agile world where programmers look at “user stories” and estimate sprints while an entire team discusses and sets the schedule – you’d probably feel like you’ve made a wrong turn somewhere and ended in alternate universe.

Suffice it to say there is definitely a culture shift across an entire organization when you switch to Agile. Some companies may not be quite as ready as others to embrace the change.

One of the first victims to get hit by an Agile team is the persona of Heroic Programmer. The type of programmer who by design or default likes to single handedly save everyone’s bacon. Agile focuses on the effort, communication and process of the ENTIRE team – not just the lone gun slinger. People who need to be the Heroic Programmer in a company often will not like Agile very much.

Another victim of the rising popularity of Agile are dev shops that operate on outdated principles and support dysfunctional processes. Through the emphasis on team empowerment I find more and more developers take up the battle cry of “Give me Agile or give me … [fill in the blank]”  A recruiter friend tells me about candidates who will not entertain an offer at all if it  isn’t  in an agile shop. Kudos to them. There is much to like about improved team communication and process enhancements.

Even if your own company cannot be 100%  formally Agile, with everyone buying into the change,  perhaps you can use some aspects of the methodology to improve your own processes. That has been my experience.

One of the key features of Agile (SCRUM) methodologies is the prevention of process hijacking. In  my 20 years of work experience I found two major types of development shops – those driven by engineers and those driven by sales people. Whichever group controls the process ends up setting the corporate culture. Agile shares responsibility between both groups and therefore prevents either group from hijacking the environment. That’s admirable.

Agile teams work hard to deliver predictable, high quality results in a relatively difficult setting. Difficult because software development all too often seems to go against a normal  predictive process. Unlike a Model A Ford, software cannot be built in an assembly line fashion. Hence the difficulty.

The mandate of Agile development is to create an environment where that uncertainty is removed through an iterative process and  much, much, much more communication than any other development methodology I have seen.  Meetings before things get built, meetings while things get built and more meetings after your done building things.

I have seen two interesting situations in all these meetings: programmers who by nature might be more introverted become much more open, and stakeholders who are used to issuing an edict and walking away become much closer to the development team. Both are good things.

There are a number of books on the market that describe Agile and SCRUM better than I can, but in a summarized form, Agile consists of

• A prioritized product backlog of themes, epics and user stories
• A self managing and empowered, cross functional team
• A SCRUM Leader / Master
• One or more product owners / stake holders
• Sprints that break user stories into tasks and produce a certain amount of work over a specified time, most frequently 10 business days.

Of course those are only the highlights. There are a number of important “ceremonies” and behavioral changes that come with the SCRUM territory. For example the practice of daily stand up meetings where team members briefly touch on what is on their plate.

In my own experience, the area where Agile makes the greatest impact is in the shared responsibility and empowerment of the team. By giving the team a way to estimate, manage and build code quickly and iteratively the group as a whole tends to be very productive.

During my time as VP of Server Development of SkillJam Inc. I was able to move a dev team toward a more agile approach by making sure a few key features were in place. First and foremost, we instituted a good product backlog.  I purchased enough licenses of Fogbugz to take care of all developers, QA People,  project managers and product stakeholders in the organization. I cannot tell you how important some of the seemingly simple features of Fogbugz became once we started cranking out code. Something as simple as an email conversation thread being automatically stored and associated with the task or issue in Fogbugz is a life saver. (Instead of logging into Fogbugz to record a comment, an email reply to a mail sent from Fogbugz automatically shows up in the right position within the task comments. Nice!)

Having set up FogBugz we got busy producing a big picture prioritized list of work. Any bug, any feature request and even entire new products where organized by themes , user stories and tasks within the stories.

After some arm wrestling with the key product stakeholders and other Senior Management Team members I was able to get buy-in for the concept of 10 business day sprints ( 2 calendar weeks). These sprints could not be interrupted for unplanned tasks under any circumstances. That’s not to say an emergency shouldn’t be addressed by the team.  But let me tell you,  its not easy convincing a CEO that giving up the ability to demand something RIGHT NOW is good for business. In the end it was the prospect of better schedule adherence, better quality and lower defect counts that sold this effort.

As I said, we were not formally Agile across the entire board. Our organization loved a considerable amount of design up front. That meant more information was written down and documented in formal requirements than you would typically see in an agile shop.

At the same time, our developers and qa analysts met several times a week for progress checks – admittedly this could have been handled with daily stand up meetings to which we could have invited visitors ( as long as the visitors kept quiet), but in a nod to the PMO we continued with the legacy status meetings.

As work progressed through a sprint we needed to look ahead at the pipeline ( the product backlog ) . So toward the end of a given sprint the team leaders and myself would spend some time planning the next iteration to make sure we could address any problems before we had our next Sprint is to kick off. For the actual kick off itself, the entire dev team (including qa people) gathered around a large conference table where we projected to prioritized backlog out of Fogbugz on a large screen. The team as a whole took on the User Stories needing to be built – based on prioritization – and committed to a certain amount of work being done in the sprint. Frequently we invited stakeholders and product owners to these meetings in order to get immediate feedback to questions that helped us with the production of a Sprint schedule.

That’s it. Simple and straight forward. I was blessed with a fantastic team who had been working together for a while already. That made SCRUM planning easier. And yes I know this wasn’t the  comprehensive SCRUM setup that you read about in the many Agile books on the market, but even in this pared-down format we were able to reap the benefits of planned iterative production.

I love the way MVC separates out the code that used to get thrown into the same code behind in web forms. It really shines in that regard. Who knows maybe it will have a positive influence on webforms apps in that regard.

For the moment my bottom line is that it takes a lot longer to write the same functionality in MVC than webforms. That’s partly due to the learning curve and partly due to the fact that there is more code to write. The fact that there is more code will be interesting to watch. Maybe in time people will find some efficient practices that will help to bring down the overall effort required. If thats not the case I wonder how someone would justify the cost of building a system one way versus another. And dont tell me to build it in MVC because webforms are evil. Thats nonesense. We have almost ten years of webform driven sites out there.

Lastly, for all those folks who are jumping on the “Viewstate is evil” bandwagon – which seems to be the same lemmings who jumped on Datasets as a DAO in 2002 – have a look at the following performance test. 6-8 requests per second for an MVC app is laughable. I sure hope that’s not the case presently. Its worth some internal testing to see. By the way, just on a purely subjective note, for the past year I have frequently spent time on MarketWatch. Since they switched to MVC I have noticed a small performance penalty. Purely subjective. Heck maybe its even a matter of browsers.

My main issues with MS MVC is the fact that it tries to be something very similar to RoR. I feel this way because Model View Controller can be done without MS MVC. As a matter of fact as I have mentioned ad-nauseaum Dan and I have built and MVC driven framework a while back. So if I am interested in just the simplest most straightforward way to plug MVC into ASP.NET – thats the way to go – just build a small and simple action framework / router. No tag-soup either. I suppose I have to revisit this after ASP.NET 4.0 is out because it will incorporate routing.

Having said all that as a preamble, I would be silly not to study up on MS MVC. There are some aspects I really do like about it. The great enforcement of separation of concerns for starters.

As I looked over the web to find some decent examples of people who have blazed trails in this area I came across Karl Seguin’s Canvas MVC sample app. It was written with one specific purpose in mind: as a simple learning application that illustrates a good way to build an MVC app. In my opinion this app is a resounding success. It has enough code to illustrate the majority of work that one has to deal with (i.e. data entry, paged lists etc). And it doesn’t try to throw every possible scenario in the mix. The result is a well structured easy to follow sample. Believe me I have looked at numerous different ones out there and this is by far one of the better designed samples. Love it !

Fast forward to 2009. ASP.NET MVC hits the market. Only its not just routing and actions its a whole big kit and kaboodle. Reminds me a little of RoR. Along the way all of a sudden a lot of folks are complaining that Viewstate is the big evil and if only they could code without it. Meh. Sounds a little like people flocking to the latest shiniest thing. And there is nothing wrong with that. Its just ironic that many of the personality types that flock to this technology are the same type of folks who argued that datasets would save the world. Geez.

MVC is good stuff and my buddy Phil has poured his heart into this project. So I certainly hope it will thrive and progress. What has me confused at the moment are a plethora of adjacent projects that are growing up around it. I have a hard time discerning which to take serious. What do I make of an open source “architecture” that spits out a bunch of scaffolding pages and uses itself two other OSS projects. By the way, am I the only one who sees a trend of OSS projects where someone produces an entire new something that is 50% built on top of someone elses new something. Hope that makes sense. Did the RoR community go through a similar evolution?

As a consultant I need rock solid tools that are not necessarily going to be impacted by a deprecation of some obscure aspect because the manufacturer used some OSS project that is no longer popular. I really should research how the RoR community deals with this.

And while I am on the subject of “architectures”…. seems that there are some ASP.NET MVC “architectures” floating around that are primarily fantastic code generators. I would actually call them Form Wizards on Steroids. Hence the pic.

In 1992 I worked on a pretty large MS Access app. Yes we did those and they were good. Dont laugh. MS Access was the MVC of its day. Anyways, the IDE had a Form Wizard. Point it at a query or table and it would generate all the code you need. I learned an important lesson back then about code gen. Especially UI code gen. Dont use it. Wizard generated code is great … al the way until you need to change it. And Wizard generated apps tend to contain a lot of unnecessary stuff that you wouldn’t produce by hand.

Now mind you I am not talking about the utilization of some other view engine. Spark for example looks pretty cool. I am talking about the notion that a utility spitting out a bunch of views and controllers and tests is a measure of productivity. It is not.

Speaking of productivity. Rob Connery had a very cool demo of MSpec on his site. Finally a spec / test system that makes sense. (Besides Fitnesse that is).

Along came Object Relational Mappers. Be that NHibernate or commercial ones. At its core, the ORM among many other very useful things, provides a way to quickly and relatively painlessly abstract your code away from its database. Therefore – and this is important – you can write an application against one database, then swap out back-ends for another database and with a minimum of changes be up and running in no time. One of the best examples of this was watching my buddy Dan change an ENTIRE learning management system from SQL Server to MySQL in next to no time. I think he said it took less than a week.

So I ask you, if you are MS and you see an upsurge of tools that allow developers to do this. And there is more and more buzz in the various blogs about ORM’s and the benefits of other technologies, like ActriveRecord, MVC etc etc. What would you do? Based on 15 years of watching MS, I believe the embrace and extend methodology will be applied here with great success. In the end I suspect MS will provide something to the dev community that looks and smells a little bit like an ORM but under the hood it will be just a tad different, making it easy to use with MS Tools and through its very usage making it hard to switch databases, effectively negating one of the core points of an ORM.

With everything you know about MS… doesn’t that seem like a reasonable idea? No wonder some people have begun to call the MS Entity Framework a persistence mapper not an object relational mapper.

I would also like to juxtapose the opinions of Sam and Fowler with the absolutely shameful way that MS has been acting toward Jamie Cansdale who had the gall to be smart enough to figure out how to create a VS Express Add-In without using MS’ plugin architecture and keys. This whole scene stinks to high heaven. No wonder people are leaving the realm of MS centric development. And the more people leave the more brainshare alternate technologies gather. In addition the more people leave the more pressure there will be on companies to implement projects with alternate technologies. I remember the days when VB was very young and most corporations were advertising for PowerBuilder developers. We are now in a similar cycle. For Pete’s sake if you have guys like Scott Hanselman going to Ruby conferences and blogging that he is a “closet railer ” (ok thats my term ), then you know somethings up.

UPDATE 6-3-07: Frans Bouma who is an ISV responds at length to the ridiculous crap the MS is pulling with Jamie. You know, the “anything but MS” camp has been accusing MS for many years of this crappy behavior. I always thought there was a lot of posturing but it looks like they were right all along. MS really is a bully.

I am working on an implementation of an open standard for the hospitality industry. One thing really caught my attention at this company – the all pervading sense of quality that drives the organization. From schematics to manuals to pretty rigorous QA testing, you can tell here is a business that really wants to succeed by providing a superior service. I mentioned this observation to one of the VP’s . He chuckled and said ” When you have 5 star hotels as customers, you need to be a 5 star vendor”

That actually made a lot of sense to me. One of the reasons there is so much crappy software floating around – especially in the companies working in web applications – is the fact that a lot of this software is ordered by business units instead of IT departments. And so many business units in corporate America have no clue what quality software is. Its not their core competency so I dont blame them. That’s not to say that they wouldn’t like a working product. Of course they would. But it has been my experience that projects which are ordered and managed by business units directly, tend to go astray more than those run by IT. As a matter of fact I know one development outfit where the prospect of rigorous oversight would instill a great deal of concern because this company almost runs all projects on the notion that they can thoroughly befuddle their customers with technical details, which then gives them more time on the project and occasionally makes them look like a hero to the unsuspecting business manager. This would never fly with IT involvement.

To prove that every sweeping generalization has an exception, this stands in marked contrast to the way hotels seem to deal with their vendors. Especially luxury hotels. Try and tell the Peninsula in Beverly Hills that their high powered guest can’t do something with your product because ….. [insert typical excuse here]……. Try that and see how long you will be in business with the Peninsula. The VP I spoke with indicated this is the case for most of their high end customers. His company, in a sense, has to be on par with the level of quality that guests are used to. These are guests with private jets, very expensive high quality German cars, who expect to get what they pay for.

So there you have it. I found a quality company in a most unexpected place for the most unexpected reasons. But when you think about it, the company has to operate this way in order to compete in the hospitality industry, which at this level of service has some very high demands and expectations. Which in turn drives the product quality. Neat.

He says: “...Scientists should stick with science. What they invent and discover should be moved to the real world by engineers, not by scientists as well. Because, if you leave it to scientists, you’ll end up with horrible syntaxis like the ROW_NUMBER() function in SqlServer 2005….”

Read the entire post. Its certainly eye opening to everyone who hasn’t had to implement paging in another db outside of SQL Server.

[Remember WinFS? Remember we spent thousands of years of developer effort and billions of dollars, and delayed Vista by years... and ended up canning the whole thing. Well the only thing we salvaged from the whole sorry fiasco, was something called the ADO.Net Entity Framework. It allows you to use three synonyms, -- entity, object and model -- with three other synonyms -- schema, domain and framework -- to produce up to 36 different connotations from just two simple concepts. There's no problem so simple that developers can't over engineer it. Anyway -- when we say 'Significant Investments' we mean billions of lost dollars! and, well, let's be honst, we really mean, F*** YOU DHH! You cannot hope to challenge us! We are massive! We are the borg! We will add your technical and cultural distinctiveness to our own! We do not sleep!]

Read more.