This month marks the 20th anniversary of the fall of the Berlin Wall. Having grown up in East Germany and defected in 1979, I have  a special connection with this symbol of oppression. To most of us living behind this wall it really seemed insurmountable. For starters, unlike the West Berlin side where people had free access to the wall, in the East you could not even get close to it.  If you tried to approach it you would get shot. While the West has used the wall time and time again as a staging ground of political and cultural assaults on the oppressive East German and Soviet regime, in the East the government cleared entire buildings of occupants because of the close vicinity to the wall.

From Kennedy’s famous quote of “Ich bin ein Berliner” ( I am a Berliner), to Reagan’s challenge of “Mr. Gorbachev – tear down this wall ” even down to U2 performing concerts in Berlin singing about “scale these city walls” – no other symbol in modern times has stood for the oppressive regime fostered on the former Soviet Occupation Zone  after WW2.

As much as I may think this anniversary is important, it only seems that way to a minority of people in the world. German Chancellor Angela Merkel during her recent visit to Washington DC invited US President Obama to the 20th Anniversary festivities.  I am told he politely declined. The Berlin Wall doesnt rank that high on current political agendas anymore. In Obama’s defense, even within Germany itself I find an entire generation of kids having grown up after the fall of this wall who nowadays have next to no connection at all with it , nor with  the former East German country.  Many of these teenagers who are  now the same age as my own daughter point to a distorted image being handed down by parents and relatives that are having a hard time adjusting to life in the competitive world of free economies. Many members of my generation are infected with nostalgia about the old system. Freqently you will hear descriptions like ” things weren’t really that bad. We all were in it together. Everybody had work and everybody had a roof over their head” – unfortunately, much like an Adult Survivor of Child Abuse will see the former abuse in less severe terms, these folks often see East Germany through rose colored glasses of nostalgia and  don’t remember having to stand in line at stores to get something as simple as bread – or the fact that every other person you meet was likely to be an informant.

Yes life is different in a free society when your no longer told what to read, think or believe.  But overall I much prefer it this way.

President Obama could not make it to the 20th anniversary, neither could President Reagan for obvious reasons – but U2 can ! Bono and his group are performing under the Brandenburg Gate in a massive concert being televised via MTV Germany. Good for him.

For many years after our defection I used to have nightmares about the Stasi finding us in the US and taking us back to East Germany. As I watched the wall coming down in 1989, almost ten years after we made it into the West,  sitting in the comfort of my living room in Los Angeles I did not realize how much fear I had carried with me.  It wasn’t until tears streamed down my face and I cheered for all the East Berliners crossing over into the West that I sensed a mighty feeling of relieve.  A weight that I did not know had been there was lifted off my shoulders.

Thinking back 20 years, it still amazes me  how profoundly an event that affected 15 million East German’s in the middle of Europe also affected one singular person  thousands of miles West of the Berlin Wall. Even though I had been free “on paper” for ten years already,  I felt a profound sense of freedom rolling over me as I saw wave after wave of East Germans pass into the West.  I laughed, cried, and clapped my hands remembering the old saying ” Would the last person leaving East Germany please turn out the lights” .

Let’s get one thing out of the way. If you are a programmer, chances are you would love SCRUM. Love with a capital L.  I mean really…. Think about it… if you come from the typical top down process where “Moses delivers the ten commandments to the tribes”  -  in other words the Dir of Development sits the team down and explains XYZ has to be done in 4 weeks – if you come from that sort of place, and find yourself in an agile world where programmers look at “user stories” and estimate sprints while an entire team discusses and sets the schedule – you’d probably feel like you’ve made a wrong turn somewhere and ended in alternate universe.

Suffice it to say there is definitely a culture shift across an entire organization when you switch to Agile. Some companies may not be quite as ready as others to embrace the change.

One of the first victims to get hit by an Agile team is the persona of Heroic Programmer. The type of programmer who by design or default likes to single handedly save everyone’s bacon. Agile focuses on the effort, communication and process of the ENTIRE team – not just the lone gun slinger. People who need to be the Heroic Programmer in a company often will not like Agile very much.

Another victim of the rising popularity of Agile are dev shops that operate on outdated principles and support dysfunctional processes. Through the emphasis on team empowerment I find more and more developers take up the battle cry of “Give me Agile or give me … [fill in the blank]”  A recruiter friend tells me about candidates who will not entertain an offer at all if it  isn’t  in an agile shop. Kudos to them. There is much to like about improved team communication and process enhancements.

Even if your own company cannot be 100%  formally Agile, with everyone buying into the change,  perhaps you can use some aspects of the methodology to improve your own processes. That has been my experience.

One of the key features of Agile (SCRUM) methodologies is the prevention of process hijacking. In  my 20 years of work experience I found two major types of development shops – those driven by engineers and those driven by sales people. Whichever group controls the process ends up setting the corporate culture. Agile shares responsibility between both groups and therefore prevents either group from hijacking the environment. That’s admirable.

Agile teams work hard to deliver predictable, high quality results in a relatively difficult setting. Difficult because software development all too often seems to go against a normal  predictive process. Unlike a Model A Ford, software cannot be built in an assembly line fashion. Hence the difficulty.

The mandate of Agile development is to create an environment where that uncertainty is removed through an iterative process and  much, much, much more communication than any other development methodology I have seen.  Meetings before things get built, meetings while things get built and more meetings after your done building things.

I have seen two interesting situations in all these meetings: programmers who by nature might be more introverted become much more open, and stakeholders who are used to issuing an edict and walking away become much closer to the development team. Both are good things.

There are a number of books on the market that describe Agile and SCRUM better than I can, but in a summarized form, Agile consists of

• A prioritized product backlog of themes, epics and user stories
• A self managing and empowered, cross functional team
• A SCRUM Leader / Master
• One or more product owners / stake holders
• Sprints that break user stories into tasks and produce a certain amount of work over a specified time, most frequently 10 business days.

Of course those are only the highlights. There are a number of important “ceremonies” and behavioral changes that come with the SCRUM territory. For example the practice of daily stand up meetings where team members briefly touch on what is on their plate.

In my own experience, the area where Agile makes the greatest impact is in the shared responsibility and empowerment of the team. By giving the team a way to estimate, manage and build code quickly and iteratively the group as a whole tends to be very productive.

During my time as VP of Server Development of SkillJam Inc. I was able to move a dev team toward a more agile approach by making sure a few key features were in place. First and foremost, we instituted a good product backlog.  I purchased enough licenses of Fogbugz to take care of all developers, QA People,  project managers and product stakeholders in the organization. I cannot tell you how important some of the seemingly simple features of Fogbugz became once we started cranking out code. Something as simple as an email conversation thread being automatically stored and associated with the task or issue in Fogbugz is a life saver. (Instead of logging into Fogbugz to record a comment, an email reply to a mail sent from Fogbugz automatically shows up in the right position within the task comments. Nice!)

Having set up FogBugz we got busy producing a big picture prioritized list of work. Any bug, any feature request and even entire new products where organized by themes , user stories and tasks within the stories.

After some arm wrestling with the key product stakeholders and other Senior Management Team members I was able to get buy-in for the concept of 10 business day sprints ( 2 calendar weeks). These sprints could not be interrupted for unplanned tasks under any circumstances. That’s not to say an emergency shouldn’t be addressed by the team.  But let me tell you,  its not easy convincing a CEO that giving up the ability to demand something RIGHT NOW is good for business. In the end it was the prospect of better schedule adherence, better quality and lower defect counts that sold this effort.

As I said, we were not formally Agile across the entire board. Our organization loved a considerable amount of design up front. That meant more information was written down and documented in formal requirements than you would typically see in an agile shop.

At the same time, our developers and qa analysts met several times a week for progress checks – admittedly this could have been handled with daily stand up meetings to which we could have invited visitors ( as long as the visitors kept quiet), but in a nod to the PMO we continued with the legacy status meetings.

As work progressed through a sprint we needed to look ahead at the pipeline ( the product backlog ) . So toward the end of a given sprint the team leaders and myself would spend some time planning the next iteration to make sure we could address any problems before we had our next Sprint is to kick off. For the actual kick off itself, the entire dev team (including qa people) gathered around a large conference table where we projected to prioritized backlog out of Fogbugz on a large screen. The team as a whole took on the User Stories needing to be built – based on prioritization – and committed to a certain amount of work being done in the sprint. Frequently we invited stakeholders and product owners to these meetings in order to get immediate feedback to questions that helped us with the production of a Sprint schedule.

That’s it. Simple and straight forward. I was blessed with a fantastic team who had been working together for a while already. That made SCRUM planning easier. And yes I know this wasn’t the  comprehensive SCRUM setup that you read about in the many Agile books on the market, but even in this pared-down format we were able to reap the benefits of planned iterative production.

Let’s start with some basics. Object Relational Modeling has been around for quite some time. The work by Dr. Raymond Chen has informed the design process of ORM systems for almost 30 years now. I posted 3 years ago that even Microsoft based its own decisions on the work of Dr. Chen.

Fast forward 3 years and it seems that many MS shops think that Business Objects, or ORM, in .NET 3.5 requires Linq.  Far from it. There are other, potentially more rewarding ways of doing ORM in .NET. Two prominent and mature alternatives on the market  are NHibernate and LLBLGen.

I should qualify one thing right up front: Linq tries to be more than an ORM tool. By definition Linq is “ a set of proprietary query operators that can be used to query, project and filter data in arrays, enumerable classes, XML (XLINQ), relational database, and third party data sources “

In short, Linq is more than a typical ORM mapper against a database. That is its strength and also potentially its downfall when compared to ORM mapper. While I personally think its great to use Linq to query things like Directory Structures or Strings, I also personally think that both NHibernate and LLBLGen are better at ORM – much the same way that SQL Server is better at managing data than say an XML document. Specifically, for example, Linq requires a developer to jump through some hoops when lazy loading an object graph. In addition the query tree expression generation code that Linq employes makes some odd decisions on occasion. That can lead to severely slow query performance.

As it stands, Linq is the 800Ib gorilla in the room because it originates with Microsoft. That means developers need to be able to work with it and the third party tool market has to cater to this by potentially adjusting its products to work with Linq.

Linq is extensible via a Provider pattern that implement the IQueryable<T> interface. A quick search of the Internet brought back examples of providers for CSV, OpenMapi and even Twitter. But writing a good Linq provider is no picnic at all. Many providers on the market today only cover a subset of the Linq spectrum. Writing a full-fledged Linq provider has taken one of the best programmers in the business almost a year and it is 1.2MB large. That should be a caveat for anyone contemplating that this is an easy task.

While Linq as a technology wants to be more than just an Object Relational Mapper for databases, both NHibernate and LLBLGen are exactly that and don’t try to be anything else. As a matter of fact, in the big scheme of things both tools are “closer to the metal” than Linq.  So much so that both tools offer a Linq provider that allows a developer to run Linq against NHibernate or Linq against LLBLGen. In that manner both tools broaden their market scope and build on the knowledge that developers have acquired when coming to the ORM space from MS centric projects. I am told by users who have tried the NHibernate Linq provider that it is still a bit immature at the time of this post. This just underscores the difficulty of writing a good one.

What then is the difference between NHibernate and LLBLGen? As far as I am concerned the differences fall into three large areas:   product philosophy, developer productivity and vendor quality / product maturity .

Product Philosophy:
NHibernate approaches the concept of building entities and mappers from the stand-point of designing business objects first, without any regard for databases or persistence structures at all.

LLBLGen, on the other hand, adheres closely to the work of Dr. Raymond Chen mentioned earlier. It builds entities on the basis of a database schema.

The tool of choice here really depends on how you approach the ORM space. Do you prefer to work with objects first and disregard persistence? For the time being NHibernate makes that easier than LLBLGen. I should point out that the next version of LLBLGen, due in a few months, has one incredible advantage over the competition – it will support Model-First development. By encompassing both design philosophies the LLBLGen designer tool – which by the way is the only fully functional designer tool in the ORM market – will cater to both camps. Essentially LLBLGen will allow you to work with NHibernate files, its own providers as well as Linq. Its a win win win situation.

Personally I have always worked with the db first approach.  I understand and sympathize with the object first approach. But the majority of my larger projects have existing databases. For instance I can’t very well insist that Chevron change its db structures, just to accommodate my ORM system.

LLBLGen makes work with existing data structures a breeze. Unlike NHibernate it supports multiple databases within one project, foreign key relations, prefetch paths and typed lists

As a side note, if you want to learn more about the object design first approach I recommend Jimmy Nilsson’s book “Applying Domain Driven Design Patterns”.  It’s a fantastic work on that subject.

Developer Productivity:
LLBLGen runs within a  separate development IDE (not within Visual Studio) and is by far the more productive tool. Turning a database into entities and using them in a project requires but a few clicks and option selections.

In NHibernate the developer is asked to hand code empty business objects (usually POCO’s) , then  manipulate an XML mapping file. I realize the typical argument here is to point out that this approach offers more control to the developer. Yes it does.  But the fine grained approach impacts productivity whenever changes need to made after the initial design. Especially with larger projects the need to work at a fine grained level can become cumbersome. After all the mapping files in NHibernate do not live in a vacuum.

A couple of years ago I had the pleasure of watching a longtime Hibernate architect become completely addicted to the productivity of LLBLGen. He literally was blown away by how much he could accomplish in very little time. In the end, the formerly dreaded change / update cycle became one of his biggest tools in being productive. My colleague took part in building one of the larger real estate loan processing engines in the US using Hibernate and Java – he knew those products inside out.

Vendor quality / Product maturity:
If you are faced with a tricky medical decision, whom would you trust more – a GP who deals with a variety of medical issues or a specialist that eats,  sleeps and breaths the specialty you need?  In other words if you can see the quality of the people whom you trust would that help in your decision process? It does in mine.

How smart are the people behind the tools you choose? In the case of LLBLGen and NHibernate I’d say the development is headed by a couple of programmers who are absolutely the best in the field (Ayende Rahien and Frans Bouma).  Contrast that to another ORM tool on the market – the MS Entity Framework and its problems over the past three years and you can see how essential the capabilities of the core devs in a tool like this really are.

Both LLBLGen and NHibernate are mature within the marketplace. NHibernate builds on its “big brother” Hibernate for Java.  While NHibernate is a failry mature open source project, LLBLGen is the most mature commercial tool of its kind in the .NET market.

Open Source projects are great in many ways – especially after they have settled down a little. But as a personal choice I will frequently pick a good commercial product over an open source one. There are several reasons for that. When your reputation is on the line with the projects you implement, its good to have a person who is directly responsible to you by the fact that you paid for their product. Granted with some vendors that’s a useless measuring stick but in a comparison between NHibernate and LLBLGen, I favor the latter because a very specific person is responsible for the performance of the product.

Overall there is a great sense of design and continuity in the product. I have used it since 2003 and I am here to tell you that in 6 years, through dozens of commercial projects for a raft of important customers – from Chevron to ITT Corp and Chrysler; from Lycos to Gameshow Network – accounting for several thousand man-hours of work it has never let me down.

Since this is not a product review I would urge you to visit the llblgen website and have a look. Features like visual inheritance definition, non-overwrite of custom code, support for all relationship types (1:1 ; 1:M ; M:1 and M:M) and much more can be reviewed there. And of course take a look at the NHibernate site as well. There is much to like in both tools. As it stands when someone asks me “Linq or NHibernate” I am very likely to answer “Neither my friend… but let me tell you about the tool that you should use ! ”

Update:  10-31-2009 – I just had to include Mohammed Meligy’s post because his experience parallels mine. Here is a link.

Overall I still wonder how much need there will be for MVC in the long run as opposed to Silverlight or even something like Adobe Flex. In other words will the web move more toward Rich Apps or will it move more toward jazzed up HTML apps. Fact is there are things you can do in MVC that are not possible to build quite as nicely and easily in regular ASP.NET. For example there is a plethora of CSS tools out there and freely available plugins and whatnot for jQuery that may work easier in MVC pages.

In order to render something similar to a user control in ASP.NET MVC one creates a partial view – which ends up with the extension of ascx.  This partial view can be embedded into and rendered through other pages (views) . That’s normally done via the RenderPartial method. It all works very well until you need to render a partial view that sits in another folder.Then for whatever reason the system starts to complain bitterly. Yes I know … your supposed to put it into the Shared folder. But somehow that doesnt sit well with my sense of organization. Why have one folder that has a bunch of partials from all over the place.  ( I have not tried adding sub folders to /Shared. Maybe thats an idea as well)

Seems to me that I should be able to create a partial view that has a table of data and then embed this partial view in a variety of other views- wherever I might need this list. And the places where I might need this list could be scattered all over the Views structure.

I am not the only one who ran into issues like this.  Here is a post by a fellow who had similar issues with entire pages.

An internet search turned up the following idea which may possibly do the job:

protected void Application_Start()
{
	var engine = (WebFormViewEngine) ViewEngines.Engines[0];
	engine.MasterLocationFormats = new string[] { "~/Views/{1}/{0}.master", "~/Views/Shared/{0}.master" };
	engine.ViewLocationFormats = new string[] { "~/Views/{1}/{0}.aspx", "~/Views/{1}/{0}.ascx", "~/Views/Shared/{0}.aspx", "~/Views/Shared/{0}.ascx" };
	engine.PartialViewLocationFormats = new string[] { "~/Views/{1}/Partial/{0}.aspx", "~/Views/{1}/Partial/{0}.ascx", "~/Views/Shared/Partial/{0}.aspx", "~/Views/Shared/Partial/{0}.ascx" };
}

Having spent some time with ASP.NET MVC – the revolutionary ” new ” direction of ASP.NET software development I have a few observations:

ASP.NET MVC is more than just the code necessary to make MVC compliant apps in C#. Seriously. It is a lot more. Just as the .NET System.Web namespace put a wrapper around HTML so does ASP.NET MVC.  Its not as deeply permeated as the regular Webform system but there is still an awful lot more to this implementation than simply the code needed to run an MVC app.

Here is the deal – in order to have a plain and simple MVC you don’t need all the stuff provided by ASP.NET MVC.  Basically you need a bit of dependency injection (just a few lines), an Action Framework and Routing Structure. That’s it.

Of course ASP.NET MVC has all that – and I must say the Action Framework is done very nicely – but at the same time it also contains a fullfledged attempt to substitute an entirely new paradigm for the existing webform system. Consequently you have all sorts of wacky stuff going on like Views that look a look like tag soup , partial views that are similar to user controls, a magical way of passing data back and forth among the models, views and controllers and number of other features that are intended to make the system more testable.

I have spent several days so far building an internal test app on the ASP.NET MVC framework and I am here to tell you this is no walk in the park. Considering this is a released product, it strikes me as ironic that there are only a handful sample apps out there which contain any sort of complexity.  There are a handful open source apps out there which range from absolutely pedestrian to entirely new generations of MVC that dont much resemble the MS release version (on the surface anyways)

In addition there seems to be a lot of flux in what is accepted as a good design and architecture. Several of the more prominent samples (Kobe, Oxite, MVC Commerce) are being rewritten based on extensive criticism by the OSS community. This begs the question who is the OSS community and how is it these folks have so much criticism of these products. I really hope its more than the handful of “usual suspects” in the .NET community. You know… the guys that like to congregate around the Alt.Net water cooler.

Here is my issue. When I am learning a new tool Ilook for definitive examples of good design and usage of that tool. And I just don’t see enough of those examples. Having 6 month old sample apps criticized as being incorrect architecture and in need of updates only adds to the frustration levels.

Consider how many millions of ASP.NET programmers there are in the world – heck the Indian sub continent alone can probably account for a few mill – the dearth of good ASP.NET MVC reference material is alarming. One would think that there would be more excitement around this stuff.  Of course I don’t have the same visibility into acceptance rates as some of the MS Regional directors. Its quite a dilemma I think. For almost 10 years MS web technology has gone one way ( web forms) and now here comes this entirely different way of doing it.  Like I said before, I am still waiting to see how ASP.NET 4.0 deals with routing (it still uses web forms). In the meantime I will keep plugging away at MVC.

One thing is for sure, in order for this Revolution to take hold there will need to be a much greater acceptance by the .NET community and that in turn requires more than a handful of books  and a dozen sample apps. In the end I am not even sure that the greater developer community at large will switch to MVC as opposed to Silverlight. Yes MVC will always have its audience , but I am thinking along the lines of the thousands of corporate developers. These guys would have a hard time with yet another paradigm shift and the reward seems to be so much greater in the area of Silverlight apps for these fellows.

I sure hope that in time my posts about this framework will become more positive.  I suppose that once I get past the frustrations there is a lot to like there.

I love the way MVC separates out the code that used to get thrown into the same code behind in web forms. It really shines in that regard. Who knows maybe it will have a positive influence on webforms apps in that regard.

For the moment my bottom line is that it takes a lot longer to write the same functionality in MVC than webforms. That’s partly due to the learning curve and partly due to the fact that there is more code to write. The fact that there is more code will be interesting to watch. Maybe in time people will find some efficient practices that will help to bring down the overall effort required. If thats not the case I wonder how someone would justify the cost of building a system one way versus another. And dont tell me to build it in MVC because webforms are evil. Thats nonesense. We have almost ten years of webform driven sites out there.

Lastly, for all those folks who are jumping on the “Viewstate is evil” bandwagon – which seems to be the same lemmings who jumped on Datasets as a DAO in 2002 – have a look at the following performance test. 6-8 requests per second for an MVC app is laughable. I sure hope that’s not the case presently. Its worth some internal testing to see. By the way, just on a purely subjective note, for the past year I have frequently spent time on MarketWatch. Since they switched to MVC I have noticed a small performance penalty. Purely subjective. Heck maybe its even a matter of browsers.