C#

Enterprise Integration with POCOs

Gerry Heidenreich — Wed, 27 Aug 2008 23:20:36 GMT

Our custom development here at Whyte Hirschboeck Dudek regularly requires integration with our enterprise systems, from Active Directory to Time & Billing, our custom Marketing Dashboard, Document Management System, Customer Relations Management, etc...

A few examples of data we may regularly need include:
- Contact information
- Member of a security or distribution group in Active Directory
- Current secretaries assigned to lawyers, or vice-versa
- Who is associated with what clients and projects

Motivation
A small disclaimer: we are lucky to be in a Microsoft house, with Visual Studio 2008, Active Directory, Exchange, (mostly) SQL 2000 & 2005, and some APIs to play with. Our systems are positioned to play nice with each other.

There are so many different infrastructure frameworks, paradigms, and architectures that it is overwhelming to try to keep up (just learning about them, never mind putting them into production!). Sure, service-enabling your enterprise with SOA seems to fit the bill through total abstraction, but the up-front costs are huge, and the ROI is uncertain at best.

As you probably know, POJO is a formalized ('Fowlerized") term for a 'plain old java object', and POCO is the CLR representation (PORO is Ruby, etc). It is a domain entity that carries no special baggage. It doesn't extend anything or carry any dependencies on other frameworks or methodologies, only the CLR.

When writing the application itself (a webpart or user control, clickonce app/utility, extranet, report, etc) 100% focus should be on the application's purpose and logic. I should never have to tell the application what a Client is, just that I need one. I shouldn't have to say that a client has matters, my client object should expose that for me! Also, I don't want to get my objects by writing up services, proxies, ORM metadata, and various providers unless it is absolutely necessary. I know these things are what's cool, and I need to grow up, but I like simplicity: I want my app to be as simple as possible to develop, and I want anything written outside the bounds of my app to be available as a single .dll file that describes my business entities so they can be reused later by other projects.

.Net provides some amazing capabilities for wiring an app together: Lambdas, LINQ, Extension methods, Generics, System.Converter<T1, T2>. It is much more capable than it used to be to work with objects quickly and easily on its own. (With this in mind, I am anxious to see what the Entity Framework does for us as far as value:complexity!)

The following explains our approach, starting with the fundamental bits in our framework, the core library...

Enterprise Core Library
The foundation of our enterprise integration efforts. The classes contain regularly needed functionality like:
- Logging (to a single EnterpriseLog table)
- Connection strings to various systems (SQL, Ldap, Service URLs)
- Email, Notifications
- Active Directory querying
- ClickOnce helpers (add to startup, autoupdate methods, etc)
- SQL access methods for reading and executing SqlCommand objects.
- List to RSS mapping

Our custom apps need a few keys in their .config, to access Active Directory, and to access our WhdEnterprise.ConnectionStrings table.

ConnectionStrings
Connecting to a system from our apps are trivial: ConnectionString.Retrieve("TimeBilling") grabs the appropriate string from the WhdEnterprise.ConnectionStrings table. When (not if) the Time & Billing system moves servers, update the row in the table instead of mass-updating config files for all broken apps that use it. A dictionary cache exists to keep performance from being an issue.

ConnectionStrings may be used for more than just databases... consider LDAP string for Active Directory, and URLs for web services.

User
One of our most fundamental objects is in our Core library: WhdUser.

WhdUser.Current uses Environment.UserName to identify the current user when needed, and optionally (lazily) grab email, phone extension, and computername from Active Directory.

Once the current user is identified, a domain-specific user can be instantiated (ex: List<Client> clients = TimeBillingUser(WhdUser.Current).Clients). Each system has its POCO user object, and controller to retrieve the relevant domain-specific info. Our time & billing system for instance, has a TimeBillingUser, which may have properties and methods like Secretaries, Clients, Projects, HoursWorked(), HoursBilled(), HoursCollected(), etc. This data is lazy-loaded when appropriate.

Organic, Evolutionary library growth
"We stress the importance of creating objects not to meet mythical future needs, but only under the demands of the moment. This ensures that a design contains only as much information as the designer has directly experienced, and avoids premature complexity." - Kent Beck & Ward Cunningham, A Laboratory For Teaching Object-Oriented Thinking

We extend our libraries only as needed, to meet current requirements. The caveat is that we must keep our libraries backwards-compatible. This can be accomplished easily by writing tests as the library is developed, and making sure those tests stay with the library project, and are run every time the project is extended.

For example, to get a list of our most active lawyers, we may write a library that contains objects for Timekeeper, and Timecard, then map these into our Time & Billing system that contains our timekeeper and financial (hours worked) data. At a later date, we may need a list of clients a lawyer does work for, so we would create a Client object, and map it into the appropriate table(s).

At this point, we have a single DLL (Whd.Enterprise.TimeBilling.dll) that is deployed with the new app. Other deployments of this dll used by other apps are older, but they were deployed with the requirements that app needed. With tests in place, those older apps should not have to worry about the new TimeBilling library being incompatible.

Controllers
This is a controversial idea, and my naming could be better (not to be confused with MVC controller), but it works very nicely to develop against (especially with Intellisense). Each enterprise object has 2 classes, the POCO (plain old CLR object), and its controller class (Client, and ClientController respectively). Controller classes are full of static methods to do the work.

One of the key methods in the controller is Read(), which does the object-relational mapping. The Read() method is delegated to the Core's Sql class to handle, and returns the appropriate POCO it's responsible for. Other methods are added as needed, and if the controller class ever gets too big or busy to work with, it can then be refactored out to partial class files.

Pragmatic App Development
From the app-developer's perspective, integrating with the enterprise should be as simple as possible:
- Add projects to my solution for each library I will be using, extend it as needed to fulfill my needs (or just reference the dll!)
- Reference the enterprise projects in my app's solution.
- Simplest possible namespace structure as possible. Ideally, "WHD.Enterprise." opens up all of the domain objects and controllers I would ever need.
- [Object] & [Object]Controller is the only enterprise-related pattern I need to understand.
- Centralized logging (Logging usage has proved its value already)

Hopefully this is a good start to explain my thought and dev processes. Don't hesitate to post any questions and/or criticism, I am sure that in future posts I will delve deeper into some of the bits, maybe even post some code.

.Net Open Source

Gerry Heidenreich — Thu, 17 Apr 2008 03:08:30 GMT

I finally had time tonight to scrape through Jeff Atwoods spreadsheet o' candidates for his Open Source .Net grant.

ScrewTurn Wiki got the $5k (+ $5k more matched by Microsoft),was a great choice... it is the only wiki I have ever considered putting into production. There were some awesome projects in the list (even nProf, but it's dead now, 404 and all), quite a few new ones I'd never heard of...

I've played with most of these before, just laying down single list of them on my site:

nHibernate - ORM framework that's been around forever

MEDIAPORTAL - Haven't played with this one at all, but it looks promising

The Thirsty Developer, Project Euler

Gerry Heidenreich — Fri, 11 Apr 2008 16:28:31 GMT

I can't believe that I didn't post about this! My site was acting up when we did this, so I probably didn't have a place to post it.

Larry Clarkin & Dave Bost from Microsoft have a podcast called The Thirsty Developer.

Back in December I met up with Larry, Dave, and Damon Payne at the Ale House in downtown Milwaukee to talk about Project Euler. I remember having a lot of fun, despite being a bit nervous (my first experience on that side of a podcast!). It was very loud in there as well; I think Larry did a great job of editing.

[Here is the Project Euler episode]

[The Thirsty Developer]

[Subscribe to The Thirsty Developer Podcast]

[Subscribe to The Thirsty Developer Blog]

Linq to everything

Gerry Heidenreich — Tue, 08 Apr 2008 22:48:05 GMT

Charlie Calvert has a list of Linq Providers here.

Amongst the ones listed were a few that popped out at me:

Linq to LDAP and Linq to Active Directory

Linq to C#

Linq to Geospacial Data

Update: From the looks of things, there is no Linq to Link yet. I believe that Damon Payne is working on this provider though. His work is clearly inspired by the NIN cover of Legend of Zelda theme song.

Community-Sourcing

Gerry Heidenreich — Wed, 02 Apr 2008 19:10:06 GMT

I have been a HUGE fan of Digg.com for years. The content is generally good, the community is fun and (again, generally) intelligent, but the model: Submit/Vote/Discuss/Report... brilliant.

MyStarbucksIdea.com, and Dell's IdeaStorm.com follow the Digg.com model, but in the context of innovation focused on a business. They are crowdsourcing their innovation to the world, and their future offerings are going to be more organic than ever before. The new tool on their belt gives them a clearer idea of their actual customers' wishes. Minimize the assumptions. Outsource your innovation to the one group the really cares about your product, and spend next to nothing for the data you get from it... brilliant.

Wikipedia (currently) defines Crowdsourcing as "... the act of taking a task traditionally performed by an employee or contractor, and outsourcing it to an undefined, generally large group of people, in the form of an open call."

I am in the habit of repurposing (or desigining my own) social-network ideas as internal solutions, and along the way I have occasionally had my share of failage/lesson-learnage, but I've also scored some wins. Like everybody else that thinks they have a new idea worthy of its own name, I have started calling it 'community sourcing'.

I reformed the definition above to fit my needs:

Community Sourcing
The act of taking a task traditionally performed by a specific member of the group, or consultant and exposing it to a controlled, generally large group of people who share the same interest as the group, in the form of an open call.

The term seems to be out there (google 2080 hits), and the purpose looks similar.

As far as walking-the-walk goes, we have working 'community sourced' systems used every day for content-management, marketing, and project management. Newer and (therefore, I hope) less-used solutions include link-tracking (think del.icio.us), and yes, a submit/vote/discuss/report app, which, in my humble opinion, is... brilliant.

EnterpriseLog, usage-logging

Gerry Heidenreich — Tue, 01 Apr 2008 23:03:50 GMT

Most development work contains some type of logging. Usually, it's to a local log file or an email, for exception tracking at the most.

Recently, I decided to throw together an EnterpriseLog table. We tossed our logging bits into a central library... I don't know why it took so long to centralize this particular functionality, but I'm glad we did it.

Our EnterpriseLog table is providing useful data, as well as helping us answer some good questions:
- What applications are people using?
- Tracking ClickOnce downloads, system updates made by clickonce apps, etc.
- What terms are people searching for?
- Exceptions
- Usage

Usage Logging
Usage logging is has been very interesting so far. An immediate benefit we have found is keeping track of how formatted information is being entered in unexpected ways. It is amazing how many different ways there are to enter a phone number! With a quick look at EnterpriseLog usage data for an application, we are able to recognize, learn, and accommodate actual usage patterns. Continuing with the phone # topic:

With any phone number, a quick regex levels the playing field by stripping out all non-digit characters:
string nums = Regex.Replace(textBox.Text, @"[^\d]", "");

If the result string is 10 chars (in the US), format appropriately for area code and exchange. If it is 7 characters, try to infer the area code from the exchange. If it is only 4 characters, assume it is an extension and build the real number from there if possible.

In this case, usage logging helped me to skip the annoying validation and instructions and let my users do what they do. We have been able to identify actual usage, throw assumptions out the window, and help them arrive at the appropriate result invisibly:
textBox.Text = String.Format("({0}) {1}-{2}", areaCode, exch, num);

Most of us base our UI development on assumptions about our users, past experience, and hopefully a book or two. It is the assumptions part I have trouble with. I'm a data guy. I prefer to question the assumptions, set up some auditing and collect some data, and turn the assumptions into concrete UI features that keep the user in the flow.

Salesperanto AND Coderian?

Gerry Heidenreich — Fri, 14 Mar 2008 16:03:47 GMT

"Stop Thinking Like A Programmer"

I swear it's the theme this week.. I was actually told this by someone at work. In my own defense, I was thinking like somebody that would rather script a solution than wait 3 weeks for it (ok yeah, that's like a programmer)....

Think about this...

1. Business: Coders should be able to think in terms of features, interface complexity, barrier to entry, design and visualization, and capable of elevator pitching their product (notice I didn't say solution?) to a customer in these terms.

2. Innovation: getting the "I can do that" people (e.g. your engineers/architects/coders) to be able to speak directly to the "it would be cool if..." people (e.g. your billers, customers, parents, etc)

3. Apple's innovation/momentum and Microsoft's shift in perspective & ability to compete: Microsoft has always been guilty of "thinking like programmers", and it has been very profitable for them, but things are changing, and they are reacting accordingly.

4. Black & White: on one side are the geeks that appreciate your architecture and could debate code/frameworks/paradigms all day. On the other side are your customers, who want to know how you are going to make them more profitable/efficient/confident/marketable/competitive. Not much of a grey area here. 2 different languages: Salesperanto AND Coderian.

5. Intentional Programming [wikipedia]: Your skillset is in demand, but we are getting closer to the day that "Everyone Writes Software"... Lutz has a section dedicated to this. Developers must learn to understand the intent of their users. Stop thinking in syntax, start thinking in semantics... Mashups, FDD, REST, RDF, Pipes & Popfly (Google SETS prediction).

There is a pattern developing here, and there is A LOT of money being tossed around because of it (check this out!).

Self-installing Windows Services

Gerry Heidenreich — Tue, 27 Nov 2007 14:21:00 GMT

I recently did some Windows Service work (pluggable enterprise-wide notification service, very cool... totally worth it's own article someday when I have time), and I'm thinking that the service installage/uninstallage is squirrelly... I made some batch files to do it for me and never thought about it again. W. Kevin Hazzard reflected installutil.exe, and integrated (un)install right into his service.

He has a good article on CodeProject to make Windows Service installing and uninstalling easier:
http://www.codeproject.com/dotnet/WinSvcSelfInstaller.asp

.Net now "Shared Source" NOT Open Source

Gerry Heidenreich — Wed, 03 Oct 2007 16:49:00 GMT

Scott Guthrie announced on his blog, a few hours ago, that .Net source code will be opened up to the public, under a ms-rl license (for reference, read-only).

Scott's announcement, already flooded with lots of comments & trackbacks, mostly positive, is here.

Starting with Visual Studio 2008 (Orcas), currently set to be released later this year, we will be able to reference the internal state of .Net objects as if they were local.

This means a few things:
1. F11 will step you into the actual .Net object being called, where you can reference in-state .Net classes.
2. You will see real objects, variables, line numbers in your call stack for the .Net classes being referenced.
3. (I assume) you will be able to “Go to definition“ and view actual .Net class source code instead of an interface. Of course, we've always had the ability to reflect on the libraries, and with a little work, figure out what was happening... but this will make things a lot more simple and accessible (have a look at Reflector to figure out the Asp.Net treeview control!)
4. WWBAD (What would Brad Abrams do?) Now we can see for real instead of reverse engineer it and spend our time figuring out what the variable datetime17 is doing. In other words, quality of code should improve. As we constantly reference the .Net library, some of the msft QA, v3.5 juju should rub off on us and help us fall a little more in line with standards, best practice, etc.

Anyway, it's read-only. It's not Open source. But it's another big step in what I think is the right direction for Microsoft...

Some good MS pages to check out on open/shared source initiatives:
- Open Source at Microsoft
- Microsoft Shared Source Initiative
- MS Developer Tools (A goldmine of open/shared source projects, most on CodePlex)

Euler12 (spoiler warning) - GetDivisors

Gerry Heidenreich — Thu, 03 May 2007 18:19:00 GMT

Thanks to Mr. Payne, I've been spending more hours awake coding, and less time sleeping, in the name of 'fun and recreation'...

As Damon points out in his esoteric and colorful post yesterday, I've been working my way through the Euler (prounounced 'oy ler!) puzzles at Project Euler. I've banged out solutions for puzzles 1-13, but #10 is the only one I can't get under 1 minute (In fact, it's the only one that isn't solved almost instantly).

Problem 12 is interesting: Which is the first triangle number to have over 500 divisors?

Finding triangle numbers is trivial; the actual challenge in this puzzle is efficiently counting divisors for increasingly huge numbers.

The initial approach that almost works:
For number n, iterate from 1 to n, increment divisorCount where i % n == 0
(Once you try this out, you will find that with this approach, your highest divisor count will quickly reach 320 divisors, and will ultimately hang at around 480 before it starts to crawl...)

Finding the first triangle number to have over 1000 divisors, in under 10 seconds:
When things get even slightly complex, I turn to pen & paper... within minutes I saw the pattern I was looking for. The following method will return a list of divisors for an int:

static List<int> GetDivisors(int num) {
   List<int> divisors = new List<int>();
   int divisorCandidate = 1;
   int multiple = 1;
   do {
      if (num % divisorCandidate == 0) {
         multiple = num / divisorCandidate;
         divisors.Add(divisorCandidate);
         //for squares, divisorCandidate == multiple (3*3=9 etc), so only count it once
         if (divisorCandidate < multiple) divisors.Add(multiple); 
      }
      divisorCandidate++;
   } while (divisorCandidate < multiple);
   divisors.Sort();
   return divisors;
}

I doubt it's even remotely useful in production, but this method is a great example of how a little analysis of the problem (especially away from the keyboard) can result in a much more elegant solution to your problem than brute force.