Coding

Enterprise Integration with POCOs

Gerry Heidenreich — Wed, 27 Aug 2008 23:20:36 GMT

Our custom development here at Whyte Hirschboeck Dudek regularly requires integration with our enterprise systems, from Active Directory to Time & Billing, our custom Marketing Dashboard, Document Management System, Customer Relations Management, etc...

A few examples of data we may regularly need include:
- Contact information
- Member of a security or distribution group in Active Directory
- Current secretaries assigned to lawyers, or vice-versa
- Who is associated with what clients and projects

Motivation
A small disclaimer: we are lucky to be in a Microsoft house, with Visual Studio 2008, Active Directory, Exchange, (mostly) SQL 2000 & 2005, and some APIs to play with. Our systems are positioned to play nice with each other.

There are so many different infrastructure frameworks, paradigms, and architectures that it is overwhelming to try to keep up (just learning about them, never mind putting them into production!). Sure, service-enabling your enterprise with SOA seems to fit the bill through total abstraction, but the up-front costs are huge, and the ROI is uncertain at best.

As you probably know, POJO is a formalized ('Fowlerized") term for a 'plain old java object', and POCO is the CLR representation (PORO is Ruby, etc). It is a domain entity that carries no special baggage. It doesn't extend anything or carry any dependencies on other frameworks or methodologies, only the CLR.

When writing the application itself (a webpart or user control, clickonce app/utility, extranet, report, etc) 100% focus should be on the application's purpose and logic. I should never have to tell the application what a Client is, just that I need one. I shouldn't have to say that a client has matters, my client object should expose that for me! Also, I don't want to get my objects by writing up services, proxies, ORM metadata, and various providers unless it is absolutely necessary. I know these things are what's cool, and I need to grow up, but I like simplicity: I want my app to be as simple as possible to develop, and I want anything written outside the bounds of my app to be available as a single .dll file that describes my business entities so they can be reused later by other projects.

.Net provides some amazing capabilities for wiring an app together: Lambdas, LINQ, Extension methods, Generics, System.Converter<T1, T2>. It is much more capable than it used to be to work with objects quickly and easily on its own. (With this in mind, I am anxious to see what the Entity Framework does for us as far as value:complexity!)

The following explains our approach, starting with the fundamental bits in our framework, the core library...

Enterprise Core Library
The foundation of our enterprise integration efforts. The classes contain regularly needed functionality like:
- Logging (to a single EnterpriseLog table)
- Connection strings to various systems (SQL, Ldap, Service URLs)
- Email, Notifications
- Active Directory querying
- ClickOnce helpers (add to startup, autoupdate methods, etc)
- SQL access methods for reading and executing SqlCommand objects.
- List to RSS mapping

Our custom apps need a few keys in their .config, to access Active Directory, and to access our WhdEnterprise.ConnectionStrings table.

ConnectionStrings
Connecting to a system from our apps are trivial: ConnectionString.Retrieve("TimeBilling") grabs the appropriate string from the WhdEnterprise.ConnectionStrings table. When (not if) the Time & Billing system moves servers, update the row in the table instead of mass-updating config files for all broken apps that use it. A dictionary cache exists to keep performance from being an issue.

ConnectionStrings may be used for more than just databases... consider LDAP string for Active Directory, and URLs for web services.

User
One of our most fundamental objects is in our Core library: WhdUser.

WhdUser.Current uses Environment.UserName to identify the current user when needed, and optionally (lazily) grab email, phone extension, and computername from Active Directory.

Once the current user is identified, a domain-specific user can be instantiated (ex: List<Client> clients = TimeBillingUser(WhdUser.Current).Clients). Each system has its POCO user object, and controller to retrieve the relevant domain-specific info. Our time & billing system for instance, has a TimeBillingUser, which may have properties and methods like Secretaries, Clients, Projects, HoursWorked(), HoursBilled(), HoursCollected(), etc. This data is lazy-loaded when appropriate.

Organic, Evolutionary library growth
"We stress the importance of creating objects not to meet mythical future needs, but only under the demands of the moment. This ensures that a design contains only as much information as the designer has directly experienced, and avoids premature complexity." - Kent Beck & Ward Cunningham, A Laboratory For Teaching Object-Oriented Thinking

We extend our libraries only as needed, to meet current requirements. The caveat is that we must keep our libraries backwards-compatible. This can be accomplished easily by writing tests as the library is developed, and making sure those tests stay with the library project, and are run every time the project is extended.

For example, to get a list of our most active lawyers, we may write a library that contains objects for Timekeeper, and Timecard, then map these into our Time & Billing system that contains our timekeeper and financial (hours worked) data. At a later date, we may need a list of clients a lawyer does work for, so we would create a Client object, and map it into the appropriate table(s).

At this point, we have a single DLL (Whd.Enterprise.TimeBilling.dll) that is deployed with the new app. Other deployments of this dll used by other apps are older, but they were deployed with the requirements that app needed. With tests in place, those older apps should not have to worry about the new TimeBilling library being incompatible.

Controllers
This is a controversial idea, and my naming could be better (not to be confused with MVC controller), but it works very nicely to develop against (especially with Intellisense). Each enterprise object has 2 classes, the POCO (plain old CLR object), and its controller class (Client, and ClientController respectively). Controller classes are full of static methods to do the work.

One of the key methods in the controller is Read(), which does the object-relational mapping. The Read() method is delegated to the Core's Sql class to handle, and returns the appropriate POCO it's responsible for. Other methods are added as needed, and if the controller class ever gets too big or busy to work with, it can then be refactored out to partial class files.

Pragmatic App Development
From the app-developer's perspective, integrating with the enterprise should be as simple as possible:
- Add projects to my solution for each library I will be using, extend it as needed to fulfill my needs (or just reference the dll!)
- Reference the enterprise projects in my app's solution.
- Simplest possible namespace structure as possible. Ideally, "WHD.Enterprise." opens up all of the domain objects and controllers I would ever need.
- [Object] & [Object]Controller is the only enterprise-related pattern I need to understand.
- Centralized logging (Logging usage has proved its value already)

Hopefully this is a good start to explain my thought and dev processes. Don't hesitate to post any questions and/or criticism, I am sure that in future posts I will delve deeper into some of the bits, maybe even post some code.

GiveCamps: Geeks Giving Back

Gerry Heidenreich — Fri, 08 Aug 2008 19:24:59 GMT

Overview: A Give Camp is a gathering of professionals (developers, graphic designers, database ninjas) that volunteer their time and resources to design and implement solutions (websites, applications, content-management systems, etc) for various charities and non-profit groups. Proposals can be submitted by non-profit groups and charities before the camp, and they are reviewed and selected, and teams of volunteers are organized to represent a project, create a plan and execute it, all over the period of a single weekend.

Projects / Features:
Web sites, new sections
Access Databases
Mobility Projects
New Content Management System (Graftiti, DotNetNuke, Sharepoint)
Intranet
Membership Tracking App
Paypal integration
Hosting
Training
Social Media Presence & Advocacy

Volunteer Qualifications:
Most importantly: A desire to contribute
Development experience (any level of expertise)
Sharepoint
Database Administrators
Developers! (.Net, Java, C++, PHP, Excel, Html)
Flash Developers & Designers
Photography
Social Media

Other GiveCamps:
Upcoming:
- Indianapolis, Indiana, January 23-25, 2009
- Milwaukee, WI, TBD (Nov-Dec, 2008?), Scott's announcement
- Northwest Arkansas .Net UG, TBD, Jay's announcement

Past:
- Dallas, Texas, January 18-20, 2008
- Kansas City, April 25-27, 2008
- Ann Arbor, Michigan, July 11-13, 2008

Important URLs:
- Givecamp.org
- We Are Microsoft: 18 projects for charities (brief case studies! - click links to see project details)
- Jennifer Marsman's review of the Ann Arbor Give Camp
- Twitter chatter "givecamp"
- GiveCamp Facebook Group

People:
- Chris Koenig, Dev Evangelist @ Microsoft, Founder of first GiveCamp
- Jennifer Marsman, Dev Evangelist @ Microsoft
- Dave Bost, Dev Evangelist @ Microsoft
- Larry Clarkin, Architect Evangelist @ Microsoft
- Scott Isaacs, President of WI .Net Users Group, leading up GiveCamp MKE
- Paul Hacker, leading up Indianapolis GiveCamp
- Jay Smith, President, Northwest Arkansas .Net User's Group

Accomplishments to date:
- GiveCamp #1: "We Are Microsoft" 18 projects from the first GiveCamp in Dallas, TX
- GiveCamp #2: "Coders For Charities", Kansas City, Missouri (overview)
- - Boy Scouts Troop 813
- - Berean Bible Church
- - Task Force Omega of MO
- - Missouri Pit Bull Rescue
- - Metropolitan Org to Counter Sexual Assault
- GiveCamp #3: Ann Arbor, Michigan (15 projects, charity names only, no details found)

Getting Started:
The GiveCamp cookbook

Media Coverage, spreading the word:
- Local news (Kansas City Fox 4)
- Press Release (Kansas City)
- Radio (Kansas City, Dick Dale Morning Show)
- Twitter
- Blogs
- Camp homepages
- GiveCamp.org

This post will be updated as I find more info for each section, and I will have a series of posts on our Milwaukee GiveCamp as it comes together. I am sure I have missed people and news, but this should serve as a good start to encapsulate the GiveCamp movement as it takes off.

Depot, an exercise in Community-Sourcing

Gerry Heidenreich — Sun, 08 Jun 2008 04:43:59 GMT

No downloads or pics, just a quick rundown of a very cool app idea while it's in my head.

About a year and a half ago, I wrote a small winforms app. It's stayed very rough around the edges and hasn't gone anywhere from the original prototype. This prototype (I called it Depot) was written as a proof-of-concept of the simplest possible community-sourced bookmarking / tagging / searching tool that could possibly exist. A self-organizing business-specific link / text library could provide immense value to a company.

Depot hinges on 4 basic features common with collaborative apps:

1. Producing: Adding content in the form of URLS and/or text (2 different fields that can be used individually or combined)

2. Tagging

3. Searching for any item by any combination of title words or tag

4. Sharing: All content is automatically shared, and open to edit & extend, by anyone within the network

The search is an autocomplete textbox, that works with any combination of title words and tags. Typing 'catering' displays all catering items, but as you start to type 'catering madison', the suggestions filter appropriately. As you would expect, changing the text over to 'thai madison' updates to items tagged or titled with thai and madison.

The url + text fields is an interesting feature - a user may want to toss in a quick note for a catering url someone else added, like "Beware the red curry!!!".

The app seemed to work beautifully, but the algorithm is not built to scale up yet. Everything is cached heavily on the client-side. There are no concurrency checks. Also, to be fit for production, it will need some kind of user-auditing, history, and probably some kind of browser integration (or at least bookmark / favorites sync).

I don't know yet what will become of Depot. I hope to find the time and motivation soon to dust it off and start polishing it up for a pilot group. If nothing else, I got an ornery hog of a tag-search algorithm that may come in useful someday.

Stackoverflow.com = joelonsoftware + codinghorror

Gerry Heidenreich — Thu, 17 Apr 2008 03:37:58 GMT

Jeff Atwood has paired up with Joel Spolsky and today they both announced stackoverflow.com, a new community support site that doesn't exist yet... but will be run by Jeff, created by the community for the community, and narrated weekly via podcast. Looks like a digg-like structure where diggable items are answers to questions posted by the community. Submit questions, submit answers, vote for answers...

[Jeff's announcement]

[Joel's announcement]

[stackoverflow.com]

[podcast #1]

This will be interesting...

The Thirsty Developer, Project Euler

Gerry Heidenreich — Fri, 11 Apr 2008 16:28:31 GMT

I can't believe that I didn't post about this! My site was acting up when we did this, so I probably didn't have a place to post it.

Larry Clarkin & Dave Bost from Microsoft have a podcast called The Thirsty Developer.

Back in December I met up with Larry, Dave, and Damon Payne at the Ale House in downtown Milwaukee to talk about Project Euler. I remember having a lot of fun, despite being a bit nervous (my first experience on that side of a podcast!). It was very loud in there as well; I think Larry did a great job of editing.

[Here is the Project Euler episode]

[The Thirsty Developer]

[Subscribe to The Thirsty Developer Podcast]

[Subscribe to The Thirsty Developer Blog]

EnterpriseLog, usage-logging

Gerry Heidenreich — Tue, 01 Apr 2008 23:03:50 GMT

Most development work contains some type of logging. Usually, it's to a local log file or an email, for exception tracking at the most.

Recently, I decided to throw together an EnterpriseLog table. We tossed our logging bits into a central library... I don't know why it took so long to centralize this particular functionality, but I'm glad we did it.

Our EnterpriseLog table is providing useful data, as well as helping us answer some good questions:
- What applications are people using?
- Tracking ClickOnce downloads, system updates made by clickonce apps, etc.
- What terms are people searching for?
- Exceptions
- Usage

Usage Logging
Usage logging is has been very interesting so far. An immediate benefit we have found is keeping track of how formatted information is being entered in unexpected ways. It is amazing how many different ways there are to enter a phone number! With a quick look at EnterpriseLog usage data for an application, we are able to recognize, learn, and accommodate actual usage patterns. Continuing with the phone # topic:

With any phone number, a quick regex levels the playing field by stripping out all non-digit characters:
string nums = Regex.Replace(textBox.Text, @"[^\d]", "");

If the result string is 10 chars (in the US), format appropriately for area code and exchange. If it is 7 characters, try to infer the area code from the exchange. If it is only 4 characters, assume it is an extension and build the real number from there if possible.

In this case, usage logging helped me to skip the annoying validation and instructions and let my users do what they do. We have been able to identify actual usage, throw assumptions out the window, and help them arrive at the appropriate result invisibly:
textBox.Text = String.Format("({0}) {1}-{2}", areaCode, exch, num);

Most of us base our UI development on assumptions about our users, past experience, and hopefully a book or two. It is the assumptions part I have trouble with. I'm a data guy. I prefer to question the assumptions, set up some auditing and collect some data, and turn the assumptions into concrete UI features that keep the user in the flow.

Salesperanto AND Coderian?

Gerry Heidenreich — Fri, 14 Mar 2008 16:03:47 GMT

"Stop Thinking Like A Programmer"

I swear it's the theme this week.. I was actually told this by someone at work. In my own defense, I was thinking like somebody that would rather script a solution than wait 3 weeks for it (ok yeah, that's like a programmer)....

Think about this...

1. Business: Coders should be able to think in terms of features, interface complexity, barrier to entry, design and visualization, and capable of elevator pitching their product (notice I didn't say solution?) to a customer in these terms.

2. Innovation: getting the "I can do that" people (e.g. your engineers/architects/coders) to be able to speak directly to the "it would be cool if..." people (e.g. your billers, customers, parents, etc)

3. Apple's innovation/momentum and Microsoft's shift in perspective & ability to compete: Microsoft has always been guilty of "thinking like programmers", and it has been very profitable for them, but things are changing, and they are reacting accordingly.

4. Black & White: on one side are the geeks that appreciate your architecture and could debate code/frameworks/paradigms all day. On the other side are your customers, who want to know how you are going to make them more profitable/efficient/confident/marketable/competitive. Not much of a grey area here. 2 different languages: Salesperanto AND Coderian.

5. Intentional Programming [wikipedia]: Your skillset is in demand, but we are getting closer to the day that "Everyone Writes Software"... Lutz has a section dedicated to this. Developers must learn to understand the intent of their users. Stop thinking in syntax, start thinking in semantics... Mashups, FDD, REST, RDF, Pipes & Popfly (Google SETS prediction).

There is a pattern developing here, and there is A LOT of money being tossed around because of it (check this out!).

.Net now "Shared Source" NOT Open Source

Gerry Heidenreich — Wed, 03 Oct 2007 16:49:00 GMT

Scott Guthrie announced on his blog, a few hours ago, that .Net source code will be opened up to the public, under a ms-rl license (for reference, read-only).

Scott's announcement, already flooded with lots of comments & trackbacks, mostly positive, is here.

Starting with Visual Studio 2008 (Orcas), currently set to be released later this year, we will be able to reference the internal state of .Net objects as if they were local.

This means a few things:
1. F11 will step you into the actual .Net object being called, where you can reference in-state .Net classes.
2. You will see real objects, variables, line numbers in your call stack for the .Net classes being referenced.
3. (I assume) you will be able to “Go to definition“ and view actual .Net class source code instead of an interface. Of course, we've always had the ability to reflect on the libraries, and with a little work, figure out what was happening... but this will make things a lot more simple and accessible (have a look at Reflector to figure out the Asp.Net treeview control!)
4. WWBAD (What would Brad Abrams do?) Now we can see for real instead of reverse engineer it and spend our time figuring out what the variable datetime17 is doing. In other words, quality of code should improve. As we constantly reference the .Net library, some of the msft QA, v3.5 juju should rub off on us and help us fall a little more in line with standards, best practice, etc.

Anyway, it's read-only. It's not Open source. But it's another big step in what I think is the right direction for Microsoft...

Some good MS pages to check out on open/shared source initiatives:
- Open Source at Microsoft
- Microsoft Shared Source Initiative
- MS Developer Tools (A goldmine of open/shared source projects, most on CodePlex)

Profile your ASP.Net 2 apps with nProf

Gerry Heidenreich — Fri, 14 Sep 2007 15:15:00 GMT

I'm going to make this short and sweet - there's not much [non-Polish (?!) ] documentation out there for nProf, so here are my notes for getting it to hook into your ASP.Net 2 app and show you some bottleneckage:

Preparation:

1. Set processModel username:
Open your machine.config file (C:\WINDOWS\Microsoft.NET\Framework\v2.0.50727\CONFIG\machine.config), hit Ctrl+I (incremental search) and type processModel, hit F3 (~2 times) until you find the <processModel...> tag. Add the userName attribute within it: userName="SYSTEM"

2. Make sure your app is hosted in and accessible via IIS (nProf won't hook into Cassini) - ex: http://localhost/test

Install nProf:

There are a bunch of versions available, and they're all alphas... The one that worked for me is here (nprof-0.9.1-setup.exe on sourceforge).
Install the .exe, and when prompted go ahead and run nProf.

Profiling:

Note: DO NOT stop profiling via nProf. Instead, recycle your app pool, or restart IIS inside your IIS manager. nProf will see it's recycled and stop for you. If you use nProf's stop buttons, it will hang and you'll lose your profile data.

File > New, Create Profiler Project dialog opens, select ASP.NET radio button, click Create Project.

Click Project > Start Project Run (or just hit F5)

Open browser, browse to IIS-hosted asp.net app (use IIS address, ex, http://localhost/nProfTest). Browse through pages that are slow, or save separate profiles for individual pages. You will see nProf's Messages window start rolling with information.

Once again, don't click 'Stop Run' in nProf! Instead, go to IIS Manager, and either recycle your app pool, or restart IIS. This will properly stop nProf, and you will see your resulting profile data.

Analysis:

Back in nProf, you will see the threads called by your aspnet_wp.exe process, and when you select them, you will see the used namespace hierarchy below. Find the thread with your library profile. Uncheck the parent namespaces and re-check your own. On the right, you will see the method signatures called, number of times called, % time taken, and more. Click the '% of Total' header to sort, and you will see your bottleneck methods at the top.

Note: Saving & Opening is kludgy: You can try to save, but I've had no luck re-opening my profiles... instead I took a screenshot of my trouble methods just for reference.

Intelligent Image Resizing - Amazing video demo!

Gerry Heidenreich — Thu, 23 Aug 2007 13:10:00 GMT

Via Digg > OhGizmo!, the video & article is here: Smart Image Resizing Cuts The Useless Out Of Your Pics

I'll check out the 20mb (pdf) whitepaper sooner or later, but here's what's going on as far as I can tell...

Technique:
This engine is resizing images by removing paths of pixels vertically or horizontally instead of columns of pixels. The path they add/remove is determined by interestingness. They refer to this technique as “retargeting“ (as opposed to resizing or resampling).

Identifying Image Interestingness:
They are determining the interestingness (or in their terms: gradient magnitude) by starting with a grayscale version of the source image, then creating a new image using The Sobel Operator on the source image. The resulting image represents large gradient activity (interestingness.) with light, and boringness... with dark.

Original grayscale, and the Sobel gradient image of the same picture:

Discovering the least-interesting path:
The engine then uses an 'importance or energy function' to discover the least-interesting path, composed of 1px from each column or row. This function seems to represent the most consistently shaded 1-pixel path from one end of the image to the other...

“Retargeting” the image:
Once the least-interesting path is discovered, it is then either removed or copied, depending on if you are “retargeting” your image to make it larger or smaller.

Maybe ImageManipulator needs some Retarget methods... now if only optionsscalper could help me work through this math ;)

Here's the video: