Bad Data – The Black Mold of the Industry


By Jane Stabile, President IMP Consulting

I’ll admit it, I am addicted to HGTV.   In particular, I love the shows about renovation.  You can buzz in and out of the room folding laundry or unloading groceries, and just really be in front of the TV for the “big reveal,” which is when the renovation is done and the homeowners get to see the results.  My favorite show, “Love it or List It” has homeowners choosing to stay in their homes, post-renovation (“love it”) or sell their homes (“list it”) for something better that they have found.

No contingency:  The homeowners typically have an older home, and they set a firm budget for the renovation, which they cannot possibly go beyond.  The designer then begins the plan, with glowing descriptions of how she will achieve everything on the owners’ “must have” list, and stay within the budget.  After the second commercial break, the inevitable bad news is revealed:  there is a heretofore unknown problem with the house that needs immediate repair, and anywhere from 20-50% of the reno budget is gobbled up by the fix.

The most recent “hidden” problem was black mold.  Although the homeowners and the designer both remarked on the strong musty smells in the basement, no one took the time to investigate the issue prior to the grand plans for the redesign.  When Eddie (the construction lead) opened up the walls, cracks in the foundation revealed a long-term water problem that was feeding the black mold.  The fix was expensive and included re-grading the earth around the foundation, waterproofing, and professional mold removal.

The next scene was the inevitable drama of the designer cutting back on her deliverables, the homeowners storming about in disappointment and questioning the competence of the designer, and the designer complaining to the camera about the homeowners’ unreasonableness.  My favorite moment was when she began venting her own helplessness and frustration on Eddie and he replied, “Well, what d’ya think was goin’ on?  Couldn’t you smell the mustiness?”  Chalk one up for Eddie.

Almost every show goes the same way, and there is never a contingency budget.  Plans are made and work begins before any real analysis is done of the possible pitfalls and constraints of the current structure.  Leaky basements, old, clogged plumbing, bad wiring, etc. all conspire to drive up costs, erode the budget and stretch out timelines.

Ok, so what does this have to do with asset management?  Some time ago, I was working with a client who was planning to implement a new compliance system.  The more sophisticated engine could automate a large number of tests that the compliance team were manually calculating and monitoring, such as complex benchmark tests, as well as a portion of the derivative tests.  We walked through the pros and cons of the conversion, with much of the positive energy coming from the excitement over a much-improved future state.

The next person to present to the group was the manager in charge of data at the firm—I’ll call him “Eddie.”  According to Eddie, the data samples for derivatives that the project team had requested were not available via any feeds; most of what we had asked for was either being entered manually into spreadsheets or was unavailable in the data store anywhere.  Further, some of the fixed income analytic data was coming in from multiple places, and there was no data dictionary.  Therefore, there was no easy way to trace the source of data or confirm which data source was used for any given data field in the warehouse.

Eddie recommended that we launch a data project immediately to address the problem and recommended that the data be brought up to 100% accuracy (or close to it) prior to launching any conversion.  He also recommended the establishment of a data dictionary that would show the source for any given field and any transformation performed upon it (e.g. using the inverse of an FX rate or using a price from a secondary source if the preferred source was unavailable).  That way, he reasoned, we would have a solid foundation for the new system.

After a moment of stony silence, the senior manager in the room thanked Eddie for his time, but stated firmly that there was no additional budget for a “side” project, and we would proceed as planned.

At the risk of stating the obvious, the data problems did not go away on their own.   Instead, they had the same impact as the black mold, contaminating everything and finally driving the senior management team to approve additional budget and stretch out the timelines for the compliance implementation, delaying the fixed income desks and putting the derivatives functionality dead last.

Yes, we all smelled the mustiness, and we eventually had to fix the foundational issues.