For me, the event is one of the highlights of the Information
Management calendar and this year’s Congress featured two days of conference
proceedings, together with an additional day of in-depth workshops (including
my own half-day session on Managing for Effective Data Governance, presentation materials for which are available here.)
Participants in the conference came from commercial, public and
not-for-profit sectors and featured contributions included: Melbourne Water,
Telstra, National Australia Bank, Australia Post, Insurance Australia Group,
ANZICS, Mater Healthcare Services, Australian Bureau of Statistics. Contributions
for the vendor and service provider community included presentations from ExperienceMatters, Accenture and BigData-Startups.com.
Telstra: Winners of the DQAPAC Enterprise Award for 2014
There was so much to take in throughout the two days that it’s
pretty tough to try and summarise everything! (I will probably revisit some
topics in more depth in future posts…) But for now, here are my “top 10”
1. Data Quality is boring!
Or at least, it can be perceived to be boring by people who aren’t
actively engaged. Passionate people are needed if the data is doing to be cared
for, so you need to engage in a manner that is relevant to the business
community, not concentrate on the practitioner processes. Use language and
examples that are business-oriented. Make it fun!
Don’t repeat the failures of before. Projects should incorporate a
“Long Hard Look” exercise at the start of a project phase as part of project
initiation and planning, not at the end of a project (when the lessons will
quickly get forgotten).
3. Plan for change, and
expect to change your plans.
Having a good plan will enable you to adapt later. But don’t
adhere rigidly to the plan, because reality never turns out the way you though
it might. Be flexible, be adaptable, be responsive, and be available.
4. Identity Management and
The concepts of “Party” and “Roles” are critical to enabling a
single consolidated understanding and unique identification of each “customer.”
At some point, an individual could play any number of roles in interacting with
the organisation (e.g. customer, member of staff, supplier, broker, agent,
benefactor, consultant…. IAG currently recognise 37 valid roles that a Party
may play.) The concepts of Party and Role require constant ongoing education to
5. Customer-Centric =
To enable a customer-centric approach, an information-centric
enterprise view is foundational; it enables both cross-application integration
and analytic insight. System- and Process-centric architectures drive
compromises (because of silo thinking being built-in by design). As a result,
data quality suffers. We need to have an information-centric architecture to
6. Data Quality Declaration
The Australian Bureau of Statistics (ABS) publishes a Data Quality Declaration statement
(DQD) with each data set it issues. The DQD provides contextual and narrative
guidance to data consumers as to the relative suitability of the data set
within a given context. The consumer can then adjudge whether or not the data
set is suitable for their purpose.
7. “The data is always
A data quality error indicates a failure in the process, the
system or the people. Use the data to inform and drive process change.
8. Data Quality by Design
In manufacturing, the production line would stop if there was a
failure in the quality of products. Why is it not the same with failures in
Note that the just because we have the ability to profile a
feature doesn’t mean we should! 100% data quality is almost never necessary (at
least for analytic decision-making). Pragmatism should be applied to prioritise
profiling and remedial efforts.
9. Useful Reference
Several public domain reference frameworks were identified as
being useful to the Data Quality Practitioner community, including:
In “Big Data” environments, the velocity, volume and variety of
data processing create issues for timely measurement of veracity. Compromises
will be necessary and we therefore need to be very clear and pragmatic about
applying our data quality checks:
When to check.
What to check.
What rules apply.
How frequently to check.
Where to check (which steps in the data processing flow).
Please do leave a note to let me know your thoughts, or to share any
When I was a kid growing up in the UK, Paul Daniels was THE television magician. With a combination of slick high drama illusions, close-up trickery and cheeky end-of-the-pier humour, (plus a touch of glamour courtesy of The Lovely Debbie McGee TM), Paul had millions of viewers captivated on a weekly basis and his cheeky catch-phrases are still recognised to this day.
Of course. part of the fascination of watching a magician perform is to wonder how the trick works. "How the bloody hell did he do that?" my dad would splutter as Paul Daniels performed yet another goofy gag or hair-raising stunt (no mean feat, when you're as bald as a coot...) But most people don't REALLY want to know the inner secrets, and ever fewer of us are inspired to spray a riffle-shuffled a pack of cards all over granny's lunch, stick a coin up their nose or grab the family goldfish from its bowl and hide it in the folds of our nether-garments. (Um, yeah. Let's not go there...)
As data practitioners, I think we can learn a few of these tricks. I often see us getting too hot-and-bothered about differentiating data, master data, reference data, metadata, classification scheme, taxonomy, dimensional vs relational vs data vault modelling etc. These concepts are certainly relevant to our practitioner world, but I don't necessarily believe they need to be exposed at the business-user level.
For example, I often hear business users talking about "creating the metadata" for an event or transaction, when they're talking about compiling the picklist of valid descriptive values and mapping these to the contextualising descriptive information for that event (which by my reckoning, really means compiling the reference data!). But I've found that business people really aren't all that bothered about the underlying structure or rigour of the modelling process.
In my new discussion paper, I explore the organisational and cultural challenges of implementing information governance and data quality. I identify potential problems with the traditional centralised methods of governance and data quality management, and offer alternative organisational models which can enable a more distributed and democratised approach to improving your organisations data. I also propose a simple four-step approach to delivering immediate business value from your data. TOP REASONS TO READ THIS PAPER:
You have identified that you have data quality issues, but don't know where to start.
Your data is spread across multiple business groups and disparate business processes.
You have limited budget or resources.
Executive buy-in and "top/down" mandate for data governance is hard to come by.
Data Governance Coach Nicola Askham has been hosting a series of interviews with folks involved in Data Governance and Information Management, and was kind enough to ask me to be her latest victim guest. We covered a broad range of perspectives on the topic, including:
Why you shouldn't start out in data governance if you want a career in data governance.
The mindset of good data management.
The importance of sociology, psychology and philosophy for Information Management practitioners.
Why data quality is like a public toilet. (Yes, really.)
As well as a packed schedule of workshops, expert viewpoints and case studies, DQ AsiaPacific will also feature the gala presentation of the fourth annual DQ APAC awards, hosted in partnership with IAIDQ.
The exciting lineup of speakers and panelists at DQ AsiaPacific includes:
David Dufty, Australian Bureau of Statistics
Rick Andrews, Telstra
Althea Belford, Melbourne Water
Paul Ormonde-James of Australia Post
Sofia Chancey and Dr. Thomas Buehlmann, Accenture
James Price, Experienced Matters
Alan Doyle, NAB
Ram Kumar, Insurance Australia Group
As well as chairing the event, I will also be hosting an interactive masterclass: "Managing for Effective Data Governance." In this half-day workshop, I will explore the human aspects of Data Governance and examine what it takes to be successful in implementing effective information-enabled business transformation. Areas covered will include:
Do we need to rethink our Data Governance strategies?
Is enterprise-wide Data Management & Governance really achievable?
What techniques and capabilities do we need to focus on?
What skills and personal attributes does a Data Governance Manager need?
A full brochure for the DQ APAC conference can be dowloaded here...
Navigating the complexities of Information Management
and Data Governance
Those of you out there who know me, have
been reading my blog posts (either here or on the MIKE2.0 site) or have been following me on Twitter and LinkedIn will know that I can
bang on at some length about how Information Management/Data Governance is
multi-faceted; that the interconnections between the component capabilities are
complex and not hierarchical.
But up until now, I've struggled to think of
a way to represent all of the different aspects of the IM/DG agenda and show
how they inter-relate. I’ve sometimes alluded to there being a network of relationship
between elements, but this has been a fairly abstract concept that I’ve never
been able to adequately illustrate.
And in a moment of perspiration, I came up
The “tube lines” represent different IM/DG
competencies/capabilities, while the "stations" are indicative of more specific activities (and their associated deliverables or outputs).
This is very much a work-in-progress and I’m not saying it’s perfect by any stretch,
but as a “Information Management Tube Map Version 0.1”, I hope it stimulates
some thought and offers some form of anchor point for navigating the
intricacies of and Enterprise approach to Information Management and Data
I’ll be developing this further as I go but
in the meantime, please let me know what you think.
Being a data management practitioner can be tough.
You're expected to work your data quality magic, solve other people's data problems, and help people get better business outcomes. It's a valuable, worthy and satisfying profession. But people can be infuriating and frustrating, especially when the business user isn't taking responsibility for their own data.
It's a bit like being a Medical Doctor in general practice.
The patent presents with some early indicative symptoms. The MD then performs a full diagnosis and recommends a course of treatment. It's then up to the patient whether or not they take their MD's advice...
AlanDDuncan: "Doctor, Doctor. I get very short of breath when I go upstairs."
MD: Yes, well. Your Body Mass Index is over 30, you've got consistently high blood pressure, your heatbeat is arrhythmic, and cholesterol levels are off the scale."
ADD: "So what does that mean, doctor?"
MD: "It means you're fat, you drink like a fish, you smoke like a chimney, your diet consists of fried food and cakes and you don't do any exercise."
ADD: "I'm Scottish."
MD: "You need to change your lifestyle completely, or you're going to die."
ADD: "Oh. So, can you give me some pills?...."
If you're going to get healthy with your data, you'll going to have to put the pies down, step away from the Martinis and get off the couch folks.