Auto-classification POC: the results show (part 1)
So the auto-classification POC I’ve been involved with is over, and we’re putting the finishing touches on the report out for the client and the vendors. And although a public report is not possible given the agreements between everyone involved in the effort, there are some more general takeaways I can share, which, despite their generality, will hopefully be valuable to folks out there interested in the auto-classification and text analytics (AC/TA) space.
Infographic: ECM at a glance
Auto-classification – One Piece of the Puzzle
I’m still in the middle of the auto-classification POC I started back in December, although it’s almost done. Right now, we’ve finished the on-site work and are back at the ranch combing through data, crunching numbers, and trying to net out the results for ourselves, for the vendors, and for the client. Once that’s done, I’ll definitely share as much as I can with you all without getting on the wrong side of confidentiality with anyone, but in the meantime, I wanted to share two related aha moments I had around auto-classification in the last few weeks.
ECM Predictions 2112
Rather than hedging my bets by making predictions that are difficult to disprove (click here for a bunch of those), I figured I’d try an alternate strategy in this post: predictions that none of us will be around to verify.
So with that (and with hats off to everyone’s favorite Canadian power trio—and with apologies to all you Triumph fans out there), here are my ECM predictions for 2112…
- The end of the paper office!!!!
- Shared drive volumes reach the tipping point, organizations finally tackle the unstructured content problem.
- Big ECM-SharePoint integration ready for prime time (this time I mean it!).
- SAP will acquire Open Text.
The final word
So there you have it—ECM a century from now. I encourage you all to heckle me today (as usual), but feel free to encourage the generations that come after you to heckle the generations that come after me…and let’s get the conversation started!
Auto-classification: a bit of a stretch
Last post I kicked off a series on auto-classification, which has been increasingly top of mind for my clients of late. I want to tackle auto-classification from a few different angles:
- What it is and what it isn’t – the very name auto-classification conjures up almost magical powers that can transform a gloppy, hulking mass of unstructured content into a highly structured, polished collection of tagged documents. As you might imagine, this is not entirely true.
- How it works – not from a technical perspective, because this goes way beyond my knowledge. But I do know a bit about the people and process work these tools require to work properly, and the reality of it may surprise you.
- Whether it works – I’m involved with a POC to test some of the auto-classification solutions out there against that most elusive of things: real client data. We’ve got an organization willing to share a chunk of their shared drive content as well as some vendors willing to use their tools to auto-classify that content. I won’t be identifying either the firm or the vendors here, but I will speak to auto-classification capabilities in general and what I saw working and not working during the POC.
For this post, I want to sort out the first point, because I come across lots of misconceptions about what auto-classification is and how it works (some of them my own).
Shepley v. Auto-classification
We’ve been told that auto-classification is here, or about to be here, for years now, and every time it turns out to be a lie. The road to working, housebroken auto-classification is littered with casualties (the Wal-Mart DVD recommendation engine is the highest profile failure that comes to mind, but we’ve all witnessed demos and POCs that crashed and burned) while I know of only one real (public) success story, the DOD. But a government entity with no shareholders and massive resources and time at their disposal is not exactly the operating model corporations are looking for.
And for the last 18 months or so, my clients have been pretty silent on auto-classification—they’ve had other ECM things on their mind. But lately the buzz around auto-classification has been picking up at my clients and out in the wider world. So I figured it was high time I devoted some attention to it here.
I want to spend a few posts on auto-classification, from a number of angles:
- What it is and what it isn’t – the very name auto-classification conjures up almost magical powers that can transform a gloppy, hulking mass of unstructured content into a highly structured, polished collection of tagged documents. As you might imagine, this is not entirely true.
- How it works – not from a technical perspective, because this goes way beyond my knowledge. But I do know a bit about the people and process work these tools require to work properly, and the reality of it will likely surprise you.
- Whether it works – I’m involved with a POC to test some of the auto-classification solutions out there against that most elusive of things: real client data. We’ve got an organization willing to share a chunk of their shared drive content as well as some vendors willing to use their tools to auto-classify that content. I won’t be identifying either the firm or the vendors here, but I will speak to auto-classification capabilities in general and what I saw working and not working during the POC.
I’m also hoping to hear from lots of you all out there during the series of posts about your thoughts on auto-classification, your experiences with it, thoughts on my thoughts, etc.—so get ready to jump in and get the conversation started.
My first post will be after the winter recess, but in the meantime, I hope you all have safe and enjoyable holidays with family and friends and look forward to seeing you back here in January!
Taking the plank out of my own eye
My last two posts have cast a critical eye on the RFP process, pointing the crooked finger of judgment first at ECM vendors and then at ECM buyers. Convenient for me, since I fall into neither of these groups.
But in the last post, I promised to scrutinize the team I play for: management consultants. Although, along with Doctors Without Borders, the Make a Wish Foundation, and The United Way, we management consultants are making the world a better place one project at a time, we do have some areas for improvement…
In the last post I gave it to ECM vendors on the chin about how they can be their own worst enemy during the RFP process, and, based on the response from you all, it seemed to strike a chord with many people out there.
But in the interest of being fair and balanced about the RFP process, I thought I’d shine a light on some misbehavior of the other RFP participant: the buyer.
After all, it takes two to tango, and in most RFPs-gone-wrong, there’s ample blame to be laid at the feet of the client as well as the vendors.
With that in mind, here are the top things not to do if you’re the buyer in an ECM RFP:
- Try to compress the RFP timelines beyond all reasonability. You’re buying a multi-million dollar platform and committing dozens (or even hundreds) of your employees for weeks, months, or years of work to get that platform up and running and delivering value to the organization. You can’t make an informed decision in four weeks—heck, you can’t even make a bad decision in four weeks. So rather than creating an unrealistic schedule, rushing the vendors through the response and demo phase, and pressuring the team to evaluate them…and then pushing the deadline further and further out to give everyone enough time to get things done, take a step back and create a reasonable schedule. And if it’s longer than you planned (or than upper management is comfortable with) get real and own up to that now. Much better than over committing and having to blow past your end date.
- Do things just for the sake of doing them. RFPs have hoary and venerable traditions associated with them—many of which are completely irrelevant to selecting the best solution. Take the time to huddle up as a team to discuss how RFPs are typically run at your organization and the pros and cons of that, as well as some ways that you might run them differently to get more value. Include not only the core project team but any key decision makers as well as ancillary stakeholders like procurement, finance, legal, etc. Get all perspectives on the table up front, before you’re in the thick of the RFP and running up against your deadlines (see #1).
- Let procurement own the process. Is procurement important? Yes. Do most corporations require you to jump through certain procurement-themed hoops to purchase enterprise software? Yes. Does that mean that procurement knows the right solution for your business needs? Absolutely not. Make sure you’re clear with your procurement folks about the division of labor, i.e., what they are responsible for and what you are (see #2).
- Let the vendors own the process. RFPs are not your job…in fact, this may be the first one you’ve ever participated in. But you can bet that every member of the vendor sales teams has participated in dozens if not hundreds of them. They likely know how these things work much better than you or the rest of your team do, but that doesn’t mean that you should become passive. You’re looking for software to solve your business problems, not theirs; so step up and own the process.
- Forget that you’re the customer. You are at the center of the RFP process, you are the reason for it existing in the first place, and you are the person ultimately that must live with the solution that gets chosen. Every element of the RFP process should be engineered to deliver value to you. If some part of the RFP process does not, you should be seriously questioning why you’re doing it (see #2).
- Take your vendors’ time and efforts for granted. Yes, you are the customer and yes, you have a right to expect responsiveness from your vendors and yes, they want to sell you stuff and yes, they will likely make a very big commission for doing so. But that doesn’t mean you can treat them poorly, waste their time, or expect them to give you hours of free consulting just for the privilege of being part of the RFP. These are real people with real lives, many of whom are very committed to solving client problems with technology. Despite how it might look sometimes when they’re standing in front of your exasperated team speaking to slide 477 of their deck, they spend a tremendous amount of time and energy creating RFP responses and planning presentations—often missing out on time with friends and family to do so. Try to honor that in your dealings with them during the selection process.
- Forget that the goal of the RFP is to solve business problems with technology, not acquire software. There’s so many moving parts to an ECM RFP that it’s easy to get lost and treat the RFP like an end in itself, which of course it’s not. Selecting, purchasing, and implementing the technology is really just the prelude to the real work: delivering solutions to the organization that solve meaningful business problems. We tend to get lost in all the bells and whistles, all the feature sets and product roadmaps we hear about, and visions of FULLY EXTENSIBLE ENTERPRISE PLATFORMS LEVERAGING THE FULL CAPABILITIES STACK TO DELIVER INTEGRATED SOLUTIONS SEAMLESSLY dance in our heads. None of which matters if we can’t quickly and effectively deploy the chosen technology to deliver value to our end users—don’t lose sight of that.
Hopefully this post helps spread the blame for RFPs gone wrong, but as always I’d love to hear from you all out there: jump in and share your advice for buyers on how not to handle the RFP process.
But as I sit here in my hotel eating room service and writing this post after a long day working with clients, I feel a little disingenuous, because with this series of posts so far I’ve conveniently left the plank in my own eye.
Yes, ECM vendors have issues and yes, ECM buyers have issues, but what about us consultants? Not the integrators and implementers who actually do things, mind you, but us more strategic folks, the ones who come in with their fancy suits and their white iPad2s and all those 11×17 Visio diagrams and tell you what you should do…and then conveniently leave before the real work starts—what do we do that makes life harder for all you clients and vendors out there? Where do we need to step up our game to live up to our aspirations of serving client needs before all else?
And while I take some time in the next week to cast a critical eye on the practitioners in my own guild, feel free to get the ball rolling with thoughts and criticisms of your own: I can take it!
Two short: eight things NOT to do if you’re an ECM vendor at an RFP final presentation
Maybe it’s the lack of sleep from having a newborn or the unreliable power grid (and therefore air conditioning) this summer in Chicago, but I’ve got some sand in my shorts about how I’ve seen ECM vendors handle RFP presentations.
- Have more salespeople in the room than there are client team members. Sure, there’s a lot of technical detail in the solution you’re pitching. And, yes, there is some value in having enough experts in the room to answer any possible question. But I’d rather have two or three people speak at a business digestible level about the solution and then agree to get expert follow up where needed.
- If you do bring an army of salespeople, they shouldn’t be on their phones or computers doing other work during the presentation. If they’re that busy—and have enough bandwidth during the presentation to be buried in email or other work—why are they here?
- Be unable to provide a clear idea of what your solution costs – I know that this depends on a lot of factors, but it’s possible to represent this in a transparent, reasonable way. Don’t use it as an excuse to hedge by being vague or ambiguous.
- Try to pack 6 hours of material into a 3 hour session. Yes, ECM is complicated and we could spend a whole day just talking about a single capability, like OCR, let alone all the functionality included in the solution. But at a certain point you need to be able to net it out in the time allotted or cut something. If you can’t come in on schedule in a 3 hour presentation, how will you fare during the implementation?
- Show up as the incumbent with anything less than a full knowledge of the client context – this is your client, who has already bought stuff from you, who you are supposed to have a relationship with. If you come to the table ignorant of the basics, why should they buy more stuff from you?
- Be less organized than the client – You do RFPs all day long, they don’t. You should be Johnny-on-the-spot with follow up items, questions, requested information, etc. Don’t make the client work for it. And let them know daily where you are with your items. This RFP is a big deal for them—heck, some of their jobs may be on the line over the results. Do everything you can to make it less stressful for them and easier to do business with you.
- Be unclear or ambiguous where you are strong and weak relative to the RFP requirements – In terms of weaknesses, this is obvious: if you can’t meet a requirement (or don’t meet it well), so be it. No one meets them all. But hedging or dodging only casts doubt on your integrity as a vendor on top of your inability to meet a requirement. And in terms of strengths, don’t undersell or miss the chance to point out where you’re strong. It’s obvious to you, and it might be obvious to someone who’s read the response, but not everyone in the room has.
- Make sure the client gets a feel for who you are as an organization and as individuals – The client isn’t so much buying a piece of technology as they are entering into a relationship, and that requires them to trust you and your firm. They can’t do that if they don’t know “who you are”—so spend time while you’re there showing them.
There. I feel better already.
But how about you all: got any good advice for vendors on how not to handle the RFP process? Jump in and share your best horror stories.
And in the interest of fairness, I’d love to hear from the vendors: what are the things that buyers should avoid? How do they make the process more difficult, less effective, etc.? What makes you absolutely cringe during the RFP?
Let’s get the conversation started…
Bundle of joy
My wife and I just welcomed our second child into the world last week, so I’m taking a few weeks off from the blog to spend some quality time with my family.
While I’m doing that, here’s some oldie but goodie posts you may not have seen before:
- No one cares about ECM
- What does an organization really value?
- Everything I needed to know about ECM I learned in Divinity School
- Irrelevant taxonomies


