How To Deal With Failure

Failure is a difficult topic. No one wants to write about it or be caught reading about it, but its something we all have to deal with at one point or another. Failure can be personal, and hidden away from others, or it can be professional and public. Relationship failure can be one of the most difficult types of failure to get past. Failure can be individual or shared. It might be your own failure, or a failure of someone else, or even a group failure. It might affect you directly or you might be helping someone else recover. Recovery can require honest self-evaluation and difficult, even life changing decisions.

Failure can even be difficult to define. Depending on the endeavor, failure might mean anything in the range of losing all the assets assigned to that endeavor through to the breakdown of a relationship. It also might mean that a given project simply doesn’t produce the desired result. For the purpose of this article, the topic will be restricted to engineering related projects, and these projects will always be associated with some business operation. Business is an inherently risky activity – success is not guaranteed, and you may wind up losing money or assets. Most of what you can say about project success or failure can be stretched to include other types of endeavor, so read this with engineering projects in mind, but be looking to implement ideas in other areas of life and work.

Sometimes failure is an absolute thing like you run out of money or your product doesn’t work. Sometimes it’s a relative thing, like a competitor wins a bid for a big contract.

Causes of Failure

When you break it down, there are many potential causes of failure:

Failure of leadership

Failure of leadership can take many forms including bad example set, bad culture established, flawed goals, harmful group dynamics allowed to fester, and many more.

Lack of communication or group discipline

Sometimes the right idea didn’t make it to the right people, for example, the interface design people don’t have the same workflow in mind as the coders.

Incorrect interpretation of data

Narratives attached to data are very often based on wishful thinking or emotions rather than science. It takes a lot of discipline and analysis to derive the true meaning from raw statistics. Reality is often complicated, and where we want to see a single root cause for some trend, there may be multiple and even contradictory causes.

Impatience/Too few iterations

Sometimes we generalize where we should be looking for details, and we settle for something close enough that doesn’t really relate to market conditions.

Not willing to hear criticism – believe your own BS

Understand the difference between dedication and stubbornness. As painful as it can be, teams need to include someone who is not a bright sky optimist. Teams need to hear the dissenting point of view to make sure their ideas can stand up to scrutiny and criticism. There is nothing worse than a BS echo chamber to doom what should be a ground-breaking project. One great way to handle this is to actually use your own product as a part of your business process. A company that produces the ProductCenter PDM software uses their own software as a CRM. This leads to a rock solid product where the bugs get identified and figured out quickly, and most people in the company can do tech support. It’s a brilliant plan, and it’s good for everyone involved.

Knowing the difference between arbitrary limits and real boundaries

If your project fails because of arbitrary deadlines or limits, that’s really an error that can be avoided. Deadlines are important to drive urgency, but if there’s nothing really holding you to a date or limit, you should see what you can do to allow your project to succeed. This seems obvious, but many groups get caught up in delivering less just to satisfy a deadline that may have been unrealistic to start with.

There are as many different ways to fail as there are projects or project leaders, this is by no means an exhaustive list.

Recognizing Failure

A project can only be said to have failed when it is no longer making progress. The team might quit because they are working from bad assumptions, or because they have run out of resources (time, money, etc.). This is to say that as long as you keep working, you have the chance of rescuing your project, and you really only fail when you quit.

Running an engineering or design project is in many ways like gambling at an odds-based game:

• You need to start with a well defined strategy that takes contingencies into account
• Success is not guaranteed
• Success is measured by return on your investment
• If you are losing, how much you lose is always up to you
• Even winning is often preceded by a period of losing
• It is best to have in mind a maximum amount you are prepared to lose – you need to avoid a project failure turning into a more general failure of the company.

One of the tough things about failure is that it is sometimes hard to recognize or admit. Very long term goals can make what might seem like a short term failure just look like a temporary setback. Of course short term failure is also less consequential than the failure of a long-term project. Failure takes on many forms, and those forms can depend greatly on how you define it. Because the idea of failure can be so emotional and so variable, it is best to have actual goals or limits to define when it’s time to cut your losses, or when you can still make some adjustments and try again.

Social and professional coaches encourage people to “never give up” or do “whatever it takes” to achieve goals. While they sound macho and noble, these clichés should not  be taken at face value – you must be able to establish limits and boundaries. Persistence and toughness are admirable qualities, and often lead to success, but failure can trump all of your best intentions. The nightly news is full of people who don’t understand the limits of extreme statements, and are willing to double down on ideas that are not destined to succeed.

If you were to start gambling with the “never give up” attitude, you may get lucky, but chances are you would leave having lost everything. No one really plans to fail, but failure should be part of any plan that involves risk. You need to establish how much you can afford to lose. It’s very romantic to avoid even considering failure, but you need to have in mind some line beyond which you cannot cross.

Failure in business is more common than success. Some estimates put the failure rate of restaurants in their first year between 30 – 90%. Failure in this sense can be defined by the inability to cover your debts, combined with ceasing operations. The causality can go either way (bad debts cause end of operations or end of operations cause bad debt) but the result is the same – there are some people left wanting money from you while you no longer have the means to make that payment.

Recovering and Learning from Failure

So much of business and project management psychology seems to promote maintaining a positive outlook regardless of the circumstances. This has some obvious benefits, but there are also some potential risks. If you get in the habit of only seeing the sunny side of the situation, it can become difficult to be honest with yourself about your own shortcomings, or even to recognize when the situation begins to take on problematic characteristics.

Since the only positive value of failure is what you learn from the experience, your ability to turn this negative into a positive is completely dependent on your ability to be brutally honest with yourself.

Many managers employ a post-mortem phase at the end of all projects, regardless of the success of the project. This is a systematic and built-in method specifically aimed at learning from mistakes and failures. The post-mortem may not benefit you or your team for the present project, but the goal is to avoid the same mistakes and gain additional insights to use for future projects.

Recovering from a failed project can be tricky. Making adjustments on the fly can help you bring goals into better focus, but can also allow you to fall prey to classic project management errors such as project creep and micro management. On one extreme, you can run a project into the ground for a hard stop failure, and on the other extreme you risk being labeled as a micro-managing control freak. Ideally, pursue a middle ground by making the team aware of a bit of drift in the project goals in real time, which can influence the outcome better and more organically than making direct management tweaks.

Planning to Avoid Failure

In addition to mistakes, poor planning and overlooking details, failure sometimes happens through circumstances of chance, actions of other people, or events that are one way or another not under your control. There are many things you can do to avoid failure in engineering projects:

• Get the right people on your team, including subject matter experts, people who understand the market, the technology, the users, etc.
• Solicit input from a range of people, including people you may not agree with
• Try to foresee potential areas of failure of difficulty
• Allow for a realistic schedule

Maybe the best piece of advice to help you avoid failure is to be flexible and open to wisdom from unexpected sources. Analysis is good, but too much of it paralyzes. Instinct is good, but too much of that can lead to an unsubstantiated course of action. The best leaders solicit advice, and can discern good advice from bad advice.

Look at each decision through the lens of what you’ve learned from mistakes and failures of the past. This is the best way to make sure that failure really isn’t a waste of time and resources. Past experience of failure also helps you evaluate risk in each situation, and understanding the difference between risk due to chance or risk due to human error will lead you eventually to success through better and better decisions.