Category Archives: Automation

Bring Your Shadow IT Into the Light: Part I – Defining the Problem

Let’s face it, if more than two people have ever worked in your IT department, you have some number of unknown systems or some amount of configuration changes which were added outside official channels. These unknown systems and configurations are commonly referred to as shadow IT due to the nature of how they are typically created outside the view of any formal processes and without any organizational approval. The typical approaches to this problem are to either forge ahead and hope that any shadow IT resources will be brought into the light at some undefined time in the future or to lock down the entire enterprise and pull out the thousand-pound corporate sledgehammer to beat anyone into submission that attempts to work outside specifically defined boundaries. While the popularity of each of these approaches is typically divided along the boundary between small-business and enterprise, both approaches fail in their primary objective of limiting the existence of unknown systems, applications, or configurations within the corporate IT infrastructure. Additionally, each has its own side-effects that can be equally detrimental to the long-term success of the IT department.


The failure of these approaches to eliminate shadow IT is either a result of failures in the processes that were designed to prevent it or a total lack of those processes in the first place. Whenever IT changes are made outside of a formal process, those changes drive non-compliance and result in items being missed from your configuration management database. Those items, whether they are applications, settings, personally owned devices connected to the corporate network leveraged for corporate work, or virtual machines spun up as favors, have just added themselves to your undocumented shadow IT resource pool, increasing the amount of shadow IT in your organization and putting you at risk.


So, why should we be concerned with shadow IT anyway? How can a small database configuration change or a virtual machine that was spun up to perform a small task have any effect on your company? Well, there are several ways these things can affect your bottom-line. The biggest two are “compliance” and “technical debt”.


The issue with compliance is that shadow IT forces companies to attack the issue from the wrong side to prove they are compliant. If you know that everything connected to your network and all the configurations for those items are based on a compliant policy, then only the policy itself needs to be verified. However, if there are unknown rogue resources running in your environment, you instead must rely on expensive tools (from a licensing and labor perspective) run at regular intervals to verify what is there and whether it is configured in a compliant manner. This approach is much more expensive and only partially proves compliance. However, relying on your policy to verify compliance can only be done if the policy in place is always adhered to and that adherence can be proven.


Equally concerning to your bottom-line is the technical debt associated with shadow IT. Systems or configurations that should be correctly implemented are instead left unmanaged and unknown. Application fixes must work around anomalies caused by undocumented process interactions. Fixes to problems that are done outside the process can revert settings that leave applications in an unknown and potentially vulnerable state. All this technical debt accrues interest and gets more expensive to fix over time just like debt from a loan if it isn’t paid off. Unless the shadow IT causing the technical debt is not found and eliminated it can be crippling to your company.


But, before we discuss how to resolve the shadow IT problem, it needs to be pointed out that the motivational factors that lead to it are the same for both the “no process” and the “process intensive” approaches at containment. It isn’t enough to just implement a strategy to resolve an issue if the underlying motivational factors that help propagate it are not first understood and rectified. So, what is it that motivates individuals to cut corners despite knowing it will come back to bite them and knowing that it is against the rules and could get them fired? The answer is simple. Pressure to achieve unreasonable goals combined with either a formal process that doesn’t contain all the elements required for employees to be successful or a lack of a process altogether that forces employees to cut corners to meet those goals. When people in the management structure value outcomes over process and deadlines over quality, they are forcing employees to sacrifice their company’s strategic goals for tactical wins. Even in companies that officially state preferences for process and quality, if anyone along the management chain abandons those preferences and continually pushes for unattainable deadlines, the process is typically the first casualty. In some cases, it is management itself that goes outside the process to get things done by skipping levels of management and making requests directly to low-level employees who are powerless to push back. Other times that your processes may be ignored are during production outages when pressure for a quickly implemented fix may force things to be done outside the official process. If the process doesn’t account for emergency situations, a choice might need to be made between policy adherence or business restoration, and once again, it is the process that will lose that battle. In a nutshell, holding individuals accountable for unattainable goals, whether due to inadequate time or inadequate resources to meet those goals, results in a large motivation to ignore policies if those policies are perceived to be contrary to meeting those goals. Without anything to automatically enforce adherence, policies will be bypassed at some point.


While each of these approaches fails in its primary objective, there are side-effects of each that are quite different. The obvious problem with not having a process at all is it takes a hands-off-and-hope approach to bringing shadow IT resources into the light. Unless the implementer of the shadow resource decides to personally bring it into the light it will be lost and forgotten until it is finally discovered by individuals without proper knowledge of all the systems that leverage or integrate with it. However, even with a well-defined process, there are problems with a squash-it-like-a-bug approach to enforcement. Rigid and non-automated enforcement of policy limits innovation by drastically lengthening time-to-market for new ideas and alienates your brightest and most creative employees. While employees should always follow your rules, and in most cases, do, they are under no obligation to maintain employment with you if they can find other opportunities where they can use and foster their creativity. Creative individuals want to work in a place that allows them to not only do their job effectively but also experiment and find solutions to problems that you didn’t even know existed. When they get tired of waiting for your processes to provide quick access to new technologies they will seek work elsewhere. Where will they leave all the shadow IT resources they created that are now unknown and unowned? In the hands of equally frustrated, but less talented, employees that want to leave but lack marketable skills to do so. The last issue with this approach is that it exacerbates the problem of shadow IT rather than having the intended effect of eliminating it by forcing it further into the shadows.


There is hope. With a well-defined approach, you can have a fully illuminated IT infrastructure without alienating your workforce or IT Security team. I will provide an overview of the approach to best ridding your organization of Shadow IT once and for all in the next installment in this series, Bring Your Shadow IT Into the Light: Part 2 – Illuminating the Shadows. After that, get a more technical discussion of the solution with Bring Your Shadow IT Into the Light: Part 3 – Particles and Waves, Lighting the Way to Total Illumination.