You are currently browsing the category archive for the ‘Information Management (IM)’ category.

Ventana Research coined the term “enterprise spreadsheet” in 2004 to describe a variety of software applications that add a desktop spreadsheet’s user interface (usually that of Microsoft Excel) to components that address the issues that arise when desktop spreadsheets are used in repetitive, collaborative enterprise processes. Enterprise spreadsheets are designed to provide the best of both worlds in that they offer the ease of use and flexibility of desktop spreadsheets while overcoming their defects – chiefly inability to maintain data integrity, lack of referential integrity and dimensionality, absence of workflow and process controls, limited security and access controls as well as poor auditability. All of these issues can cause serious problems for business use, which I’ll discuss below.

Companies should investigate enterprise spreadsheet applications that can address desktop spreadsheet issues because they can provide better results, save an organization substantial amounts of time and provide greater accuracy and security. Most products are designed to be a cost-effective replacement for the desktop variety. Enterprise spreadsheets fill various roles to suit specific business needs. Some take the form of a simple data collection program that maps the spreadsheet’s two-dimensional grid to a relational or multidimensional data store. Others offer business intelligence software capabilities that enable self-service automated reporting from enterprise data sources. Still others may be relatively elaborate applications that incorporate programmed workflows, access controls, audit trails and more sophisticated visualization methods than are available in Excel and utilize a relational or multidimensional data store. Dozens of applications that incorporate enterprise spreadsheets are available today.

Enterprise spreadsheets fill an important role in corporate computing environments. For example, the spreadsheet interface has become common in business planning applications because companies find it easier to train people to use this software when it has a familiar look and feel. It might servevr_ss21_combining_spreadsheets_is_a_chore_updated as an alternative user interface to an ERP system for specific tasks, where the object is to simplify and shorten data input or facilitate some analytical interaction (such as comparing two columns of numbers to spot matches or data inconsistencies). An enterprise spreadsheet also may serve as an automated conduit for moving data from one enterprise system to another, using macros to automate actions on the data as it moves between systems. Using macros makes it easier for business people, not IT specialists, to program these actions because far more people understand how to use them than are able to program applications or data connectors. These spreadsheets may serve as a data on-ramp or off-ramp to some process handled in an enterprise system such as ERP or CRM that requires human intervention. In this role, an enterprise spreadsheet provides the programmed workflows, security and referential integrity to fill a process gap that an enterprise system cannot address or is too expensive to implement and maintain in that core system. In addition to collecting data, enterprise spreadsheets may enrich data from an enterprise source in a controlled environment, federate data from multiple systems, perform checks or reconciliations vr_ss21_errors_in_spreadsheets_updatedbefore data enters an enterprise system or perform some analysis for decision support.

I regard the electronic spreadsheet as among the top five most important advances in business management to come along in the last 100 years. It revolutionized almost all aspects of running an organization. It was the original “killer app” that made it necessary for people to go out and buy a personal computer. Yet it has inherent technological defects when used in repetitive, collaborative enterprise processes. One is a lack of data integrity, which maintains the accuracy and consistency of data – spreadsheets are notoriously error-prone. More than one-third (35%) of participants in our benchmark research on spreadsheets said that data errors are common in the most important spreadsheet they use in their job, and another 26 percent said errors in formulas are common.

A related drawback is that desktop spreadsheets lack referential integrity; that is, the meaning and context of an individual cell is defined by row and column headers rather than being defined within the individual cell. This creates the familiar problem when a group of spreadsheets supposedly containing the same data are combined, but someone has added or deleted a row or column: The result is inaccurate. In our spreadsheet research more than half (56%) of spreadsheet users – even those who have been using them for more than a decade – said that they find it usually or always time-consuming to combine data from multiple spreadsheets.  Another problem is that desktop spreadsheets, being two-dimensional grids, have limited ability to manipulate and report data having three or more dimensions. While accountants can work around this limit, it’s a problem for most business users because businesses work in multiple dimensions such as organizational structures (regions or divisions, for example), products (from families down to individual stock keeping units), customers (national accounts down to drop-ship locations), dimensionality and time.

They also lack programmed workflow: People attach spreadsheets to email messages, making it difficult to keep track of the latest versions. More than one-fourth (28%) of research participants said that processes that run on desktop spreadsheets frequently break down because people using them don’t know what to do next or forget to pass them along. Likewise, they lack process controls to ensure that they are reviewed properly and that any deficiencies found in the spreadsheet are noted and automatically returned to the preparer for correction. Finally, desktop spreadsheets have limited security, access controls and audit functions as well as poor auditability. To address these deficiencies, people – especially those in finance organizations – have to spend a great deal of time reviewing and correcting spreadsheets. So while our Office of Finance research finds that the accuracy of information gleaned in desktop spreadsheet processes is acceptable, fewer than one-third of organizations said that the information the finance department provides is timely.

Because enterprise spreadsheets address these issues, they can reduce the incidence of errors and malfeasance that make using spreadsheets in repetitive, collaborative enterprise processes problematic. In many cases, enterprise spreadsheets support what we call “continuous accounting” in that they ensure data quality from end to end in financial processes. Of course, not every enterprise spreadsheet product addresses all the issues I’ve mentioned. But often this doesn’t matter if, for example, the offering is designed to perform a limited set of functions such as reporting or acting as a data conduit connecting two systems in an otherwise controlled environment.

Businesses should recognize that they no longer have to put up with the shortcomings of desktop spreadsheets. They have options to have the best of both worlds, allowing people to continue working in a familiar environment but without the drawbacks that spreadsheets impose when they are used improperly. There is no good reason not to consider adopting such an application. After all, people routinely spend time exploring application options for their smartphones but rarely spend any time getting to know business software that can increase their productivity. There are many forms of enterprise spreadsheet applications to solve a range of business issues. I recommend that businesses investigate options that give users the ease, convenience and familiarity of spreadsheets without the hassle and risk that often goes with them.


Robert Kugel

Senior Vice President Research

Follow Me on Twitter @rdkugelVR and

Connect with me on LinkedIn.

Optimization is the application of algorithms to sets of data to guide executives and managers in making the best decisions. It’s a trending topic because using optimization technologies and techniques to better manage a variety of day-to-day business issues is becoming easier. I expect optimization, once the preserve of data scientists and operations research specialists will become mainstream in general purpose business analytics over the next five years.

Optimization was first adopted by businesses in the middle of the 20th century, aided by the introduction of digital computers. The first technique that gained broad for a few specific purposes was linear programming, one of the most basic optimization methods. Linear programming enables analysts to quickly determine how to achieve the best outcome (such as maximum unit volume or lowest cost) in a given situation. They do so using a mathematical model that captures the key variables that go into the decision and any constraints that may affect that decision. A food processor, for instance, may use three types of cooking oil to make a product. To maximize its profit, the company needs to determine the exact proportions of the three oils that result in the lowest production cost. However, it can’t just choose the cheapest of the three in every case because for flavor and shelf-life requirements there’s a limit to the maximum percentage of each oil that it can use. Linear programming using the simplex algorithm quickly solves the problem.

As computing capabilities became increasingly affordable, companies could use more complex algorithms to handle ever more difficult optimization problems. For instance, the airline industry used it to determine how best to route aircraft between two cities and to staff flight crews. Not only can softwarevr_Big_Data_Analytics_01_use_of_big_data_analytics find the best solution for scheduling these assets in advance, it also can rapidly re-optimize the solution when weather or mechanical issues force a change in how aircraft and crews are deployed. Airlines were also in the vanguard in the 1980s when they started using revenue management techniques. In this case, the optimization process was designed to enable established airlines to compete against low-cost startups. Revenue management enabled the large carriers to offer low fares to price-sensitive but flexible vacationers without sacrificing the higher fares that the less flexible businesspeople were willing to spend. The same approach was adopted by hotels in pricing their rooms. Starting in the 1990s markdown management software, which I have written about gained ground. It enables retailers to make more intelligent pricing decisions by monitoring the velocity of purchases of specific items and adjusting prices to maximize revenue. To be feasible, each of these optimization problems require large data sets and sufficient raw computing power.

We’re now on the cusp of “democratizing” what I call optimization analytics. Big data technologies are making it feasible and affordable for even midsize companies to work with much larger data sets than they have been able to in the past. Our benchmark research on big data analytics finds that about half of participating companies already use analytics with big data. This is partly the result of more powerful and affordable data processing resources but also because companies have invested in systems to automate many functions. The rich data sets created by these business applications provide corporations with the raw material for analysis. This data has the potential to enable businesses to make more intelligent decisions. From a practical standpoint, though, the value of these large data sets can only be realized by moving optimization analytics out of the exclusive realm of data scientists and into the hands of business analysts. These analysts are the ones who have a sufficient understanding of the business and the subtleties of the data to find useful and repeatable optimization opportunities. Three-fourths of companies in our research said that they need these business skills (“domain expertise”) to use big data analytics successfully.

vr_Big_Data_Analytics_14_big_data_analytics_skillsOptimization analytics is a breakthrough technology with the potential to improve business performance and create a competitive advantage. You can’t do optimization in your head, and it’s not feasible in desktop spreadsheets for anything but the most basic use cases, such as linear programming optimizations on relatively small data sets. This is a good reason for almost any company to consider adopting optimization software.

Another reason why companies will find it attractive to apply optimization analytics broadly is that the results of applying optimization routines may be superior to using common rules of thumb or relying on instinct and experience. One of the most important lessons for executives about optimization analytics is that optimal solutions are sometimes (but – crucially – not always) counterintuitive to established norms. For instance, in markdown management, retailers often have found that smaller, more frequent price reductions maximize profits and produce a considerable improvement in sales over the end-of-the-season price slashing that was once considered to be the best practice. In financial services, charging your best customers more for loans and other services turns out to be the optimal choice for the bottom line of financial institutions. Another important insight from our collective experience with optimization is that while the value of these analytics as realized in a single event or transaction may be small, it can have a measurable impact on profitability and competitiveness when applied broadly in a business.

At this point optimization analytics is in a dual mode. On the one hand, there are proven examples of the narrow application of optimization such as those mentioned above. On the other, bringing optimization analytics to the masses is only beginning. Some vendors have made progress in simplifying their analytics, but mainstream products are only on the horizon. It’s also important to recognize that, as with past breakthroughs in information technology, there are bound to be more duds than success stories in initial attempts at using optimization analytics. Experience suggests that a small number of companies that have strong analytical skills and a rigorous approach to managing company data will prove to be the leaders in finding profitable opportunities for applying optimization technologies and techniques. Others will do well to find these examples and consider how to apply them to their own organizations.


Robert Kugel – SVP Research

Twitter Updates


  • 124,337 hits
%d bloggers like this: