Best practices for a Business Intelligence (BI) project

Posted by Albert Vanderplancke on Feb 17, 2016 5:22:20 PM

Topics: Development integration & analytics

By Ian HightCube_with_blocks_flying_around_it.jpg

In this article I take a deep dive into how to develop a best practices approach to BI based on seven project phases.

Business Case

The first phase is to set out the strategic business case for the project based on the vision and goals. This business case will answer the first two questions we set out in the last article, around which area of the business needs BI support and are the goals and measures to put in place. It is here that the business sets out the vision for the project: does the business look like and how does it behave differently when the project is completed? Who are the business owners seeking support and is their role to play; resources do they bring to the table and decisions do they need to make in the project plan?

Scoping

The next phase is project scoping, activities which relate to the initiation of the project and includes the key step of developing a project charter. The project charter is the planning team’s concise statement of core goals and values and defines the scope of the project, the business needs, outline of the (internal) customer’s requirements, project justification, feasibility study, project scope, milestones, resources (including a budget summary) and the desired outcomes that will fulfill the business requirements. 

When developing the project charter, the team also needs to take into account relevant aspects of the organisation’s environment, the factors and systems that can affect the project’s likelihood of success. Such aspects include governance, company structure, external stakeholders, assets such as human resources, intellectual property, patents etc and the prevailing market-place conditions.  

The project charter should also set out the ‘process’ assets that the team will use, both directly and indirectly. These include procedures, plans, guidelines and formal and informal policies whose effects must be considered. Be sure to include the lessons and documentation from past projects (successful or otherwise) and list those assets in schedules. There is potentially a long list of such assets and examples are calendars, schedules, templates, reports, audits, accounting codes, contracts, risk analyses and past project evaluations – the list goes on. 

(A note here: if the project is a new venture for the organisation and so there are insufficient skills or experience to carry out the project, then the work should be outsourced. Outsourcing businesses will have the experience and so will be able to help develop the project charter along with the internal project owner.)

A well-written project charter is a powerful tool to be used each day to judge the effectiveness of the development effort. It becomes the reference point for settling disputes, helps avoiding “scope creep” (a major cause of project failure), enables the team to judge the potential benefit of new ideas as they arise, measures progress against the timeline and keeps the team focused on the overall desired results.

Statement of Work (SOW)

The third phase, which is based squarely on the work in the charter, is the Statement of Work, a description of build, test, rollout and support components of the project. If the project is being outsourced, the SoW is a key document and will form part of the final contract. 

The SoW template will usually cover a wide range of detail on the development team needs to do, the budget for those activities and the timeline. For example, it sets out project analysis and design for the BI architecture, design of data models, reports data quality assessment and the metadata repository. In addition, any special technical or functional requirements along with a project plan with details of key milestones. 

The SoW tests and validates the components of the project charter and if changes are needed based on that testing, the charter can be updated.

Building

This phase is where the team rolls its sleeves up and gets down to work. This is the code construction phase during which all physical data models, code and BI reports are built and the agreed BI solution is finalised ready for testing and then rollout.

Major tasks here include developing the physical database, the Extract, Transform and Load (ETL) code (and testing it), the data migration plan and the conversion plans. Logical and physical data models for the data audit and metadata repository are built, the ETL/interface application documentation is designed and construction and unit testing of BI reports, DDL scripts for physical database implementation, ETL modules and ETL load scheduling procedures are also all included in the task set. Business Intelligence (BI) project plan

The team should also plan how user testing will be managed: for example, if a reporting prototype has been created, it can be reviewed and refined based on user feedback. In summary, this phase plans data migration and conversion to production, the rollout plan and the testing plan. It is essential in this phase to refer back to the project charter as the guide to what and how to build; if there are potential conflicts the charter may need to be amended, which may involve going back through some of phase two.

Testing

This key phase will confirm that the completed BI solution meets the business goals, conforms to the project charter and has followed the Statement of Work. In effect, this phase prepares the new system for migration to the production environment.

Key activities focus on unit, system integration and regression testing and gaining user acceptance. It includes quality checking of all code and all BI reports developed in the previous phase, verifying that the BI solution functions as expected.

During this phase the ETL code is tested for performance and the whole system is tested for integration and data flow. Users test the system for acceptance based on the user acceptance test plan developed in the previous phase. System administrators may be required to maintain the system. Once all tests are completed and passed, the system is signed off ready for rollout; business users are then trained in the use of the new system and user and system documentation (such as operations manuals and runbooks) is also prepared.

Rolling out

In this phase the project team determines the organisation's readiness to deploy. The rollout planning and production execution activities include producing user and system documentation and user and administrator training courses plus deployment of BI databases, metadata repository, code and BI reports.

The tested system is migrated to the production environment so that the agreed and authorised set of users can start accessing it. The team monitors the migration, ensuring milestones are adhered to, communicating at all times with lines of business management and users whose jobs are affected by the new system. In this way, the right expectations are set (and met) so users begin their new experiences as fans of the system rather than detractors.

Supporting

This phase is the continuing monitoring and maintenance of the new BI system, to ensure that the system functions and performs as the business expects. This phase will last until the next project, either to upgrade the existing system or replace it with a new one.

This final phase consists of the post-production implementation activities such as BI application monitoring and support, possibly including monitoring of data load logs, data volumes, data auditing, report usage frequencies and change tracking. This phase also defines the problem and change management processes needed to identify future issues and develop changes to the system.

A word on project management

Project management underpins the entire Business Intelligence project, starting from scoping and planning to roll-out and production support. Project Management activities include ensuring the required resources and infrastructure are available and that the project is implemented on time and on budget. The sheer complexity of a major BI project requires that a proper project management approach be used, preferably a tried and tested automated methodology. Without belabouring the point, there are a range of productivity tools, such as Clarizen, Microsoft Project and @task. Offerings are continually being reviewed and compared - here is an example: http://www.projectmanagementsoftware.com/