"BACK TO BASICS" MANAGEMENT SYSTEM AUDITING PROCESS

International Assessor/Auditor Conference
Business continuity and management system auditing
"BACK TO BASICS"
MANAGEMENT SYSTEM AUDITING PROCESS

(Alfred) W.K. Au
Conformity Assessment Branch, Standards Council of Canada


Abstract

The management system auditing process is growing more complex. This is due to the advancement of technologies, diversity of operational processes, growth of organizations, and evolution of management systems. As well, there are increasing expectations from all stakeholders, including the public, for more enhanced, more capable and more integrated systems. This complex environment causes some dissatisfaction with the management systems auditing process.


Management system audits embrace the process of planning, execution, review and enhancement of
underlying systems. It has been a vital contributor to confidence in the conformity assessment of
management systems. And, the competence and, in particular, code of ethics of an auditor has served as a
crucial enabling factor for success in the management system auditing process. This paper focuses on
examining the reasons for potential ineffective audit processes and stakeholders' dissatisfaction with
management systems auditing processes. "Healing" of the prime causes would help elevate the
fundamental principles of the management system auditing process and provide effective and value-added
audit practices. Taking into consideration the declining confidence in the conformity assessment of
management systems, the characteristics of a "Back to Basics" management system auditing process is
illustrated with an outline of a generic process guideline followed by DO'S and DON'T practices.

International Assessor/Auditor Conference
Business continuity and management system auditing

Introduction

By searching the web 111, there is a wide range of definitions of audit. All the key words, such as
"systematic", "independent", "examination", "review", "records or evidence", "documented process",
"performance", "practices", "compliance", "conformance or fulfilled", "system", "operations",
"requirements and/or applicable regulations, standards, specification, and criteria", are well aligned with
the definitions of audit established in ISO 19011:2006 X41. Audit is commonly utilized in different
organizations and industries as a methodology or tool for evaluating their practices and performance, determining their extent of conformance and/or compliance, and identifying areas of excellence for further development or enhancement.
Over recent years, studies regarding the management system audit have emphasized the importance of the
implementation of an effective and value-added audit. Information indicates that the focus of most studies
is on the competence of auditors. There is no doubt the competency of an auditor is one of the essential
elements contributing to an effective and value-added audit. Based upon ISO 19011:2006, management
system audit embraces the process of planning, execution, review and/or enhancement of underlying
systems (41. This suggests an effective and value-added audit would require implementation of a highly-
regarded management system auditing process which assures the reliability of the outcomes generated
from the process.

The rapid growth of the global market has diversified the strategies of organization management systems
with a different focus on technologies, business practices, operational processes, and financial and other
resources. The management system is recognized as a tool for developing and executing a successful
strategy. The demands on the management system auditing process have steadily increased over the last
decade, and expectations are intensifying. By the same token, with the swift growth of certification bodies
addressing the increasing demand for management system certification, competition is getting tougher
leading to management system assessments of inferior quality and causing a possible decline of
stakeholder confidence.

This article attempts to examine the reliability of and confidence in the management system auditing
process. Fundamental principles of the management system auditing process will be reviewed. As well,
simple and practical tools will be offered to ensure the consistency of the management auditing process
and the delivery of an effective and value-added audit. The objective is to restore the credibility of the
management system audit as demanded by stakeholders.

International Assessor/Auditor Conference
Business continuity and management system auditing

Management system auditing process

A management system audit relies significantly on an all-embracing audit system with process established
in accordance with the concepts of the PDCA ("Plan-Do-Check-Act") Cycle by DR. W. Edward Deming,
an iterative four-step problem-solving process typically used in modern quality control. Key components
of a management system auditing process should include: planning of audit activities, audit execution,
monitoring and reviewing the audit activities, and enhancement of audit activities. In this section, the principles and the importance of a PDCA management system auditing process are examined.
Proper application of the PDCA cycle on a management system auditing process assures embracement of adequate practices with desired characteristics. It is crucial to uncover and/or determine what is expected from the process before planning the details. This "Back to Basics" management system auditing approach (Figure 1) channels one's thoughts through the basic guideline before moving forward to establishing the solutions for meeting the process objectives and expectations with the expected outcomes.

The ABC guideline, which is described as follows, is applicable for an overall process, sub-processes, phases or stages of a process.
• Appreciate the objectives of a process.
• Be aware of the expected outcomes and expectations of a process.
• Create tools and guidelines for assuring consistency

International Assessor/Auditor Conference
Business continuity and management system auditing

P - Plan C - Do C - Check A - Act


figure 'l "Back to Basics"
Management System Auditing Process



A) Planning

Planning is a process for determining a course of action with anticipated outcomes to ensure the achievement of the process objectives and/or expectations. With regard to the expectation of a management system auditing process, the process should be value-added, supportive, reliable, reputable, consistent, effective and efficient. The objectives of the planning process generally include the demonstration of conformance and/or compliance to applicable standards, regulations and laws, and identification of enhancement opportunities.
Habitually, the planning function in a management system auditing process is straightforward and
easy to manage. All too often, the complexity of an organization, diversity of technologies, processes
and scopes of management systems, and evolution of management system practices, would have
injected additional variables affecting the collection of requisite information for proper planning. This
is not a real concern to most people because they believe the issue could be fixed during the audit.
However, missing or inadequate information may result in breach of a client's expectations. This is
caused by an inappropriate auditor being assigned, insufficient time allocated for an audit, inadequate
coverage for the required scope of certification, erroneous scheduling, inappropriate pricing,
ineffectiveness of an audit, or complete failure of an audit. All such possibilities could lead to a loss of

International Assessor/Auditor Conference
Business continuity and management system auditing
confidence in the management system auditing process. This may lead to a loss of confidence in management system certification.
At this moment, the "Back to Basics" approach will be adopted to question the degree of information
required for effective planning. The required information may vary from one organization to another.
Prior to determining the amount of information required from an organization or operation for
successful audit planning, requirements established in the applicable standards (e.g. ISO 17021:2006,
IAF Guidance Documents, and IAF Mandatory Documents), regulations and/or specifications, and
procedures shall be understood. With the up-to-standard procedures in place, individuals who have
been assigned with the responsibility for obtaining the required information for planning should
understand clearly the applicable standards, requirements and/or specifications and procedures to
ensure proper application. Requirement of effective resource management would be necessary for
assuring involved personnel are competent for carrying out the required duties. Putting it in a nutshell,
an effective planning process is built upon adequate understanding of applicable requirements, the
amount of information required, tools required for obtaining information and processing the
information and, proper scheduling and competency of personnel responsible for the planning process.

Occasionally, inconsistent information will be identified in management system documentation
causing confusion and obstruction for planning a proper management system audit planning. For
example:
l) Communication information sent to audited organization - "Manufacturing and
marketing of devices mainly for Type A and Type B devices";
2) Previous year audit report from the assessment team - " Design, production and
distribution of devices mainly for Type A and Type C"; and
3) Certificates issued by the Certification Body - "Design, Development, Manufacturing
and Sales of A type Devices, B type Devices, C & Cl type Devices, D type Devices and E type Devices"



B) Execution

Audit execution is a systematic, independent and documented process for collecting and evaluating objective evidence to determine the extent of conformity to the audit criteriat41.



International Assessor/Auditor Conference
Business continuity and management system auditing
The execution process begins with communicating to client and auditee about the objectives of the
audit and reviewing management system documentation. It is important to recognize that information
provided in the audit objectives would serve a vital element of communication between the auditing
organization and the audited organization along the execution process. However, the objectives of an
audit are not often explicitly communicated between the auditing organization and the audited organization.

An objective is a planned or intended outcome which provides direction and specific focus for a
particular audit. Very often, the objectives are addressed as internal audit, surveillance audit,
certification audit or re-certification audit. Stating the type of an audit does not give a clear and
accurate picture of expectations regarding the outcomes of an audit as objectives vary from one audit
to another audit even for the same type of audit. The management system documentation review or
readiness review process is mostly short-circuited due to heavy competition in certification. In most
situations, the auditor's notes of an on-site document review are often "cut and pasted" from the
previous document review. This diminishes the intention of having a proper document review. Hence,
the on-site audit is being put at risk with the lack of document review which contributes as an essential
part of an appropriate preparation.

Other than communicating with the audited organization and the preparation of an on-site audit, the
execution of an on-site audit is the foremost examined issue of the reliability of an audit. Execution of
an on-site audit includes conducting the opening meeting, collecting information to reaching audit
conclusions and conducting the closing meeting. Each stage of an on-site audit has its own purpose
and the assessment team should be well aware of it in order to meet the objectives with the expected
outcomes.

a) Opening meeting

The purpose of an opening meeting is to confirm the audit plan, describe how the audit activities
will be taken, confirm communication channels and finally provide an opportunity for the
auditee to ask questions 141. This is considered an easily managed stage of an on-site audit.
However, reality often shows that the declared purpose is not always covered in the on-site audit
procedure. Besides, it is very seldom that the rights of the audited organization are clearly
explained during an opening meeting. The assessment team should be aware that they are not
only dealing with "Management System" personnel but also management and staff of the
audited organization. By having a well covered opening meeting, the boundary and limitations
of an audit are clearly communicated and understood. This assures the objectives and expectations of an audit are met.




International Assessor/Auditor Conference
Business continuity and management system auditing

b) Collecting information to reaching audit conclusions
The reliability of on-site audit as well as confidence in the management system auditing process
relies on excellent performance during this fundamental and crucial stage of management system
auditing process. It is obvious that the purpose of the execution stage is to collect information
relevant to the audit objectives, scope and criteria, including information relating to interfaces
between functions, activities and processes [4j. During the course of collecting audit evidence to
reaching audit conclusions, the following practices which are often observed would lead to ineffectiveness of an on-site audit:
• notes from the previous audit and/or document review are "cut and pasted" as the outcomes
of the current on-site document review ;
• information required for demonstrating conformity to an audit requirement is not sampled
for verification or examined;
• closer examination is not performed even though unclear and/or irrational information is
provided;
• completeness of the provided objective evidence is not verified;
• the sampling size is small without considering other influencing factors, such as the volume
of activities;
• roles and responsibilities of the individual auditee are not considered for determining the
coverage of an interview;
• top management is excluded from being audited;
• specific solution is offered;
• frequent practice of using leading questions and/or assisting with the completion of the
answers to own questions;
• audit findings and/or conclusions are based on presumption without respecting the
applicable requirements;
• standard requirement checklist is used without a thorough understanding of the auditee's
management system processes; and
• objective evidence does not clearly cite the specific record which substantiated non-
conformance.

c) Closing meeting

This is the period for the assessment team to present the audit findings and conclusions, and
ensure that they are understood and acknowledged by the auditee. This implies another
opportunity for the assessment team to convey the audit outcomes to the auditee in a formal and open manner. In most cases, however, the following practices are occasionally observed:

International Assessor/Auditor Conference
Business continuity and management system auditing

• the conclusion is made on the effectiveness of a management system without considering the
status of audit outcomes from other sites relating to the same certification;
• inconsistent information regarding the handling of audit findings; and
• the right of the audited organization regarding appeal and compliant processes is not clearly
explained

Apart from the above mentioned practices during the execution of an on-site audit, the code of
conduct and ethics of auditors is one of the essential factors of a successful management system
auditing process. In accordance with ISO 17021:20001, ISO 19011:2006[41 and ISO 9001 Auditing
Practices Group Guidance on Auditor Code of Conduct and Ethicstst, an auditor is expected to be a
specialist who is able to demonstrate applicable personal attributes, the ability to apply required skills
and knowledge and high standards of ethical conduct. While the position of auditor may be seen as
powerful and privileged, the auditor should keep in mind that superior, arrogant and/or authoritative
behaviour during an on-site audit would only damage his/her reputation as a professional and defeat
the audit objectives. "RESPECT" should be deep-seated as the basic rule of thumb in a professional
code of conduct and ethics.

Offering value-added services without providing consultancy services is always a challenge during an on-site audit. Very often, it is observed that the auditor offers specific solutions which may cause unpredictable damage to a system. There is no universal solution of a perfect system for every organization. As none of the organizations or systems is identical in reality, the solution of a perfect system for one organization may cause a disaster in another organization.

In accordance with ISO 17021:2006 Clause 3.3, management system consultancy is defined as
participation in designing, implementing or maintaining a management system which includes
preparing or producing manuals or procedures and giving specific advice regarding development and
implementation of a management system.

Generally speaking, sharing generic experiences and/or examples regarding implementation of a
specific requirement does not constitute the providing of consultancy and/or specific solutions. In fact,
the best solution offered to an auditee would be to explain the requirement and the rationale for the
requirement frankly with generic experiences and/or examples offered as practical help. This assures
the understanding of auditee on the requirements and the principles in implementation to fulfill the
requirements. Besides, by sharing generic experiences and examples would offer inspiration and
insight to the auditee for possible enhancement and strengthening of processes. This is the value-
added help that the auditee would expect from the audit at large.

International Assessor/Auditor Conference
Business continuity and management system auditing

C) Monitoring and review
Quite often, the management system auditing process is considered completed upon issuing the audit
, ,:port with or without closure of nonconformities that are classified as "MINOR". As a common
routine, the management system auditing process will be recycled when the next audit commences.
Under this approach, the "CHECK" phase of the PDCA cycle is not well managed in most of the
management system auditing processes.
Jiased on the "Back to Basics" approach; the monitoring and review process should be properly
managed as a crucial part of the management system auditing process. The monitoring and review
u;:ess should incorporate i) the collection of performance data, ii) analysis of audit outcomes, which iclude nonconformities and improvement items, and iii) the documentation of the collected data and the analysis of results. An effective monitoring and review process would be a powerful tool providing guidance, coaching, feedback, trends of performance of a management system along with promoting continual improvement to the auditee.
Likewise, coverage of the principles of the monitoring and review process should be extended to cover the competency of auditors who play a vital role in the management system auditing process. This would help coaching in-experienced auditors and advancing the experienced auditors to a higher level of competency. Having the synergy of the above-mentioned circumstances, not only a valueadded on-site audit is guaranteed but also a "value-added audit service" will be offered.

D) Enhancement

The "ACT" phase of the PDCA cycle is usually not managed or not even utilized in the management
system auditing process. The performance of the management system and the outcomes from the
management system audits are hardly ever considered during the planning process of a management
system audit.

The management system audit is often managed on a routine basis, by means of yearly or bi-yearly
surveillance audits and 3 yearly recertification assessments. With reference to Clause 9.4.1.2 of the
ISO 17021:2006, consideration of the performance of the management system over the certification
cycle and previous surveillance audit outcomes is mandatory when planning for the recertification
auditJ21. Definitely, other factors which may affect the performance of a management system are
subject to consideration as part of the planning component of any upcoming audit. This will ensure the
relevancy of the focus of the upcoming audit to the auditee's needs with regard to the performance of
their management system.

International Assessor/Auditor Conference
Business continuity and management system auditing

By the same token, enhancement is not only relevant to the focus and objectives of an audit but also
applicable to the assignment of a competent assessment team. This will guarantee the competent
assessment team will better understand the stakeholders' needs and expectations of the audit. In
addition, this will ensure an adequate preparation for the audit and suggest a value-added audit
service.

Conclusions

In the last decade, we have all witnessed the growing demand on management system standards as well as management system certification. The management system certification service is predominately providing a management system audit and issuance of certificate only. The management system audit has become the solitary tool for management system certification. The value of the management system certification has been challenged due to substandard certification caused by the deterioration of audit quality and the inappropriate utilization of the auditing process.

Expectations of a management system certification service is more than just a management system audit for determining conformity to applicable standards but a management system audit service incorporating proper planning, a thorough on-site audit, a systematic performance monitoring and review mechanism for improvement and nonconformities, and the facilitation of enhancement of audit activities to yield the outmost benefits of a management system auditing process.
Furthermore, being a vital element to a successful management system auditing process, a competent auditor not only demonstrates the ability to apply skills and knowledge but also behaves with high standards of ethical conduct.

With regards to the competency of auditors, respecting the auditee's needs with high standards of ethical codes in a professional manner helps gain respect from the auditee, clients and stakeholders. This would allow the restoration of confidence and credibility of the management system certification and the conformity assessment at large.
The DO'S and DON'T practices of a management system auditing process as suggested in the table under
Annex 1 would help to elevate the reliability of the process and deliver an effective and value-added audit
service.


International Assessor/Auditor Conference
Business continuity and management system auditing

References


1) Google Search
2) ISO 9001 Auditing Practices Group Guidance on Auditor Code of Conduct and Ethics
3) ISO 17021:2006, Conformity assessment - requirements for bodies providing audit and certification
of management systems
4) ISO 19011:2006, Guidelines for quality and/or environmental management systems auditing
5) Robert S. Kaplan and David P. Norton (2008 January 1) Mastering the Management System, Harvard
Business Review