During one of my document checks I asked an  accredited calibration agency for a copy of a calibration certificate be­cause I knew the ‘certificate’ on file was simply a Work Order issued by the technician who had checked the equip­ment, and which only referenced the certificate number. I wanted the correct certificate to be on file and listed in the Master Document List.

On the day of the assessment, despite many requests from the QAM, we still hadn’t received the certificate and, of course, the assessor wasn’t satisfied with the document we had on file. We faxed the calibration agency and informed them that we must have the proper certificate sometime that day. What we received was another copy of the Work Order used by the technician who had calibrated the equipment, with the original certificate number erased and another number hand-written above. Unfortunately, this came into the assessor’s hands as he was conferring with the QAM. I was standing to the side and he turned to me and commented that he considered it to be fraud (which it was). The Operations Manager put in a personal call to the subcontractor to request the certificate. The owner of the agency said that he wasn’t prepared to send a copy because “people sometimes altered the certificates in their own favour.” ! He did eventually agree to fax a copy to the assessor’s office.

Lesson: recognise the authenticity of the quality system.

59

Too often employees believe that the quality system is ‘man­agement business’ and they are shocked when they find out that everyone is involved. We do not want to wait until assessment to put this right. I have found it helpful to include training so that staff:

• know where to find the Quality Policy Manual
• know where to find the Quality Policy statement by the CEO
• can describe, in brief terms, what the Quality Policy means to them
• know where to find their copy of the Quality Assurance Procedures
• if asked “how do you know how to do your job?” will have an appropriate response in their own words, such as “I have xxx number of years as … I have xxx certificates and I have working instructions on the intranet.”
• can find their working instructions on the intranet
• can realise and explain the impact of their work on the service or product on which they are working, or for which they are responsible. This is critical, as it depends on staff understanding the nature of ISO9001 and its effect on their work.  (A requirement of the standard).

I use my ISO Cheat Sheet to make this training less of an ordeal, and to prepare everyone for questions which are outside of their normal range of activity.

Employees become accustomed to these questions.

Amid all the glamour of design, new processes, new technology, measurement and “lean management”,  the less glamorous elements of a quality system might not be given the priority and attention they deserve.

For example, housekeeping is very important. The assessor will be looking at the general cleanliness and care which is evident (or not) at your facility. Avoidable dirt and clutter could be regarded as lack of commitment to the quality system..  The assessors will be looking for anything lying loose which should be under control. This includes parts lying about in the yard or documents scattered on someone’s desk or (worse) have been lying for fifty years in a dark corner or hiding in binders which have faded into the background.  They are uncontrolled as well as being untidy. They could also contain valuable information from past experience which has been overlooked. It pays to take the time for a thorough review of archived documents.

The “yard” is part of your facility so take the time to clean it up, especially if it is apart from the main site. It could be included in the certification audit.  The assessor will expect to find that all quality-critical items are secured, maintained and protected from damage, corrosion, and other conditions (such as dirt or damp) that could cause the product (or the conditions around it) to deteriorate.

Your storage areas will be audited, so if you have items waiting for the customer to decide what needs to be done, include these items and their condition in the internal audits.  Make sure they are segregated and identified.

One of my valuable military aerospace contacts audited a supplier recently and found the housekeeping conditions unacceptable to the point where he shut down the audit; so be warned.

Management Reviews of the quality system (quality management) should be scheduled as rigorously as Material Review Boards or other corporate reviews. Agenda items are assigned to the appropriate levels of authority.

The QUALIDOC MANAGEMENT REVIEW AGENDA is as follows:

A.  Quality Management Representative (QMR) report – status of quality system:

• results from quality system audits (internal and customer)
• corrective and preventive action
• regulatory, industrial and international standard compliance
• customer complaints
• quality objectives
• on-time delivery status

Other authority levels: CEO, Sales Manager, Purchasing Manager, Operations Director, Financial Director

B.  Actions resulting from risk assessment and management (including use of customer-required parts and services)

C. .Quality objectives (strategies for continuous improvement)

D.  Cost of quality system

E. Benefits of quality system

F. Training

G. Customer satisfaction levels (on-time delivery performance and implied needs)

H. Changes in processes and business strategies that could affect the quality system

I am the first to say that I know next to nothing about statistics. I barely scraped through the mandatory course at university which an unfeeling administration required that I take to complete my studies. My interest and spirits heightened considerably when I realised that the course was more about recognising spurious statistics rather than how to compile and use statistics.

The text for the course was a book by Stephen K. Campbell: “Flaws and Fallacies in Statistical Thinking”. In his chapter on ‘Far Fetched Estimates’ I was delighted to read his section on `Statistics Unknowable Because Some Physical Barrier In­terferes with Data Collection’. He quoted a researcher (Seligman) who was anxious to identify the basis for claims made by various newspapers that the rat population in New York City was in the range of 8-9 million. Apparently the only real study on the matter had been conducted in 1949. Under a grant from the Rockefeller Foundation researchers had actually counted rats in a certain area and extrapolated these figures to the whole of New York City. The whole exercise became redundant when one of the researchers remarked to Seligman:

“You can count a rat on the eighth floor, and then another on the seventh floor, and then another on the sixth, but after all, you might just be seeing the same rat three times.”

The anecdote above is a lightbearted reminder that,despite the enthusiasm for measurement by many of my colleagues in the quality assurance world, the categories to be measured, and why they are being measured,  must be defined carefully otherwise the statistics will not produce useful results. I might be preaching to the converted but I know from experience that category errors can lead to expensive misunderstandings of condition and objectives.

Baseline configurations should include contingency time to cover ‘unknowable statistics’; i.e., conditions which can’t be an­ticipated from trends whose antecedents might be ambiguous. No one wants to waste resources being the victim of “spurious statistics” when careful selection of measurement techniques can bring rewarding results.

I’ll close this post with another quotation from Stephen Campbell:

‘The story is told about an event during a trial follow­ing a calamitous wreck at a railway crossing. A railway worker testified that he had signalled vigorously by waving a lantern. His animated demonstration of the vigour with which he waved the lantern greatly im­pressed the jury, which eventually found the railway and its personnel innocent of negligence.

After the trial, the lawyer for the railway commended the man for the effectiveness of his testimony. The rail­way worker, obviously greatly relieved, said: “Thanks. But for a while there I was afraid the other lawyer was going to ask whether the lantern was lit.”

“I am so clever that sometimes I don’t understand a single word of what I am saying.”

― Oscar Wilde, The Happy Prince and Other Stories

I hope I never give that impression with my writing. I sometimes edit my writing to the extent that I come full circle, back to where I started (well, almost).

Unfortunately, in quality system development, Documentation Control still takes far too big a toll on certification assessments and can create chaos if not given sufficient attention. One of my first tasks in any contract is to review the documentation, get rid of anything that is redundant and make sure all the rest is fit for purpose.

It is one thing to identify which documents are needed (and well done if you have managed that) and another to make sure they contain the correct information in terms that are familiar to the reader, and clear if the reader is unfamiliar with the process.

Many years ago I was given a contract by a national railway company to work with retired engineers who were rewriting maintenance procedures for locomotives. Their idea of a pro­cedure was to extract lumps of information from General Motors and other Original Equipment Manufacturers (OEMs) hefty instruction manuals, and plonk them onto sheets of paper ad hoc.

Many painful months later I had managed to persuade them to set out a logical sequence of events with the readers’ needs in mind; e.g., if oil on a hot engine was a safety hazard, then Page 8 was not the place to state it. Warnings, (especially safety) had to be on Page 1, Para 1.

The reader’s needs are paramount. If the reader loses interest or focus you have wasted your time, par­ticularly when you are trying to persuade employees of long experience and standing in the company that changes are needed. You can’t make a big splash when the tide is out. Before distributing the documentation even in the first draft. ask yourself:

• Is the information in the logical sequence which is most beneficial to the reader who is following the instruction?
• What is the preceding connection which the reader may need to know so that he or she can pick up the threads?
• Can the reader discover quickly where he/she is, by page and part?
• Is high order language used to ensure the appropriate level of formality without slipping into obscurity?

Many people are mad about flow charts and are convinced that they are the answer to all information processing needs. Flow charts are excellent when they portray step by step proc­esses which require no assumptions to be made on the part of the reader. Too often I have found the categories have been named without sufficient accuracy, and so the reader is left floundering and making assumptions, which can be awkward, if not disastrous. Flow chart categories must be specific and connections  must be watertight. They should also show responsible authorities within each step; that is, Operating Superintendent, not John Brown. John Brown’s name will show on the Signatories List which is part of the Master Document List.

More critical questions to be asked before developing documentation are:

Who is the reader?

What does he or she need to know?

When do they need to know?

Who approves the first and subsequent drafts and authorises publication?

Who is in charge of the Master Document List? (Usually the Quality Management Representative or delegated authority, shown on the Corporate Organisation Chart)

Who needs access on a Read Only basis?

Are all interested parties identified who might need copies and are the copies controlled or uncontrolled?

It pays to submerge our egos when undertaking this very important task. We can’t afford to imitate Oscar Wilde; too much is at stake.

First published in American Society of Appraisers, MTS Journal 4th quarter, 2017

QUALITY, LIKE BEAUTY, IS IN THE EYE OF THE BEHOLDER

Many years ago I visited a small company which was the sole-source subcontractor for a critical military winter item. As I chatted with them about their business, I asked how they recorded their nonconformances. The Managing Director waved towards a couple of sad-looking photographs of welds on the wall, one marked ‘Before’ and one ‘After’.  I asked the usual questions about qualification to Welding Board standards and he observed that “our employees are trained on the job and they {the products} meet customer requirements”. Then I asked him how they managed Process Control, which comprised manufacturing the items in one piece, after which they were painted. He said they hired local people to use them as seats on the beach for a specified time. If the paint wore off before that time expired (presum­ably where Inspection and Test came in) they were perceived as ‘nonconforming’. (I swear this is a true story.)

The product met their customers’ specifications.  They had good contracts and obviously their customers (ex­acting military) were happy with the results.

I make this point because with the current passion for data collection and numbers we are inclined to forget the judgement of experienced people who are responsible for determining the acceptable level of quality for products and services in their own environment. They are also responsible for training the apprentices and other novices coming into the industry. The numbers mean nothing unless they derive from well-defined categories of instruction; in other words, vocabulary.

The word “quality” can be attached to an adjective such as “excellent” or “poor”, for example, depending on the perception of the person evaluating the object, service, or other condition.  For quality management, however, the word is attached to specific values such as customer requirements, international quality management and industrial standards. In aerospace it might apply to safety and economical performance, rather than luxurious cabin surroundings. In marine freight forwarding it could apply to the condition of pulp and paper travelling from the mills to the rail car and then the ship.

The word “quality” has no meaning used on its own without criteria for acceptance, which must be in the eye of the experienced beholder or decided against a value set within discrete environments.