HHS Materiel Management (Supply) Review Guide
This guide is offered as a best practice for conduct of a review of Federal supply operations.
Table of Contents
Chapter 1 Management Indicators and Materiel Management (Supply) Performance Indicators
Evaluation of Management and Materiel Management (Supply) Performance
Materiel Management (Supply) Performance IndicatorsChapter 2 Materiel Management (Supply) Management Review Memorandum and Request for Data
Exhibits
Exhibit 2-1 Sample Materiel Management (Supply Management Review Memorandum)
Exhibit 2-3 Materiel Management (Supply) Activity Profile For FY
Exhibit 3-2 Materiel Management (Supply) Performance Indicators Worksheet
Exhibit 4-1 Report on the Materiel Management (Supply Management) Review of the (Name of Activity)
FOREWARD
This guide was developed to assist in organizational assessments of the performance of supply support throughout the Department of Health and Human Services. The performance indicators contained within are consistent with the results-oriented focus of the Government Performance and Results Act. Successful performance of the supply function will contribute to successful program outcomes. This guide also may be used in determining the adequacy of management controls under the Federal Manager's Financial Integrity Act and the Office of Management and Budget Circular A-123. It offers a common means of communicating about the strengths and outstanding performance, as well as weaknesses in the HHS logistics system. Areas requiring attention can be identified and references to specific policy and procedures are useful for achieving corrective action.
We welcome your suggestions concerning this guide, to further enhance the quality of logistics throughout HHS. Please call 202-690-5663, to make recommendations to the logistics policy staff of the Office of Grants and Acquisition.
Terrence J. Tychan
Deputy Assistant Secretary for Grants and Acquisition Management
INTRODUCTION
Purpose of the HHS Materiel Management (Supply Support) Review Guide This Materiel Management (Supply) Review Guide was developed to assist in assessing the performance of the HHS materials management (supply) activities in a structured and consistent manner in order to identify opportunities for improvement; to determine compliance with Federal and HHS regulations and policies; and to assure that adequate management controls are in place as required by the Federal Managers' Financial Integrity Act (FMFIA).
Purpose of HHS Materiel Management (Supply) Activities The purpose of each HHS materials management (supply) activity is to provide the supplies and materials needed to carry out the mission of the organization served by the activity at the right time and the right place, and to do so in a manner that reflects the appropriate balance between cost effectiveness and responsiveness.
For the purpose of this guide, a materials management (supply) activity is defined as an OPDIV organizational component that has responsibility for maintaining an inventory of supplies or materials (an asset for which accountability is required). In general, the term "supply activity" is intended to include central and regional storage facilities with substantial inventory values and levels of activity. However, the guide can also be used to review smaller materials management (supply) activities or sub-systems of a central materials management (supply) activity.
BACKGROUND
Materiel Management (Supply) Principles
The management principles of the materials management (supply) activity must be developed in accordance with Federal and departmental regulations, policies and procedures. In addition they must be clearly stated, developed with the participation and support of senior levels of management and communicated to all operating officials and staff.
The principles should include:
Providing a systematic, cost effective approach to materials management (supply) management.
Having the proper supplies and materials on hand when needed.
Consolidating and coordinating the buying, storage and distribution functions to get supplies to the users at the lowest possible cost.
Paying the lowest possible prices for supplies and materials, provided that the products acquired meet the minimum quality standards set by the organization.
Operating with the minimum investment in inventories, warehouse space, personnel and other assets, consistent with the organization's needs.
Operating with the maximum possible efficiency (e.g., increasing turnover, eliminating unnecessary items from inventory, reducing delivery times, reducing stock outages, eliminating unofficial inventories, minimizing transportation and storage costs).
Having knowledgeable and effective materials management (supply) and distribution facility staffs.
Developing inventory control and accountability procedures to prevent loss or unauthorized removal of stock from storage locations.
CONTENT OF THE GUIDE
This section presents a brief synopsis of the Materiel Management (Supply Support) Review Guide, the purpose of each chapter, and the relationship of the material in each chapter.
Chapter 1 describes the management and performance indicators used to measure the quality of the materials management (supply) organization's activities, the productivity of its staff, the effectiveness of its managers, the appropriateness of its organizational placement and the adequacy of its checks and balances. It also describes the standard which each materials management (supply) activity is expected to meet for each of the indicators.
Chapter 2 shows a sample memorandum announcing the start of the review and describing the review topics to be covered. It also requests the materials management (supply) activity under review to provide data not otherwise available to the review team. Those data include, among other things, information on significant management and performance indicators.
Chapter 2 includes the Materiel Management (Supply) Activity Profile which is to be completed by the materials management (supply) activity to describe the basic characteristics and scope of its activities. The Chapter also includes the form to be used by the review team to record its analysis of the information submitted by the materials management (supply) activity.
Chapter 3 contains the On-Site Review Checklist which sets forth the questions which can best be answered on site for each of the major functions in the materials management (supply) process, e.g., inventory management, storage and distribution, management of controlled substances. It also contains a worksheet to assist the reviewers in conducting a review of the materials management (supply) activity.
Chapter 3 also presents the reviewers with information on how to select a sample of materials management (supply) records such as inventory adjustments, receipts for items received and shipped, inventory locator records, and shelf life records, so that their review has a high probability of being representative of the overall population of the organization's activities. The objective is to choose the smallest number of records that is representative of the total population in order to maximize the efficiency of the review process. Chapter 3 discusses simple random and stratified random samples. It also addresses judgmental samples and when and how to use them.
Chapter 4 describes the overall summary of the study results. It requires the reviewer to make certain judgments about the quality of the materials management (supply) process; to assess the adequacy of the materials management (supply) organization, staffing, management controls, and operating practices; and to provide an overall numerical rating for the materials management (supply) activity. The judgments and assessments build upon the results of the On-Site Reviews, discussed in Chapter 3. Chapter 4 also describes the content of the Review Report and contains an explanation of the scoring system. At the time of the exit briefing, a draft report may be provided to the head of the materials management (supply) activity being reviewed. The team should prepare the final report within 30 days of the exit briefing.
The Guide contains numerous references to materials management (supply) requirements based on existing Federal, departmental and OPDIV policies and guidelines.
References include the following:
Federal Property Management Regulations (FPMR)
HHS Logistics Management Manual (LMM) General Guidelines and Standard Operating Procedures
Materiel Management (Supply) Performance Measures for the Department of Health and Human Services
MATERIALS MANAGEMENT (SUPPLY) REVIEW PROCESS
This guide may be used and adapted for formal or informal assessments of how well a materials management (supply) function is performing its mission. The reviews should focus on the activities described above, including an assessment of the quality of the materials management (supply) process, organization and staffing, and management controls. The review should also analyze significant management indicators and materials management (supply) activity benchmarks that tend to highlight potential or actual successes and deficiencies. The review process is designed to minimize the disruption to the operations of the activity being reviewed.
If a formal review is conducted, any potential or actual successes or deficiencies identified during the review should be cited in the Review Report. Those findings will be discussed with the head of the materials management (supply) activity during the exit interview, if the review is performed by another organization. If an organization is reviewed by another, the organization being reviewed will have the opportunity to identify areas of the report with which it disagrees and, after discussion, the review team may make corrections where appropriate. A copy of the final report should then be provided to the OPDIV Executive Officer and the head of the materials management (supply) activity.
ALTERNATIVE INTERNAL CONTROL REVIEWS
An external or self assessment performed by an organization in accordance with the guide-lines set forth in this document, will qualify as a management review under OMB Circular A-123. Evaluations of an administrative function (in this case the materials management (supply) management activity) determine whether adequate management controls are in place, and identify any areas for improvement and material weaknesses. In addition to an on-site evaluation, recently issued audit reports, management studies and other materials can be used to help or to make this determination.
Reports on these assessments must be clear, concise and thorough enough to convince reviewing officials that a comprehensive evaluative examination has been made of the management control area. This means that in developing its findings, the reviewer should use data collected during review sampling and illustrations based on actual observations, to the maximum extent feasible. An explanation of the problems identified should also elaborate on any impacts that such problems may make on the operations of the organization as a whole.
The majority of problems found will be administrative in nature. However, certain problems, due to the significance of the impact they make, may be classified as material weaknesses. This designation requires a judgment by agency managers concerning the relative risk and the significance of the deficiency. Problems which become officially designated as material weaknesses must be reported to the President and the Congress by the Secretary. Corrective actions are then identified and they should be considered a priority.
CHAPTER 1 MANAGEMENT INDICATORS AND MATERIALS MANAGEMENT (SUPPLY) PERFORMANCE INDICATORS
Certain management and materials management (supply) performance indicators will be used as standards to measure the:
Effectiveness of management
Appropriateness of its organizational placement
Adequacy of its management controls
Quality of the organization's materials management (supply) activities
Productivity of its staff
Management indicators are those factors that describe the ability to manage the materials management (supply) function and the people in it. For example, management indicators are such factors as the adequacy of policies and procedures; placement of the materials management (supply) activities within an organization; span of control; productivity; responsiveness to the needs of customers; and management controls; and operating practices. Materiel Management (supply) performance indicators, on the other hand, are standards by which to measure the results of materials management (supply) operations (e.g., fill rate, inventory turnover rate, inventory adjustment rate, and accuracy rate.)
EVALUATION OF MANAGEMENT AND MATERIALS MANAGEMENT (SUPPLY) PERFORMANCE
This chapter describes the objective of each function or activity measured by the management and materials management (supply) performance indicators. Using these directives, the reviewers will evaluate the materials management (supply) activity's management performance and record their findings.
A separate Materiel Management (Supply) Performance Indicator Worksheet is included as Exhibit 3-2. Reviewers will use it to record the materials management (supply) activity's actual performance against the standards the activity has set for itself in eight separate materials management (supply) performance categories. Functions that do not meet the acceptable standard are considered anomalous, indicating a potential or actual deficiency that should be included in the findings.
Management Indicators
Organization and Management
Policies and Procedures
Objective
To assure that appropriate operating policies and procedures are in effect, and are available to and understood by the staff of the materials management (supply) activity.
The responsibility for assuring the availability and understanding of the appropriate operating policies and procedures, including HHS and government-wide requirements, as well as any necessary OPDIV and local supplements thereto, rests with the management of the materials management (supply) activity and the OPDIV in which the activity operates.
The amount and quality of internal guidance provided at the local, individual office, or branch level is an important indicator of the level of detailed direction believed needed by the office's management staff.
Copies of internal documents provided in response to the Request for Data (Exhibit 2-2) should be reviewed for adequacy and necessity. They must not conflict with higher level directives.
Staffing and Training Objective
To develop staffing levels and expertise that meet the needs of the OPDIV, including personnel properly trained to carry out functional responsibilities.
The ultimate effectiveness of a materials management (supply) activity is, for the most part, attributable to the quality and commitment of its staff. Certain criteria may be used to identify potential problems among the staff. They are performance evaluation, staff turnover, overtime, grade distribution, and span of control. These criteria are discussed below.
Staff Turnover
The team will review the average annual turnover rate for the materials management (supply) organization work force. High employee turnover tends to diminish quality of materials management (supply) operations (or services), reduce productivity, and increase training and orientation expenses.
Grade Distribution
The Civil Service GM, GS, and wage grade levels for materials management (supply) personnel should be appropriate for the type of work being done. The grade structure should allow opportunities for employee growth without being top-heavy.
Training Materials management (supply) staffs should be well qualified and sufficiently trained to carry out their responsibilities. Insufficient training may lead to poor performance; customer dissatisfaction; and, faulty implementation of laws and regulations. The review team should determine the types of training completed by the employees of the materials management (supply) activity and whether the training is appropriate for the duties and levels of responsibilities of the employees. Organizational Placement, Structure, and Responsibilities Objective
To establish the management of materials management (supply) activities at an appropriately elevated organizational level with a competitive grade structure to attract a highly skilled, professional, and motivated staff. The materials management (supply) management function must be placed sufficiently high in the organization which it serves to assure continuing visibility from, and access to, senior management within the organization.
The review team should consider whether management of the materials management (supply) activity being reviewed has been placed at an appropriate organizational level to adequately monitor and review the materials management (supply) management practices, materials management (supply) system configuration, and materials management (supply) costs. If not, that should be reflected in the Report of Findings.
Materiel Management (Supply) Facilities Objective
To develop materials management (supply) facilities that optimize the use of financial, personnel, and physical resources while maintaining the required level of customer service.
The review team will determine whether the facilities are located to maximize the return for fixed cost investments with the least effect on customer service.
Management Information Systems Objective
To develop a capability to retrieve, assemble, and disseminate materials management (supply) management information required for use in sound decision making by personnel in operations and management.
Management Reviews and Improvements Objective
To carry out a program of periodic management reviews of materials management (supply) activities, including adequate follow-up to assure implementation of any recommendations resulting from the reviews.
Analysis of prior management reviews provides a basis for selecting specific areas that the review team may reevaluate to determine whether problems found earlier still exist. The effectiveness of corrective measures can be determined. Long-term existence of the same or similar problems indicates a deficiency.
If copies of any previous management review or audit reports have been provided in response to the Request for Data, Background Information (Exhibit 2-2), the team should assess them to determine whether any of the recommendations involve areas currently under review. If so, the team must review those areas and note compliance/noncompliance with the recommendations.
Cost of Materiel Management (Supply) Operations Objective
To determine if the cost of materials management (supply) operations minimizes net costs per issue while maintaining the expected level of performance and customer service.
Effective Use of Labor Objective
To ensure that the staff structure provides the expected level of performance in response to customers' needs without requiring excessive overtime; and to ensure that the physical layout of the materials management (supply) activity and other working conditions support the efficient use of labor.
Pricing Strategy and Practices Objective
To provide materials management (supply) support at the most economical cost by establishing item prices which accurately reflect the costs of acquisition, storage and distribution by the materials management (supply) activity.
Customer Service and Satisfaction Objective
To apply standard measures that will enable managers to determine how effectively materials management (supply) operations meet organizational objectives. Such measures include but are not limited to: fill rate, inventory turnover, inventory accuracy, and responsiveness to customers' complaints.
Management Controls
Accountability Objective
To have a chain of custody identifying persons designated as answerable for control of each materials management (supply) item purchased, from its receipt through its final issue and use or consumption.
Financial Accounting of Supplies Objective
To utilize accounting procedures that produce accurate records of the materials management (supply) inventory, and provide a check and balance of the asset accounts.
Operating Practices Materiel Management (Supply) Acquisition Strategies Objective
To obtain the maximum cost reductions (e.g., quantity discounts) available through consolidation of requirements, standardization, and other cost reducing techniques.
Receiving Objective
To have effective procedures to certify incoming shipments by verifying the correctness of the order, substantiating the disbursement of funds and establishing an inventory control record for the receipt of stock.
Inventory Management Objective
To stock and manage the mix and quantity of consumable materials management (supply) items which satisfy the demands of the OPDIV's personnel, taking advantage of the economies of scale whenever possible; and to promptly determine when new items should be added to inventories and when items no longer needed should be removed from it.
Storage and Distribution Objective
To operate a responsive cost effective system for the receipt, recording, storage, and distribution of materials management (supply) inventory stocks; to safeguard the inventory from damage, loss or theft; and to establish procedures that provide for efficient use of resources in all phases of the system.
Management of Controlled Substances, Sensitive Supplies and Hazardous Materials Objective
To maintain strict accountability of controlled substances, sensitive supplies, and hazardous materials from the time of receipt, through storage and distribution, including safeguards against theft, abuse, misuse, or loss. To use procedures for the receipt, storage, issue, and disposal of controlled substances that are in strict compliance with Federal regulations.
Shop Stock Inventories Objective
To keep an on-hand stock of consumable, high use, low cost items that meet the quantity levels developed for shop stocks and have procedures that will eliminate misappropriation or unauthorized use of materials management (supply) items in the shop or working stock areas.
Emergency Preparedness (Applicable only to the IHS Supply Service Center, Perry Point, MD) Objective
To have effective procedures for the procurement and distribution of materials management (supply) items, during periods of national emergency, that are in compliance with Federal emergency preparedness regulations.
MATERIALS MANAGEMENT (SUPPLY) PERFORMANCE INDICATORS
The managers of each materials management (supply) activity should monitor the activity's performance using the indicators described in this section together with any additional performance indicators considered relevant by the managers of the materials management (supply) activity. Each materials management (supply) activity should establish appropriate standards to be met for each performance indicator. The reviewers will identify each instance where an appropriate standard has not been established for the performance indicator described in this section.
In order for the reviewers to determine whether the standards are appropriate, the managers of the materials management (supply) activity must be able to explain and document the basis for the standards. The reviewers will carefully consider any unique characteristics of the materials management (supply) activity's customers, location, type of stock, etc., in reviewing the standards established by the materials management (supply) activity.
In addition, the reviewers will consider any failure to achieve the established standards (in the absence of an acceptable explanation) as an anomaly that requires corrective action.
Fill Rate
Fill Rate is the percentage of line items filled from the total number of line items ordered against stocked material.
For customers, fill rate defines the reliability of their materials management (supply) support and influences materials management (supply) costs. When fill rates are low and customers can not get what they need, they increase the size of their shelf stocks as a safety measure against future fill rate problems. Nurses, for instance, may hoard supplies they need to serve their patients. Hoarding increases the money activities spend on supplies.
In addition, low fill rates force activities to buy critical supplies from alternate sources. Since those buys are usually in much smaller quantities than materials management (supply) activities normally purchase, their unit costs are higher. Finally, buying from alternative sources creates more work and higher costs in processing, purchasing, and invoice payment documents.
The fill rate for an activity is determined by the following ratio:
Fill Rate = (Total Number of Line Items Filled from On Hand Inventories / Total Number of Line Items Ordered) X 100
Issues of partial quantities from stock are not counted in the fill rate calculation. In addition, requests held for processing pending receipt of material do not count as filled items.
A materials management (supply) activity's fill rate is one of the most common measures used to demonstrate the level of materials management (supply) support it provides to its customers. A standard fill rate should be set as a benchmark by every materials management (supply) activity to indicate the minimum acceptable level of support that the customers can expect from the materials management (supply) system. By setting and meeting a fill rate goal, the materials management (supply) staff will be able to provide its customers with an acceptable level of service at a reasonable cost.
Order Accuracy Rate The order accuracy rate is the percentage of total shortages or overages reported by the customer for a given period. Shortages occur when items are indicated as packed with an order do not arrive at the customer's location. Shortages can be caused by errors in the picking and packing process. They are measured as the ratio of lines short of the ordered quantity to total lines delivered to customers. Overages occur when items are delivered to customers that have not been ordered by them. Overages include both deliveries of items in quantities that exceed those that were ordered and items that were not ordered by the customer but were included with the customer's order. Overages can usually be ascribed to a counting error during the picking process.
The order accuracy rate for a materials management (supply) activity is determined as follows:
Order Accuracy Rate% = (Total Line-Item Overages and Shortages Reported / Total Number of Line Items Issued to Customers) X 100
The damaged shipment rate for a materials management (supply) activity is determined as follows:
Damaged Order Rate% =( Total Number of Line Items Damaged / Total Number of Line Items Issued to Customers) X 100
Calculate the following:
Total Labor Cost per Activity = Total Labor Cost in the Period / Total Number of Transactions Performed in the Period
Inventory Accuracy Rate Inventory accuracy is a measure of the correlation between the accountable record (manual or automated materials management (supply) records) inventory and the physical inventory on hand. Inventory accuracy is generally measured through physical count surveys. Stock locator records are randomly matched to physical locations, and vice versa. The objective is to have on location and in inventory, those items that are being carried on the books. The inventory accuracy rate indicates how well receipts to stock locations and issues from stock locations are recorded in the accountability records. For example, if the materials management (supply) records indicate that 27 widgets are in inventory in location A43C and an inspection indicates 27 of those widgets are physically located in A43C, the inventory is accurate. A high inventory accuracy rate indicates that materials management (supply) records are adjusted when items are picked from inventory. A low inventory accuracy rate could indicate either that materials management (supply) records are not being updated properly or that warehousing activities were not recorded.
The inventory accuracy rate for a materials management (supply) activity is determined as follows:
Inventory Accuracy% = (Number of Accurate Quantities in Locations Inventoried / Total Locations Inventoried) X 100
Inventory adjustments are manual changes to the inventory balance in the materials management (supply) records. Inventory adjustments can be made at any time, but are most commonly made after physical counts of stocked items are compared to recorded data. Inventory adjustments can take the form of gains or losses, both of which can indicate problems in inventory management. Inventory losses may occur as a result of misidentifying items when they are placed in stock, theft, damage or destruction from water or fire, or unrecorded issues from stock. A loss may also occur due to expiration of an item's shelf-life or obsolescence of items due to new technology. Inventory gains can occur as a result of misidentifying or not entering items when they are initially placed in stock, or finding items previously recorded as losses. An excessive quantity of inventory adjustments can indicate that either physical security or quality control needs improvement. The inventory adjustment rate for a materials management (supply) activity is determined in two steps: Calculate the sum of the dollar value of inventory gains (IG) and the absolute dollar value of inventory losses (IL) for the period. The result is (IG + IL).
Calculate the following rate:-
Inventory Adjustment% = ((IG + IL) / Average Value of Inventory) X 100
The inventory turnover rate indicates the number of times that the average item in stock has been replenished during the period of observation (usually a year). For example, if the value of the average inventory on hand is $1 million and total demand for a year is $4 million, then the inventory turnover rate in this case is 4 times. A low turnover rate is indicative of either a long procurement lead time, volume purchases or low usage of stocked items. Unless an item has a long lead time, low turnover is not desirable, even if quantity discounts are offered for volume purchases. The cost of carrying large quantities of stocks could exceed the savings realized from quantity discounts. A higher turnover rate could indicate more efficient use of available resources, provided customer orders are being filled and the prices of stocks purchases are acceptable.
The inventory turnover rate for a materials management (supply) activity is determined by the following ratio:
Dollar Value of Total Annual Demand / Dollar Value of Average Inventory On Hand
The stockout rate for an activity is determined by the following ratio:
(Number of Stocked Items with Zero Balance / Total Number of Stocked Items) X 100
CHAPTER 2 MATERIALS MANAGEMENT (SUPPLY) MANAGEMENT REVIEW MEMORANDUM AND REQUEST FOR DATA
Should circumstances warrant review of another organization, prior to the start of a review, a Materiel Management (Supply) Management Review Memorandum (see Exhibit 2-1) should be sent to the head of the activity to be reviewed, with a copy to the executive to whom that activity reports. The memorandum should state when the review will begin and what it will cover.
The memorandum should explain how the review will be conducted and the method of communicating review results, and should include a request for essential data (see Exhibit 2-2) that is needed by the reviewers prior to the start of the on-site review. The Request For Data, Exhibit 2-2, helps meet several objectives. First, it asks for background information which describes the materials management (supply) activity and for information on selected operating characteristics. This data will provide an understanding of the size and scope of the organization's materials management (supply) operation and staffing characteristics, and will help in selecting meaningful samples for the Materilas Management (Supply) Management Review.Second, the data provides information which, together with information obtained during the on-site review, will enable the reviewers to measure the performance of the materials management (supply) activity against predetermined management and materials management (supply) performance indicators. Unless otherwise stated, all of the information requested in the Request For Data is for the most recently completed fiscal year. All of the data should be available at the materials management (supply) activity, however some data elements may not have been recorded. Personnel at the materials management (supply) activities should be encouraged to maintain records of the requested information so that it will be available for their analyses in determining management strengths and weaknesses. The Materiel Management (Supply) Activity Profile, Exhibit 2-3, may be requested to be submitted annually to the OPDIV. At the time of a materials management (supply) activity review, it will only be necessary to provide revisions that have occurred since the most recent annual profile was submitted. The Analysis of Responses to Request for Data, Exhibit 2-4, provides a methodical basis for the reviewers to record whether or not the materials management (supply) activity's responses are satisfactory. The analysis will be used to identify subjects requiring further review during the on-site review. In addition, the reviewers should calculate the performance indicators from information provided in the Request for Data and record the indicators on the Materiel Management (Supply) Performance Indicators Worksheet, Exhibit 3-2. Remaining indicators will be calculated during the on-site review. The information described above is critical to the review process. If feasible, the team should request the required information 120 days before the on-site review, and it should be received at least 60 days before the on-site review begins.
CHAPTER 3 ON-SITE REVIEW
I. Purpose
The purposes of the on-site review are to:
Discuss any responses to the Request for Data (Exhibit 2-2) that were not clear or complete
Review samples of documentation to confirm or verify the satisfactory implementation of required materials management (supply) practices
Calculate and record materials management (supply) performance indicators.
Supplement the information obtained in response to the Request for Data with information best obtained through first hand observation and personal interviews.
Discuss potential opportunities for improving the operations of the materials management (supply) activity.
II. Review Checklist
The on-site Review Checklist, Exhibit 3-1, provides the basic requirements for sampling documentation and obtaining other relevant information during the on-site review. The results of the on-site documentation review, together with the information obtained prior to or during the on-site review in response to the Request for Data will be used to develop the Review Summary and Report (See Chapter 4). The on-site review normally begins with an entrance interview with the head of the activity or a designated representative. The team leader should introduce the team, briefly describe the purpose and methods of the review, confirm working arrangements, and meet the agency point of contact and provide a listing of the files and documents needed by the review team. Any problem areas identified during the team's analysis of the responses to the Request for Data should be addressed. The activity should be asked if it has any concerns or special items that it wants the team to know about or to review. The checklist contains questions for each major functional area of a materials management (supply) activity. The checklist provides a space indicating Yes, No, or N/A (for "not applicable") next to the questions. From time to time the checklist will be revised to reflect changes in regulation and policy.
III. Statistical Sampling Techniques For each question in Exhibit 3-1, On-Site Review Checklist, which indicates the need to review a sample of the materials management (supply) activity's documentation/records or to interview a sample of the materials management (supply) activity's staff members, the review team will select a representative sample of the materials management (supply) subjects under review and use that sample to analyze the activity's performance. Sampling economizes the collection and measurement of data. Ideally, the sample will be a scale image and thus representative of the population from which it was selected. (A population is a set of objects or individuals ("subjects"), about which it is desired to ascertain characteristics.) The objective in sample selection is to choose the smallest number that still represents the total population. While we can never be absolutely certain that a sample is exactly representative of the population, we should select a sample that has a high probability of accurately representing the overall population. Simple random samples, stratified random samples and judgmental samples are discussed below.
The simplest, and often most appropriate, sampling technique is the random sample. A simple random sample is one in which every element in a population has an equal chance of being selected. A simple random sample is appropriate for elements that are similar in nature (homogeneous). Two main steps must be followed in performing a random sample: determining a sample size, and selecting the sample. Determining Sample Size
The review team will want results that are acceptably close to the total population's performance. To ensure such results requires a tradeoff between accuracy and cost. Large samples yield more accurate statistics than small samples but cost more to collect and process. The sample size the team chooses for its review then, directly affects both the cost of performing the review and the accuracy of the results.
Selecting the Sample
After the review team determines the size of the sample, it must select the sample subjects in a way that is as "random" as possible. That means it must follow a procedure that reduces any natural and human bias. It is essential that the sample selection procedure followed be designed so that subjects are not selected for review on the basis of expediency or other biasing factors. The team must make every attempt to locate and review the subjects selected for the sample. If, for any reason, a subject is missing or is otherwise not available, the team should select a replacement randomly.
Occasionally a sample selected by a simple random method will have some unique characteristic; for example, all items in a sample might be packed for shipping by the same individual. If the population is truly homogeneous, that characteristic should make no difference. If, however, you have information that the population contains subgroups that have distinct characteristics of interest, the population is heterogeneous and you will obtain better results from a stratified random sample. A stratified random sample differs from a simple random sample in the way the sample is selected. To select a stratified random sample, the review team selects elements in the sample to reflect the relative importance of subgroups (or strata) that it has identified. The team then randomly draws sample elements in proportion to each subgroup. The review team should not create a stratified sample unless the characteristics of the subgroups are germane to the study.
C. Judgmental Samples
For those areas that the reviewers iden
tify as potential problems additional sample items should be selected on a judgmental basis to further explore particular areas of concern. The findings from these additional reviews should be used to confirm or reject the initial indications of problems. In addition, an agency may request inclusion of a specific subject that it would like to have the team evaluate. This review would be done in addition to samples selected by the team.
The Materiel Management (Supply) Performance Indicator Worksheet (Exhibit 3-2) is used to record the materials management (supply) activity's actual performance against the standards the activity has set for itself in eight separate materials management (supply) performance categories. Prior to on-site review, the reviewers will calculate the performance indicators from information provided by the materials management (supply) activity in the Request for Data. Remaining indicators will be calculated on site from samples of the materials management (supply) activity records. Failure to meet the expected standard for any Materiel Management (Supply) Performance Indicator must be discussed in the Report of Findings. The failure of a materials management (supply) activity to establish any of the required standards automatically results in the activity being rated as not meeting the standard for that specific indicator. In addition, such a failure must be discussed in the Report of Findings, and in the absence of an explanation satisfactory to the reviewer, must be considered as an administrative weakness.
V. Discussion of Potential Findings
Questions in Exhibit 3-1 for which the reviewer has not received a satisfactory response should be discussed with the head of the materials management (supply) activity during the exit interview.
At that time, the review team may present a summary of preliminary findings. Reviewers should attempt to consolidate closely related deficiencies into a single finding. Only under unusual circumstances should any significant item appear in the final report that was not discussed during the exit interview. This time should also be used to express the team's appreciation for the support and assistance provided by the materials management (supply) activity.
CHAPTER 4 REVIEW SUMMARY AND REPORT
Chapters 1, 2 and 3 of this Guide, address the use of management and materials management (supply) performance indicators as tools to assist the reviewer in assessing the quality of the materials management (supply) operations. This chapter summarizes the results of the review performed in accordance with Chapters 1, 2 and 3. It helps the reviewers to measure objectively the overall quality of the materials management (supply) process. It also offers guidance in identifying potential and actual deficiencies and in bringing those deficiencies to the attention of management. Finally, it describes the types of information to be included in the Review Report and provides a standardized format which may be used.
The Report is the final product of the review. It provides the results of the review to the head of the organization and chief of the materials management (supply) activity reviewed. The Report consists of a summary of the scope of the review, findings, detailed support of the findings and a scoring summary. Each area reviewed should be described in sufficient detail so that the report becomes a stand-alone document for higher management use and can be easily understood by those not familiar with the details of the review process or this Guide. To assist the review team in preparing the report and to help standardize the report's content, a sample format for basic information for each area to be covered in the report is shown in Exhibit 4-1. That exhibit describes each area of the review and the team need only insert its findings where appropriate. While each finding should be discussed only briefly, sufficient information should be provided to make the finding understandable to those not directly involved in the materials management (supply) process. The format provided in Exhibit 4-1 should be tailored by the team leader for the specific circumstances of each review. The Report should briefly explain the overall findings in each functional area. If the findings in a functional area are disparate from the standard, this disparity should be noted. All significant findings should be included in the report and sufficiently explained so that the reader can understand the basis for the score. The review team should not include recommendations in the report; the head of the materials management (supply) activity reviewed is responsible for devising whatever corrective actions may be required. The report represents the collective judgment of the review team members.
A. Scoring
On the basis of the cumulative results of the analysis of the responses to the Request for Data, the on-site review and materials management (supply) performance indicators the reviewer calculates an overall score. In the "Scoring Summary," Exhibit 4-2, the scoring process to be followed by the reviewers is presented in detail. Also presented is the method for each of the rating categories in order to develop a single overall score for the organization as a whole. Once the results are posted, the points should be totaled and an overall score for the materials management (supply) activity should be computed. By comparing the organization's overall score against the rating methodology, which ranges from 0 to 100, the reviewers will be able to determine whether the materials management (supply) activity meets, exceeds or falls below the standards.
B. Draft Report
A draft copy of the Report should be forwarded to the Head of the Materiel Management (Supply) Activity for review and comment prior to issuance of the final report. Comments from the materials management (supply) activity must be addressed by the review team. If warranted the final report may be revised. When appropriate an attachment discussing the differences between the materials management (supply) activity comments on the draft and the final report should be added.
C. Issuance of the Final Report
The final report should be issued by the reviewing organization within 30 days after the completion of the site visit. At that time, the reviewed activity will have 30 days to submit a corrective action plan, if required. The final report should be sent as an enclosure to a transmittal letter. That letter should provide an overall quality score for the materials management (supply) activity and should include a discussion of any major discrepancies that the review team found including any discrepancies identified as "material weaknesses" for management control purposes. It should also reiterate the requirement for the materials management (supply) activity to prepare a corrective action plan in response to the team's finding.
All rights reserved.