SCORmark benchmarking FAQ's

Frequently Asked Questions

To help you secure the most value from your SCORmark benchmarking effort, please review the following list of frequently asked questions which cover topics ranging from getting started and understanding the data and metrics, to learning about APQC and reviewing the methodology.

SCORmark is a member-only benchmarking portal available for all current members of the Supply-Chain Council (SCC).  If your organization is not presently a member, please visit the SCC web site to learn more about membership benefits, including access to the SCORmark service.

SCORmark benchmarking was built to directly support the Phase 2: Analyzeactivities of the SCOR®-Model (Supply-Chain Operations Reference Model) implementations. All terms, definitions, metrics and reporting integrate within the SCOR 8.0 Model for ease of use and consistency in benchmarking.

Getting Started

1. Do I need to use my Supply-Chain Council login to access the survey?
    

   No, your membership log-in is not needed to complete a SCORmark survey though your current membership status may be verified prior to data validation and report delivery.  SCORmark is offered as a member-benefit for current Supply-Chain Council members. To learn more about joining as a member, please visit www.supply-chain.org.

    

2. Who do I contact if I need help or have questions?
    

   APQC provides a Help Desk to answer your questions.  You may e-mail a question to scormark@apqc.org or call 800-776-9676 (or outside the United States +1-713-681-4020).  The Help Desk is open from 8AM - 5PM Central time zone, Monday-Friday.

    

3. What is the process for getting started?

    

   1. Confirm that your organization is a current member of the Supply-Chain Council.

    

    

   2. Visit the SCORmark benchmarking web site to submit your data, preview the survey and metrics, and review the terms and instructions.

    

    

   3. Review the list of metrics.  To assess performance in all attribute areas, the Level One metrics are required.  Review and select the Level Two metrics that are most applicable to your supply chain competitive strategy.

    

    

   4. Assess the key supply chain performance priorities for your business entity and assign the ratings of Superior, Advantage and Parity, based on the SCOR methodology (Phase 2: Analyze).

    

    

   5. Gather and submit the data required based on the specific metrics you selected.  Through an automatic filtering tool, you will only be asked to answer the questions that are needed to calculate the benchmark for the measures you select.

    

    

   6. Upon submission, APQC will validate and generate a benchmarking report for your organization. (Note: APQC may contact you to clarify validation issues.)

4. How long will it take to complete a SCORmark survey?
   
   For organizations that are currently in the Phase Two: Analyze portion of a SCOR implementation, you should have already completed your prioritizations and identified internal data sources.  The time to complete the survey will vary based on the number of metrics and ease of access to the data required.  Organizations have completed the benchmarking in as few as 2 hours with the average taking 8-10 hours.

5. What tools are available to aid me in completing the benchmarking?
   
   SCORmark benchmarking integrates directly into Phase Two: Analyze of SCOR, so the benchmarking approach and terms taught in the SCOR Framework and Implementation training courses will correspond to the benchmarking portal.  In addition, the web site contains links to a common glossary of terms, and offers a preview of the survey and sample report.  APQC also provides a SCORmark Help Desk for specific questions and can be reached via e-mail to scormark@apqc.org.

Benchmarking Scope & Scor Linkages

1. What are the benefits of using the SCORmark benchmarking portal?

With SCORmark benchmarking, SCC members have access to one common resource to compare and contrast supply chain performance worldwide using SCOR Model metrics and to create an individual roadmap for supply chain competitive success, all at no additional fee.  Some advantages include the ability to:

• dramatically shorten benchmarking cycle time,
• access data that has traditionally been costly to obtain,
• align performance improvement plans and implement changes quickly, and
• receive a custom benchmarking report targeted specifically to how SCOR measures supply chain performance.

2. Why should I care about benchmarking and the SCORmark service?

To make timely, well-informed decisions organizations must have a window into the operational health of their business. Effective benchmarking is necessary for organizations to proactively monitor trends and gain early warning of any constraints or excesses in the real-time supply chain.  To support SCOR implementations it is critical for an organization to not only identify its metrics but more importantly to be able compare with other organizations to understand and give meaning to their performance in a given area.  The resulting reports help an organization set the direction to develop its supply chain strategy.

The SCORmark service was developed specifically to support organizations implementing the SCOR model with the metrics, language, and reporting designed to seamlessly integrate into the SCOR model.  It is being offered as a new member-benefit in support of supply chain benchmarking and to establish an open standard for performance measurement.

3. How was the SCORmark benchmarking portal developed?

The portal is the result of more than six years of planning within the SCC and related supporters.  The development began with an assessment and mapping of the key metrics that best tie to the SCOR Model.  The survey was created through a joint effort with SCC, APQC, and IBM as subject matter experts on supply chain and benchmarking to ensure proper alignment to SCOR, accurate metrics, thorough data analysis, and seamless final reporting to help SCC members move to Phase Three of the SCOR model.  SCC provided final approval and input on all aspects.  Prior to launch, a diverse group of SCC members participated in a beta test to ensure clarity, thoroughness, ease of use, and to give general feedback.

4. What is the SCORmark survey scope?

This survey covers the SCOR model Level One and Two metrics.  Participating in this survey implies that you have assessed the key supply chain performance priorities for your business entity. This is the process of assigning Superior, Advantage, and Parity objectives to the SCOR performance attributes. The results of the assessment are required to complete this survey. To properly benchmark, the Supply-Chain Council recommends that supply chain organizations focus on a limited list of metrics balanced over all performance attributes. The scorecard derived from this survey will calculate the gap between the business entity's current performance level and the desired performance levels.

5. What do the terms "superior," "advantage," and "parity" mean when prioritizing desired performance targets?

1. The SCOR methodology identifies a need to prioritize attributes (groups of metrics) to estimate your target level of performance given your business entity's performance goal for competitive advantage.  For each of the five supply chain attributes, you may select one as superior; two as advantage and two as parity.

    

    

   For each supply chain, only one group of metrics can be assigned an S for Superior position desired. Superior indicates the 90th percentile of companies in the database. Two metrics groups (attributes) can be assigned A for Advantage and the remaining groups get a P for Parity target. Parity indicates the 50^th percentile of performance in the SCORmark database. Advantage is the performance level halfway between Parity and Superior (i.e., 70^th percentile). 

6. What are the five supply chain attributes used in SCOR?

The attributes are used to help an organization determine which supply chain characteristics it views as most important to its competitive positioning.  The attributes are:

Supply chain reliability is an indicator of the ability to deliver on time, in full, in the right condition, in the right packaging, with the right documentation, to the right customer. Reliability is an indicator that describes the predictability of a supply-chain.

Supply chain responsivenessis an indicator of a supply chain's ability to respond to customer requests. It indicates the continuous average response time to events.

Supply chain flexibility indicates a supply chain's ability to respond to changing (market) conditions. One way to estimate flexibility metrics is to take the time it takes to execute each Level Two process in the supply chain, take the longest time for parallel processes, and sum sequential processes.

Supply chain costsindicate an organization's ability to manage the costs associated with operating the supply chain.

Supply chain asset managementindicates the effectiveness of an organization in managing assets to support demand satisfaction. This includes the management of all supply chain assets: fixed and working capital.

7. What metrics are included in SCORmark benchmarking?

The SCORmark product includes approximately 50 key high-level metrics such as total supply chain management cost, perfect order fulfillment, supply chain return on assets, as well as critical component metrics like delivery performance to customer commit date, upside sourcing flexibility, inventory days of supply, days payables outstanding, cost to return from customer.

8. How do I select the appropriate metrics to benchmark?

SCORmark benchmarking begins by first asking an organization to select the specific supply chain metrics to benchmark.  You may select as many metrics as are applicable to measure your business entity's supply chain performance based on your competitive strategy.  SCC recommends that supply chain organizations focus on a limited list of metrics balanced over all performance attributes.  Typically that list would include two to three metrics per attribute area for primary focus that result in about 15 total metrics to build a targeted road map for supply chain competitive intelligence.

9. What are the difference in Level One and Level Two metrics within the SCOR model?

Level One metrics provide a high-level look at performance for the supply chain organization in a specific attribute area, such as flexibility or responsiveness.  This metric level highlights any problem areas.  Level Two metrics provide an operational viewpoint and help an organization identify the root cause of the Level One metrics.  In essence the Level Two metrics provide the roadmap by pinpointing which metrics need improvement and thus, which processes.

10. What is an Open Standards model and why is it important for SCOR?

SCOR serves as a prime example of an open standards reference model where all organizations agree upon a common approach and model to discuss a process topic.  In a similar vein, APQC discovered the reason that processes are not adequately measured is because organizations lack a standardized "language" or set of definitions and metrics for processes.  This foundation led to an alliance with SCC to provide its members with access to one common database of metrics that are directly tied to the SCOR Model's view of how to organize and manage a supply chain for competitive success.

The value of an open standards approach lies in the ability for organizations to focus on change and results rather than structure or definitions.  This common approach enables benchmarking across industries, geographies or types of supply chains.  The end result is reduced time and expense for an organization to benchmark and greater assurance that the benchmarking data is consistent and applicable

Accuracy, Validity & Timeline

1. What do I do if I make a mistake?
    

   If you have yet to submit your benchmarking survey, you may change the answer at any time. If you find an error after submission, please contact the SCORmark Help Desk at scormark@apqc.org and they will be able to correct the error on your behalf or provide additional guidance.

    

2. How will I know that the benchmarking data is valid?
    

   APQC uses a multi-step validation process to ensure all data is accurate, reliable, and meaningful to your organization.  Based on its proven Open Standards Methodology for benchmarking, each survey submitted is processed through a rigorous validation process that includes scope reviews, response logic checkpoints, data quarantines, participant verification, and issues resolution.  In addition, all metrics are normalized and aggregated to ensure the data is accurate and relevant.

    

3. How many responses are needed for the data to be statistically valid?
    

   APQC follows a strict process to ensure that an appropriate level of data points per 20 data points per metric across industries.  APQC reports out as possible with segment views based on industry, revenue, geography, or supply chain type as data points allow.

    

4. How long will it take to receive my SCORmark report?

    

   On an ongoing basis, surveys are validated upon receipt and reports are typically delivered within 3-5 days after validation. The validation length of time varies depending on level of clarification or additional dialogue needed with a participant organization to ensure data accuracy; time ranges can be anywhere from one to two weeks.

    

Member-Benefits & Reporting

1. What type of information is included in a SCORmark benchmarking report?
    

   Each participant's SCORmark report is custom developed  based on that organization's selected metrics and prioritized supply chain attributes. The report includes two main sections: an executive scorecard to quickly spotlight any gaps in your targeted performance levels for each supply chain attribute and a detailed analysis page for each specific metric selected that features your response and the benchmarks against the performance target as well as best practice insight. Where available, the report also includes reporting cuts against specific peer groups.

    

    

   The report is designed to provide actionable data in a language and format that readily integrates in Phase Three of the SCOR model. 

1. It is usually expensive to get this kind of data. How are you offering the SCORmark service for no additional cost?
    

   As a nonprofit 501(c)3, APQC's mission is to conduct research, disseminate findings and network individuals and the organizations they represent to help improve performance and processes through benchmarking and best practices.  APQC secures similarly minded organizations to help support the development of new improvement tools.  The SCORmark portal is sponsored by IBM to allow APQC and SCC to offer the benchmarking as a member-benefit for SCC members.

    

2. If the data is a member benefit, how do I know it is of high quality?
    

   APQC's heritage as a nonprofit and benchmarking leader safeguards the quality of the SCORmark benchmarking portal and subsequent participant reports.  With a standard methodology, data validation steps, and well-established processes for benchmarking, SCORmark reports provide strategic and operational benchmarks as well as the ability to tie best practices and analysis that surround the specific metrics to aid SCC members to quickly take action from the report.

    

3. Will the SCORmark service continue to be offered as an SCC member-benefit?
    

   At this time it is SCC's intent to offer SCORmark benchmarking as an ongoing member-benefit.  The use of the portal and adoption rate among members will be reviewed annually to ensure that SCC members find value in the portal.

    

4. What is the demographic and geographic scope for benchmarking?
    

   SCORmark benchmarking is open to all SCC members internationally.  Using the open standard model, the metrics are defined in such a way that they will be applicable internationally.  In addition, the portal features currency conversion during calculation to allow benchmarking against a global data set as well as a normalization process to account for differences in scale or revenue.

    

5. What types of data cuts are available to better tailor the report?
    

   Specific segment reports are available when the metric selected includes enough data to generate reliable calculations and to maintain participant confidentiality.  Examples of report types include cuts by industry, supply chain type, revenue and geography.   Participants may also request custom peer groups on a per request basis for an additional fee.  Organizations may use SCORmark for internal benchmarking; APQC will provide the main contact with a consolidated benchmark report of all entities participating in the organization at no additional fee.

    

6. How often may I refresh my data?
    

   SCC and APQC recommend conducting SCORmark benchmarking annually to assess improvements from the prior year and to aid in budgeting and planning for the upcoming year. During a current year's report, most organizations use internal reporting to track progress monthly. 

BUSINESS RELATIONSHIPS & CONFIDENTIALITY

1. What level of input did SCC have in the process and what metrics are included? 
    

   SCC leadership and members of its Technical Development Steering Committee provided the guiding direction for the SCORmark benchmarking portal and to ensure alignment with SCOR.  SCC is the final decision maker about which metrics to gather data for and how the calculations are made.

    

2. Will there be a general annual benchmarking report provided to SCC members?
    

   In the future, SCC intends to release a general annual report as a member-benefit.  No timeline has been determined as of this time.

    

3. Will SCC have control over the data and if it is shared with outside parties?
    

   As an exclusive benefit for SCC member organizations, SCC will control the release of content that is specific to the SCOR Model.  SCC will receive access to the blinded, normalized, aggregate data to aid in future research for the benefit of SCC members or to generate a general benchmarking report for SCC members.  All content used in any context will be blinded, normalized and aggregated prior to release, except with regard to that specific organization's individual benchmarking report.

    

4. How do academics, consultants, and analysts use the data?
    

   The SCORmark service provides organization specific, custom benchmarking tied to SCOR Phase 2: Analyze.  Consultants may elect to aid clients in completing benchmarking on their behalf as part of an engagement. If so, the organization must allow the consultant access to the final report.  APQC will only provide reporting directly to the participating organization based on its strict confidentiality policies.

5. Who is APQC?

An internationally recognized resource for process and performance improvement, APQC works with its member organizations to identify best practices; discover effective methods of improvement; broadly disseminate findings; and connect individuals with one another and the knowledge, training, and tools they need to succeed.A leader in benchmarking for 30 years, APQC has developed renowned industry standards to promote open, consistent, ethical and safe benchmarking including the Benchmarking Code of Conduct, Process Classification Framework, and Open Standards Research.

Founded in 1977, APQC is a member-based nonprofit serving more than 500 organizations around the world in all sectors of business, education, and government. To learn more, please visit www.apqc.org.

6. How is SCORmark benchmarking different from the other supply chain research that APQC conducts?

APQC's supply chain research is based on APQC's Process Classification Framework (PCF) which provides a traditional view of the supply chain organization.  As a result APQC offers benchmarking via the Open Standards Research to any organization seeking to improve its supply chain performance.  The SCORmark benchmarking was specifically built to integrate into the SCOR Model framework so that all metrics gathered may be applied directly and seamlessly in a SCOR implementation.

7. What is APQC's role in this relationship?

APQC serves as a confidential, third-party repository for SCC members and will collect, validate, and report benchmarking data, as well as provide analysis, insight, and best practices information. APQC aided in the development of the survey instrument, testing and online database creation.

8. Will APQC keep my data confidential?

As a nonprofit research organization, APQC is the sole custodian of the database, guaranteeing objectivity and confidentiality.  As the authors of the widely adopted Benchmarking Code of Conduct, APQC subscribes to the highest levels of confidentiality. All information provided remains confidential, and names of participating organizations are not associated with specific data.

9. What is IBM's role in this relationship?

IBM sponsored the portal creation and contributed supply chain expertise to the development of the SCORmark service as part of its ongoing support for open standards performance metrics. As one of the founders with APQC of the Open Standards Benchmarking Collaborative, IBM is supportive of efforts to create common, open frameworks to discuss business processes.

10. How does IBM view the idea of "open standards"?

Open standards provide value for both business and technology. In a globally integrated world, companies have to do more than simply integrate everything inside their enterprise. They have to connect their enterprise with other enterprises, other business processes, other applications, and billions of computing devices. Open standards allow all technologies to connect and integrate, and allow IT to become more modular. Similarly, business leaders need to be able to measure progress in the same way so that they have a common view of process definitions to benchmark performance. Open standards benchmarking helps businesses make better-informed investment decisions, with broader, deeper and higher quality data for comparison.

11. Who will see my submission and benchmarking data?

Only APQC sees an individual survey submission for the purpose of data validation and reporting.  SCORmark participants, SCC and IBM have access through their participation to blinded, normalized benchmark data.