Raising the Bar on Content Quality at Amazon Web Services

When it comes to creating and managing content, sometimes the sheer volume of a knowledge base can present challenges for organizations looking to keep their content fresh and findable.

Amazon Web Services (AWS), a subsidiary of Amazon.com, and since 2006 it is the world’s most comprehensive and broadly adopted cloud.

AWS has a history of excellence in content and knowledge management, driven by the AWS Prescriptive Guidance (APG) team. This group of dedicated professionals earned APQC’s 2025 Excellence in Knowledge Management designation. 

The APG team sat down with Lynda Braksiek (KM Principal Research Lead, APQC) to describe how they instituted the “Bar Raiser” quality initiative to improve the content in their internal and customer-facing knowledge repositories. Participants included:

• Mona Arora, APG Sr. Technical Program Manager

• Ellen Crowley, APG Sr. Enablement Program Manager

• Sasha Friedman, APG, Principal Product Manager

• Steven Guggenheimer, AWS SME, APG Tech Lead—Bar Raiser Program

• Shashank Agarwal, APG Sr. Technical Program Manager—Bar Raiser Program 

• Handan Selamoglu, APG Documentation Manager

APG and the AWS Community 

The APG team was launched in 2019 to serve the knowledge management needs of AWS and has evolved steadily to become a driver for quality initiatives for AWS community content.
 
The APG journey started by collaborating on a knowledge base where AWS employees could share lessons learned and new ideas for how to solve customer challenges during cloud migration projects. Today, the program provides a suite of products that facilitate the discovery and production of best practices to accelerate AWS Cloud adoption among consultants, AWS Partners, and the public. 

The content/knowledge management system includes three components: 

• APG website (APGE): A publicly available resource to help customers adopt the AWS Cloud.

• APG Library (APGL): A custom publishing platform and knowledge base for AWS employees and AWS Partners to author, publish, and consume APG content.

• APG dashboard: Self-service reporting and tracking on APGE and APGL content performance (page views, likes, reuse with other customers) metrics.
   

APG provides authors with three content templates that map to three unique user personas:

• Strategies to provide business perspectives, methodologies, and frameworks for executives

• Guides that focus on best practices and tools for architects and technical leads

• Patterns that include both content and code to help builders implement common migration, optimization, and modernization scenarios

Raising the Bar 

As the program scaled over the next few years, APG realized they needed to focus not only on content quantity but also level up the quality requirements: In early 2022, technical writer evaluations of published content showed it was not always of high quality, with inaccurate or incomplete information. User perception in surveys also showed diminishing trust in the content.

It became clear that authors were focused on quantity over quality, driven by an incentives program focused on meeting content goals. The peer review process was ineffective, and the burden of reviewing the technical accuracy of content was falling on APG technical writers rather than SMEs with technical knowledge.

To improve content accuracy and quality, APG developed the Bar Raiser program. To kick off the initiative, the team recruited independent SMEs to review database-related content that was published on the platform. The results: 85% of database content stored in APGL Library needed significant updates. The experiment revealed that most authors didn’t keep their content up to date after publication and confirmed user feedback regarding outdated content.

This initial assessment showed that AWS lacked a standardized quality measurement framework to set quality baselines and to evaluate and scale content quality across the platform. The APG team set to work implementing a quality assurance program that would solve these problems. Strategies included:

• Recruited SMEs from across AWS to build the APG Bar Raiser program as an independent content review mechanism.

• Created a quality framework with scoring rubrics, a technical quality questionnaire, and reported scores.

• Updated author and reviewer checklists to align with the quality framework. 

• Built data and governance dashboards to track and report program insights. These dashboards enabled bar raisers to self-serve and allowed them to communicate their contributions to leadership for recognition and career development.

There are currently 150 volunteer bar raisers across fourteen technical domains. Bar raisers review all content that will be made available to internal users and the public. For public content, APG technical writers provide an additional review of AWS stylistic conventions, readability, flow, and consistency. They also provide feedback to bar raisers if they encounter any issues or have questions about the content that has been through bar raiser review. 

User sentiment and technical writer evaluations have been trending upwards and validate the effectiveness of the bar raiser program.

Creating Content with Quality Standards

To incorporate the new review process and framework, the APG team designed a publishing lifecycle to build in quality assurance from beginning to end of the authoring process (Figure 1).

Enlarge Photo

The lifecycle is built on the belief that great content can come from anywhere, so APG enables builders and consultants―the people on the ground who work with customers―to create content. The lifecycle builds in checks and balances to vet ideas and to ensure that content is updated to reflect changes to cloud services in a timely manner. The team balances ease of content creation with governance and iteration to ensure that content is treated as a living thing that’s nurtured, grown, and eventually retired. 

The APG publishing lifecycle comprises five core activities: Ideate, create, incubate, share, and refresh.

Ideate: The author identifies and proposes an idea for a topic.

Bar raisers and publishers review the content proposal before authors spend time creating the content. This is the most important stage of authoring APG content. Authors assess the value of a topic to its intended audience before beginning to write. The APG team recommends following this four-step process: 

1. Identify a topic. Pull from your own experiences. What information do your customers and peers need related to their cloud adoption efforts? Are you seeing any trends? 

2. Validate the topic. The best topics fill an unmet need for the audience.

   • Confirm the topic is unique – Confirm that your topic fills a gap and doesn’t duplicate existing AWS content. 

   • Conduct research – Confirm that all readers need information on this topic, not just your immediate customers and peers. See how publicly available information such as analyst reports and press releases show increasing adoption of the technology trend for content targeted for an external audience. 

   • Uncover trends – In the APG Dashboard, search by technology domain, date range, and other relevant criteria to surface insights about your idea. 
      

3. Choose your audience. Are you writing for internal users (AWS employees), AWS Partners, or AWS customers (public audience)? 

4. Select your content type. Determine whether your content should be a pattern, guide, or strategy. 

Create: The author creates content for review and receives feedback.

After an author or co-authors create a piece, they submit it for feedback from peer reviewers, bar raisers, and/or publishers. APG provides templates, quality checklists for peer- and self-review, and examples of successful content to help authors create quality content. 

To move on to the next stage of the lifecycle, ideas must pass through two initial tiers of QA:

1. Does it meet minimum standards? This is validated through content templates and security reviews.

2. Does it raise the bar on covering the topic? 

Incubate: The content is made available to AWS employees.

This is the “content incubation” stage of the publishing workflow, where internal users have the opportunity to review the information, reuse with other customers, and leave feedback on the article itself. Authors revise and refine their content based on this feedback. 
APG promotes the democratization of data by sharing all customer demand signals, such as how much content is created in each domain, how many page views they receive, and how content is reused in customer engagements. Up- and down-voting allows audiences to voice support for the content they want to consume. If intended for a public audience, this provides an opportunity to collect user feedback that APG considers when selecting content to be published to the external website.

Share: The content is shared with AWS Partners and/or public audience.

On the APG platform, the consultants, solutions architects, and other AWS employees who author content are subject matter experts, not professional writers. For this reason, APG has technical writers who uphold professional technical writing standards. For public content, APG requires that their technical writers review each publication for adherence to company standards and readability requirements, including logically structured and clearly written pieces that are:

• Compliant with AWS branding and legal guidelines.

• Compliant with AWS stylistic standards for voice and tone, e.g., simple language, active voice, direct and friendly tone.

• Accessible and easy for ESL customers to read and translators to translate; e.g., each illustration is described in the text and includes alternative text for accessibility.
   

Technical writers also flag missing information and inconsistencies in the technical content.

Because AWS publication guidelines are updated frequently to align with new technologies (for example, AI consumption), APG technical writers collaborate with other writers and editors across AWS to follow quality measures and consider the latest patterns and practices established for technical content.

After the technical writer review, content is finalized and published to the APG Library for AWS Partners to review and to the external APG website for public consumption.

Refresh: 

Evaluating and updating content is a continuous process in order to ensure relevance and accuracy. To make sure content remains up to date, the APG workflow includes:

• The bounty review process, which sets up a content freshness review schedule for authors every six or twelve months, depending on content type.

• A content feedback mechanism to signal when to unpublish low-performing content.

• Bar raisers that are actively involved in reviewing previously published content. They prioritize these reviews based on visibility (public versus internal) and age of content (reviewing oldest first).

• Lead bar raisers from each domain to manage reviewers. This collaboration helps motivate the review team and ensures that the bar remains high.
   

The content publication lifecycle balances the needs of different APG user groups—the authors, peer reviewers, technical writers, and bar raisers who work on the content, as well as the end users who consume the content. To address the needs of so many different user groups, APG relies on its core mission: Accelerating cloud adoption as a cornerstone of decision making. 

How Did They Do It? 

After explaining the nuts and bolts of the Bar Raiser program and content lifecycle, the APG panel answered questions from APQC to provide more detail on how their quality initiatives were developed and implemented, the challenges they faced, and recommendations for other content and knowledge managers.

What inputs and resources were needed to formulate and implement your solution? 

We developed features to improve the authoring, bar raising, and technical writing workflows (for example, by providing support, samples, checklists, and code reviews, and helping find peer reviewers). We also used customer surveys and feedback mechanisms as inputs. Bar raisers bring value to the program by reviewing content and suggesting improvements to taxonomies and publication workflows.

For those of us in bar raiser roles, reviewing content has helped make us better writers and consumers of content. We can produce more effective thought leadership because we get to have conversations with other experts through the review process. 

What challenges did the team encounter along the way and how were you able to overcome them?

The AWS service teams that build the products and the consultants and solution architects who engage with customers often have differing views of best practices. Instead of letting these disagreements paralyze our authors for fear of generating controversy, we created a system that allowed different ideas to float up to the top and where iteration was inherent. 

We have had to balance this with a governance process and guardrails to ensure the accuracy and quality of the content we publish. This started with checklists and peer reviews, but over time we realized that these measures were too easy to circumvent, so we built the bar raiser program. 

Other challenges include the unavailability of peer reviewers, lack of response and follow-up from authors and peer reviewers, as well as changing organizational processes and requirements that impact the program. Content is orphaned when authors and peer reviewers leave AWS. This means that we have to find new authors before the loss of ownership impacts content quality.

We have a small team of technical writers and a large backlog of content that authors would like to publish on the external website. Selecting the right content from this backlog and responding to constant author requests became challenging, so we developed selection criteria. We now ask bar raisers to flag content they deem to be high priority. We also rely on the number of customer engagements and internal AWS user feedback such as “likes” and tickets to make sure we’re selecting the right content for a public audience.

How did you measure and reward success?

We made an investment early on in our program to democratize our data, that is, to freely publish it for all users to see. This meant that we had to be honest with ourselves and our customers about the value of the platform and the impact of our initiatives and experiments, whether they were successful or not. 

We primarily use page views, unique users, and reuse in customer engagements as core metrics. Although these are imperfect measures (does a single page view from a single user have value?), they nonetheless indicate whether there is ongoing demand for this content. 

We also track measures such as customer feedback tickets, time spent creating content, page load time, thumbs-up/down votes, and similar data. We’ve found this very helpful for our feature roadmap. 

For example, at one point we saw a continuous stream of support tickets from authors who had trouble formatting their content. This led us to examine our text editor to see if we could fix issues. We ended up replacing it with a different text editor, and support tickets for formatting dropped to zero. 

We also measure customer satisfaction through user surveys: One year after launch, the majority of respondents noticed improved content quality in bar-raised domains. Additionally, bar raiser-reviewed content achieved consistently higher scores during independent technical writer evaluations compared with other content. This further demonstrates the program's effectiveness in raising quality standards.

To reward participation, we give substantial monetary and non-monetary awards, including quarterly Amazon gift cards for bar raisers. We have also built communities of practice where participants can earn points and get invited to summits. 

To recognize high-level contributors, we call out people who are doing great work on content in our monthly APG newsletter. We’re currently working on a spotlight series to showcase people behind the content to contribute to the library, where they can be showcased as thought leaders and experts. 

Key Takeaways

The APG panel shared the following advice for other knowledge management leaders looking to update their content platforms and raise the bar on quality:

• When you build a platform where you have to balance the needs of many user groups against one another, consider the specific pain points of each group, their severity, and how many people are in that group. 
  
  For example, APG serves multiple personas: In descending order of population, these are content consumers, content authors, peer reviewers, bar raisers, technical writers, and administrators. We wouldn’t necessarily prioritize a request for a feature that streamlines an administrator workflow, such as routine unpublishing of content, if a workaround already exists. Conversely, if there is a problem that affects content consumers who are trying to view content, fixing that issue is critical.
   

• In today’s AI-centric world, the need for correct, governed truth is even more important than before. Set up your platform in such a way that you can provide your content through “read” APIs, track users, and track how people are clicking through to the source content. You won’t know how your content is being used once it leaves your system, but you can determine when people come to read the source article. 

• Knowledge management can feel like an under-appreciated field at times, and its return on investment can be called into question. But providing quality content at scale requires investment and leadership support. Build metrics into your system from the beginning so you can report on who is using your platform, how often, and in what ways. Also, be sure to connect your knowledge management initiatives to quantifiable business value, such as productivity and revenue gains. If the platform has widespread adoption and provides the medium through which people get their jobs done, you can validate its value.

To Learn More

To read more about best practices in content management, see the Managing Content So It's Fresh and Findable collection in the APQC Resource Library.