Introduction to Search Quality Rating

 0.0 The Search Experience

 The World Wide Web is a vast collection of online information and content.  Internet search engines provide a powerful  way to explore this online universe.  There are many ways people search: people may type words into a search box in a  browser, speak to a mobile phone or assistant device, use search engine autocomplete features, etc.

 People search the Internet  for a variety of purposes, ranging from accomplishing a quick task to researching a topic in  depth.  A search may be part of a long-term project, such as a home remodel or vacation planning.  A search may be  done when someone is bored and looking for entertainment, such as a search for [funny videos].  A search may be a  single question asked during a critical moment of a person’s life, such as [what are the symptoms of a heart attack?].

 Search engines exist to help people find what they are looking for.  To do that, search engines must provide a diverse set  of helpful, high quality search results, presented in the most helpful order.

 Different types of searches need very different types of search results.  Medical search results should be high quality,  authoritative, and trustworthy.  Search results for [cute baby animal pictures] should be adorable.  Search results for a  specific website or webpage should have that desired result at the top.  Searches that have many possible meanings or  involve many perspectives need a diverse set of results that reflect the natural diversity of meanings and points of view.

 People all over the world use search engines; therefore, diversity in search results is essential to satisfy the diversity of  people who use search.

 Finally, search results should help people.  Search results should provide authoritative and trustworthy information, not  lead people astray with misleading content.  Search results should allow people to find what they’re looking for, not  surprise people with unpleasant, upsetting, offensive, or disturbing content.  Harmful, hateful, violent, or sexually explicit  search results are only appropriate if the person phrased their search in a way that makes it clear that they are looking for  this type of content,  and  there is no other reasonable interpretation of the words used in their search.

 0.1     The Purpose of Search Quality Rating

 As a Search Quality Rater, you will help evaluate search engine quality around the world.  Good search engines give  results that are helpful for people in their own specific language and locale.

 No single rating can directly impact how a particular webpage, website, or result appears in Google Search, nor can it  cause specific webpages, websites, or results to move up or down on the search results page.  Using ratings to position  results on the search results page would not be feasible, as humans could never individually rate each page on the open  web.

 Instead, ratings are used to measure how effectively search engines are working to deliver helpful content to people  around the world.  Ratings are also used to improve search engines by providing examples of helpful and unhelpful  results for different searches.

 As part of your role in the search quality rating program, it is important that you are familiar with and comfortable using a  search engine.  We encourage you to be an expert in search!  For example, experiment with using  operators  (e.g., quotes  or a hyphen) in your searches or try using  advanced search  options.

 0.2     Raters Must Represent People in their Rating Locale

 It is very important for you to represent people in the locale you evaluate. You must be very familiar with the task language  and location in order to represent the experience of people in your locale.  If you do not have the knowledge to do this,  please inform your employer/company.

 Unless your rating task indicates otherwise, your ratings should be based on the instructions and examples given in these  guidelines.  Ratings should not be based on your personal opinions, preferences, religious beliefs, or political views.  Always use your best judgment and represent the cultural standards of your rating locale.

 0.3     Browser Requirements

 Check with your employer/company for browser requirements.  You may use helpful browser add-ons or extensions, but  please do not use add-ons or extensions that interfere with or alter the user experience of the page.

 0.4     Ad Blocking Extensions

 Do not use add-ons or extensions that block ads for Needs Met rating or Page Quality rating. These add-ons or  extensions may cause you to give incorrect ratings.  As a rater, only use an ad blocking extension or add-on if specifically  instructed to do so in the project-specific instructions.

 0.5     Internet Safety Information

 In the course of your work, you will visit many different webpages.  Some of them may harm your computer unless you are  careful.  Please do not download any executables, applications, or other potentially dangerous files, or click on any links  that you are uncomfortable with.

 It is strongly recommended that you have antivirus and antispyware protection on your computer.  This software  must be updated frequently or your computer will not be protected.  There are many free and for-purchase  antivirus and antispyware products available on the web .

 See  here  for a Wikipedia page on antivirus software and  here  for a Wikipedia page on spyware.

 We suggest that you only open files with which you are comfortable.  The file formats listed below are generally  considered safe if antivirus software is in place.

• .txt (text files)
• .ppt or .pptx (Microsoft PowerPoint)
• .doc or .docx (Microsoft Word)
• .xls or .xlsx (Microsoft Excel)
• .pdf (PDF files) (Adobe Acrobat)

 If you encounter a page with a warning message, such as “Warning-visiting this web site may harm your computer,” or if  your antivirus software warns you about a page, you should not try to visit the page to assign a rating.

 0.6     The Role of Examples in these Guidelines

 The rating concepts in these guidelines apply to all types of content.  The examples throughout these guidelines are very  important to illustrate how the concepts defined in each section should be applied in rating tasks for different types of  webpages and different types of content.

 Please view each example and keep the following in mind:

• Webpages and websites change rapidly, so we use images or “snapshots” of webpages in most of our examples.
• Because some types of content are difficult to view in “snapshot” form, a description of the webpage is provided for each example.
• The information in the examples was accurate at the time it was added, but content and websites may change over time.
• Some examples show pages on desktop and some show pages on mobile devices.

 Part 1: Page Quality Rating Guideline

 1.0     Introduction to Page Quality Rating

 A Page Quality (PQ) rating task consists of a URL and a grid to record your observations as you explore the landing page  and the website associated with the URL.  The goal of PQ rating is to evaluate how well the page achieves its purpose.

 Here’s what you’ll need to be a successful Page Quality rater:

• Your experience using the web as an ordinary user in your rating locale.
• In-depth knowledge of these guidelines.
• And most importantly—practice doing PQ rating tasks!

 2.0     Understanding Webpages and Websites

 Page Quality rating requires an in-depth understanding of websites.  We’ll start with the basics.  Along the way, we’ll share  important information about Page Quality rating, so please read through this section even if you are a website expert!  2.1     Important Definitions

 Here are some important definitions:

 A  search engine  is a tool to help people find or interact with content available on the Internet.

 In these guidelines, the word “ user ” refers to a person trying to find information or accomplish a task on the Internet.  Users are people from many different backgrounds, whose experiences and needs may differ from your own: people of all  ages, genders, races, religions, political affiliations, etc.

 A  webpage  is connected to the World Wide Web and can be viewed or “visited” using a web browser (e.g., Chrome), a  browser on your phone, or a search app.  In the 1990s, webpage content was mostly text and links.  Today, webpage  content includes many forms of media (such as images, videos, etc.) and functionality (such as online shopping features,  email, calculator functionality, online games, etc.).

 A  URL  is a string of letters, numbers, and punctuation that your web browser uses to “find” and display a webpage.  Page  Quality rating doesn’t require you to have in-depth understanding of the structure of URLs, i.e., you don’t need to know the  difference between host, domain, etc.  But if you are interested, see  here  to read more.

 A  website  or  site  is a group of World Wide Web pages usually containing hyperlinks to each other and made available  online by an individual, company, educational institution, government, or organization.  Popular websites include  Facebook, Wikipedia, Yahoo, YouTube, etc.

 Note : In these guidelines, we will use the word “website” to refer to a collection of pages owned and controlled by  a single entity (individual, business, etc.).  But we will also use “website” to refer to major “independent” sections  (or hosts) of some websites that were created to achieve separate purposes.   For example, the Yahoo website is  organized into different sections (or hosts), such as Yahoo Finance ( finance.yahoo.com ), Yahoo Mail

 ( mail.yahoo.com ), Yahoo Sports ( sports.yahoo.com ), etc.  Each of these has its own purpose.  It’s OK to refer to  each of these sections as a website; for example, the Yahoo Finance website and the Yahoo Sports website.  You  may also refer to pages on Yahoo Finance or Yahoo Sports as belonging to the Yahoo website.

 A  homepage  of a website is the main page of the site.  It is usually the first page that users see when the site loads.  For  example,  http://www.apple.com  is the homepage of the Apple site,  http://www.yahoo.com  is the homepage of the Yahoo  company site, and  http://finance.yahoo.com  is the homepage of Yahoo Finance.  You can usually find the homepage of a  website by clicking on a “home” link or logo link on subpages of a website.

 A  website owner  is the person, company, or organization who is responsible for a website.

 A  content creator  is the individual(s) or entity (business, organization, etc.) who created the content on a webpage.

 Important :  You must be very comfortable exploring websites, both by clicking links and modifying URLs in the address  bar of your web browser.  Become a website detective and explorer!

 2.2       Understanding the Purpose of a Webpage

 The purpose of a page is the reason or reasons why the page was created.  Every page on the Internet is created for a  purpose, or for multiple purposes.  Most pages are created to be helpful for people, thus having a beneficial purpose.  Some pages are created merely to make money, with little or no effort to help people.  Some pages are even created to  harm users.  The first step in understanding a page is figuring out its purpose.

 Why is it important to determine the purpose of the page for PQ rating?

• The goal of PQ rating is to determine how well a page achieves its purpose. In order to assign a rating, you must  understand the purpose of the page and sometimes the website.
• By understanding the purpose of the page, you’ll better understand what criteria are important to consider when evaluating that particular page.
• Websites and pages should be created to help people. If that is not the case, a rating of  Lowest  may be    More on this later.

 As long as the page is created to help people, we will not consider any particular page purpose or type to be higher quality  than another.  For example, encyclopedia pages are not necessarily higher quality than humor pages.

 Important : There are highest quality and lowest quality webpages of all different types and purposes: shopping pages,  news pages, forum pages, video pages, pages with error messages, PDFs, images, gossip pages, humor pages,  homepages, and all other types of pages.  The type of page does not determine the PQ rating—you have to understand  the purpose of the page to determine the rating.

 Common helpful or beneficial page purposes include (but are not limited to):

• To share information about a topic.
• To share a personal experience, perspective, or feelings on a topic.
• To share pictures, videos, or other forms of media.
• To demonstrate a personal talent or skill.
• To express an opinion or point of view.
• To entertain.
• To offer products or services.
• To allow users to post questions for other users to answer.       To allow users to share files or to download software.

 Here are a few examples where it is easy to understand the purpose of the page:

 Here is an example (OmNomNomNom Page)  of a helpful page where the purpose of the page is not as obvious.  At first  glance, this page may seem pointless or strange.  However, it is a page from a humorous site that encourages users to  post photos with mouths drawn on them.  The purpose of the page is humor or artistic expression.  This page has a  helpful or beneficial purpose.  Even though the  About  page on this website is not very helpful, the website explains itself  on its  FAQ  page.

 2.3      Your Money or Your Life (YMYL) Topics

 Pages on the World Wide Web are about a vast variety of topics.  Some topics have a high risk of harm because content  about these topics could significantly impact the health, financial stability, or safety of people, or the welfare or well-being  of society.  We call these topics “Your Money or Your Life” or YMYL.

 YMYL topics may significantly impact or harm one or more of the following:  ●         the person who is directly viewing or using the content

• other people who are affected by the person who viewed the content
• groups of people or society affected by the actions of people who viewed the content

 YMYL topics can directly and significantly impact people’s health, financial stability or safety, or the welfare or well-being  of society, because of the following reasons:

• The topic itself is harmful or dangerous . For example, there is clear and present harm directly associated with  topics related to self-harm, criminal acts, or violent extremism.
• The topic could cause harm if the content is not accurate and trustworthy . For example, mild inaccuracies  or content from less reliable sources could significantly impact someone’s health, financial stability, or safety, or  impact society, for topics like: symptoms of a heart attack, how to invest money, what to do if there is an  earthquake, who can vote, or needed qualifications for obtaining a driver’s license.

 To determine whether a topic is YMYL, assess the following types of harm that might occur:

• YMYL Health or Safety : Topics that could harm mental, physical, and emotional health, or any form of safety such as physical safety or safety online.
• YMYL Financial Security : Topics that could damage a person’s ability to support themselves and their families.
• YMYL Society : Topics that could negatively impact groups of people, issues of public interest, trust in public institutions, etc.
• YMYL Othe r: Topics that could hurt people or negatively impact welfare or well-being of society.

 It’s possible to imagine a hypothetical harmful page for any non-harmful topic, such as the science behind rainbows or  shopping for pencils: for either of these topics, someone could build a page that has a malicious computer virus download.  However, for a specific topic to be YMYL, the topic itself must potentially impact people’s health, financial stability, or  safety, or the welfare or well-being of society.

 Many or most topics are not YMYL and do not require a high level of accuracy or trust  to prevent harm .  Because YMYL  assessment is a spectrum, it may be helpful to think of topics as clear YMYL ,  definitely not YMYL or something in  between.  Pages on clear YMYL topics require the most scrutiny for Page Quality rating.

 If you are having trouble deciding whether a topic is YMYL, consider the following questions:

1. Would a careful person seek out experts or highly trusted sources to prevent harm ?  Could even minor  inaccuracies cause harm?  If yes, then the topic is likely YMYL.
2. Is the specific topic one that most people would be content with only casually consulting their friends about? If  yes, the topic is likely not YMYL.

 Important : For pages about clear YMYL topics, we have very high Page Quality rating standards because low quality  pages on such topics could potentially negatively impact a person’s health, financial stability, or safety, or the welfare or  well-being of society.

 2.4       Understanding Webpage Content

 All of the content on a webpage can be classified as one of the following: Main Content (MC), Supplementary Content  (SC), or Advertisements/Monetization (Ads).  In order to understand the purpose of a webpage and do PQ rating, you will  need to be able to distinguish among these different parts of the page.

 Webpage design can be complicated, so make sure to click around and explore the page.  See what kind of content is  behind the tabs and test out the interactive page features.  Content behind the tabs may be considered part of the MC,  SC, or Ads, depending on what the content is.

 2.4.1 Identifying the Main Content (MC)

 Main Content is any part of the page that directly helps the page achieve its purpose.  MC can be text, images, videos,  page features (e.g., calculators, games), and it can be content created by website users, such as videos, reviews, articles,  comments posted by users, etc.  Tabs on some pages lead to even more information (e.g., customer reviews) and can  sometimes be considered part of the MC.

 The MC also includes the title at the top of the page ( example ).  Descriptive MC titles allow users to make informed  decisions about what pages to visit.  Helpful titles summarize the MC on the page.

 2.4.2 Identifying the Supplementary Content (SC)

 Supplementary Content contributes to a good user experience on the page, but does not directly help the page achieve its  purpose.  SC is an important part of the user experience.  One common type of SC is navigation links that allow users to  visit other parts of the website.  In some cases, content behind tabs may be considered part of the SC.

 Sometimes the easiest way to identify SC is to look for the parts of the page that  are not  MC or Ads.

 2.4.3 Identifying Advertisements/Monetization (Ads)

 Ads may contribute to a good user experience.  Advertisements/Monetization (Ads) is content and/or links that are  displayed for the purpose of monetizing (making money from) the page.  The presence or absence of Ads is not by itself a  reason for a  High  or  Low  quality rating.  Without advertising and monetization, some webpages could not exist because it  costs money to maintain a website and create high quality content.

 There are many different ways to monetize a webpage, including advertisements and affiliate programs.  See  here  for  more information on website monetization.  Note that monetization on mobile pages may be more subtle than  monetization on desktop pages.

 The most common type of monetization is advertisements.  Ads may be labeled as “ads,” “sponsored links,” “sponsored  listings,” “sponsored results,” etc.  Ads may change when you reload the page, and different users may see different Ads  on the same page.

 Website owners can choose to display Ads on their page (such as by joining an  advertising network ), but they may not  always directly control the content of the Ads.  However, we will consider a website responsible for the overall quality of  the Ads displayed.

 Important:  For the purpose of these guidelines, we will consider monetized links of any type to be “Ads.”  See  here  for  different types of website monetization.

 2.4.4 Summary of the Parts of the Page

 Let’s put it all together.

• Main Content (MC) is any part of the page that directly helps the page achieve its purpose.  MC is (or should be!) the  reason the page exists.  The quality of the MC plays a very large role in the Page Quality rating of a webpage.
• Supplementary Content (SC) is also important.  SC can help a page better achieve its purpose or it can detract from  the overall experience.
• Many pages have advertisements/monetization (Ads) .  Without advertising and monetization, some webpages  could not exist because it costs money to maintain a website and create high quality content.  The presence or  absence of Ads is not by itself a reason for a  High  or  Low  quality rating.

 On some pages, user reviews and comments may be considered MC, and on other pages they may be considered SC.  Use your best judgment and think about the purpose of the page.

 Do not worry too much about identifying every little part of the page.  Think about which parts of the page are the MC.  Next, look for the Ads.  Anything left over can be considered SC.

 2.5       Understanding the Website

 Pages often make more sense when viewed as part of a website.  Some of the criteria in Page Quality rating are based  on understanding the website that the page belongs to.

 To understand a website, start by finding out who is responsible for the website and who created the content on the page  (more information provided in  Section 2.5.2 ).  Then, look for information about the website and/or content creators on the  website itself.  Websites and content creators are usually very eager to tell you all about themselves!

 You must also look for reputation information about the website and/or content creators.  What do outside, independent  sources say about them?  When there is disagreement between what the website or content creators say about  themselves and what reputable independent sources say, trust the independent sources.

 2.5.1 Finding the Homepage

 The homepage of a website usually contains or has links to important information about the website.  Website owners  usually make it easy to get to the homepage from any page on the site.

 Here’s how to find the homepage of a website:

1. Examine the landing page of the URL in your PQ rating task.
2. Find and click on the link labeled with the name or logo of the website (occasionally labeled as “home” or “main”), which usually appears at the top of the page.

 Sometimes, you may be given a webpage or website that appears to have no navigation links, no homepage link, and no  logo or other means to find the homepage.  Even some  High  or  Highest  quality pages lack a way to navigate to the  homepage.  If you can’t find a link to the homepage, modify the URL by removing everything to the right of “.com,” “.org,”  “.net,” “.fr,” etc. and refresh the page.

 Occasionally, your rating task will include a URL for which there are two or more justifiable “homepage” candidates.  For  example, you may not be sure whether the homepage of the URL  https://finance.yahoo.com/calendar  is  http://finance.yahoo.com  or  http://www.yahoo.com .

 Important : When you have more than one homepage “candidate,” please use whichever one offers the most information  about the specific webpage in the rating task.  Use your judgment.   The goal is to understand the webpage and the  website(s) it is associated with, not find the one unique, correct homepage.

 In the following examples, we have included the URL of the page to be evaluated in the rating task, as well as the URL of  its associated homepage.  We have also included an image that shows where to click on the landing page to navigate to  the homepage.  In the image, you will see a red box around the link or logo you would click to navigate to the homepage.

 2.5.2 Finding Who is Responsible for the Website and Who Created the Content on the Page

 Every page belongs to a website, and it should be clear:

• Who (what individuals, company, business, organization, government agency, etc.) is responsible for the website.
• Who (what individuals, company, business, organization, government agency, etc.) created the content on the page you are evaluating.  Note that for pages on websites such as forums and social media platforms, people  may post content using an alias or username in order to avoid sharing personally identifiable information online.  In these cases, the alias or username is an acceptable way to identify the content creator.

 Websites are usually very clear about who is responsible for the website and who created the content on the page.  There  are many reasons for this:

• People may want to showcase their own personal talent, experiences, and perspectives.
• Artists, authors, musicians, and other original content creators may want to be known and appreciated.
• Commercial websites may have copyrighted material they want to protect.
• Businesses and organizations may want people to know who they are and what they do.     Stores want customers to feel comfortable buying their products online.

 Most websites have “contact us” or “about us” or “about” pages that provide information about who owns the site.  Many  companies have an entire website or blog devoted to who they are and what they are doing, what jobs are available, etc.  Here are some examples:

• Google Official Blog
• Marriott Blog
• Southwest Airlines Blog
• Netflix Tech Blog

 In PQ rating tasks, you will need to identify who created the MC on the page.  To help you do so, the table below  describes the primary content creators for common webpage types.  These types can overlap (e.g., a page may have  multiple articles, some written by individual authors or journalists and others written by the website owner).

 2.5.3 Finding About Us, Contact Information, and Customer Service Information

 Many websites are interested in communicating with their users.  There are many reasons that users might have for  contacting a website, from reporting problems such as broken pages, to asking for content removal.  Many websites offer  multiple ways for users to contact the website: email addresses, phone numbers, physical addresses, web contact forms,  etc.  Sometimes, this contact information is even organized by department and provides the names of individuals to  contact.

 The types and amount of contact information needed depend on the type of website.  Contact information and customer  service information are extremely important for websites that handle money, such as stores, banks, credit card  companies, etc.  Users need a way to ask questions or get help when a problem occurs.

 For shopping websites, we’ll ask you to do some special checks.  Look for contact information—including the store’s  policies on payment, exchanges, and returns.  Sometimes this information is listed under “customer service.”

 Some kinds of websites need fewer details and a smaller amount of contact information for their purpose.  For example,  humor websites may not need the level of detailed contact information we would expect from online banking websites.

 Occasionally, you may encounter a website or content creator with a legitimate reason for anonymity.  For example,  personal websites may omit personal contact information such as an individual’s home address or phone number.  Similarly, websites with user-generated content may allow the author to identify themself with an alias or username only.

 To find contact or customer service information for a website, start with the homepage.  Look for a “contact us” or  “customer service” link.  Explore the website if you cannot find a “contact us” page.  Sometimes you will find the contact  information on a “corporate site” link or even on the company’s official social media page.  Be a detective!

 Note that different locales and different social media platforms may have their own conventions and standards for what  information should be available on the website.  Please use your judgment and knowledge of your locale.

 3.0 Overall Page Quality Rating

 Now that you are an expert in understanding websites and webpages, here are the high-level steps of Page Quality rating:

1. Assess the true purpose of the page. If the website or page has a harmful purpose or is designed to deceive  people about its true purpose, it should be rated  Lowest .
2. Assess the potential of the page to cause harm as described in these guidelines. Websites or pages that are  harmful to people or society, untrustworthy, or spammy as defined in these guidelines should be rated  Lowest .    Otherwise, the PQ rating is based on how well the page achieves its purpose as described in these guidelines.

 Important : There are harmful pages that are seemingly “official”, “expert”, or “authoritative”.  Any type of website may  have pages with harmful MC, from user-uploaded videos and forum posts to harmful products sold online to pages to  pages that mimic the look of scientific papers or encyclopedia entries.  All pages should be evaluated for harm—including  pages on government websites, academic institutions, reputable online stores, charities, or other types of generally helpful  websites.

 On Page Quality rating tasks, you will use the Page Quality sliding scale (slider) to assign the overall PQ rating.  The  slider looks like this:

 You may also use the in-between  ratings of  Lowest+ ,  Low+ ,  Medium+ , and  High+ .  Please interpret the “+” as “+ ½,”  meaning that the  Lowest+  rating is halfway between  Lowest  and  Low ,  Low+  is halfway between  Low  and  Medium , etc.

 Identifying  Lowest  and  Low  quality pages can be the most difficult part of PQ rating.  The guidance in the sections on  Lowest  and  Low  is more extensive to help you rate accurately.  High  and  Highest  quality pages are typically easier to  identify, but be sure to apply the standards and examine the examples throughout these guidelines to help calibrate your  ratings.  As might be expected,  Medium  quality pages are commonly found online.

 3.1       Page Quality Rating Considerations

 The following table summarizes the things that you should consider in Page Quality rating.  Upcoming sections of these  guidelines will elaborate on these considerations and explain how they apply to pages across the PQ rating scale.

 Important : These considerations overlap.  For example, while examining the quality of the MC, you may notice factual  inaccuracies that lower your assessment of Trust.  While conducting reputation research, you may find information about  the expertise of the content creator which increases your level of Trust.  This is how PQ rating is designed to work!

 3.2       Quality of the Main Content

 The quality of the Main Content (MC) is one of the most important considerations for PQ rating.  The MC plays a major  role in determining how well a page achieves its purpose.

 The unifying theme for evaluating the quality of the MC is the extent to which the MC allows the page to achieve its  purpose and offers a satisfying user experience.  For most pages, the quality of the MC can be determined by the amount  of  effort ,  originality , and  talent or skill  that went into the creation of the content.  For informational pages and pages on  YMYL topics,  accuracy  and consistency with well established expert consensus is important.

 The purpose of the page, topic of the page, and type of website all play a role in how to evaluate the quality of the MC.  For example, consistency with well-established expert consensus is important for medical advice.  Skill is important for  how-to videos.  Talent and originality is important for artistic expression.  The amount of effort expected for a short video  shared on social media is less than for a full-length, professionally produced documentary on a streaming video website,  but both need sufficient effort to create satisfying content for their purpose.  Think about what effort, originality, talent, or  skill looks like for the type of page that you are evaluating.

 For each page you evaluate, spend a few minutes examining the MC before drawing a conclusion about it.  Read the  article, watch the video, examine the pictures, use the calculator, play the online game, etc.  Remember that MC also  includes page features and functionality, so test the page out.  For example, if the page is a product page on a store  website, put at least one product in the cart to make sure the shopping cart is functioning.  If the page is an online game,  try to play the game yourself.  Do your best to imagine that you are someone who’s very interested in the topic,  functionality, or purpose served by the page, then think about how satisfying the MC would be for that person.

 High and low quality MC comes in all formats (e.g., text, audio, video, images) and all lengths (e.g., short-form videos and  full-length professional documentaries).  High and low quality content also exists on all types of websites, from small  personal sites to large corporate sites, from forums and social media to websites that handle financial transactions.  Think  carefully about what helps the page achieve its purpose and what makes the MC satisfying for users.

 3.3       Reputation of the Website and Content Creators

 An important part of PQ rating is understanding the reputation of the website.  If the website is not the primary creator of  the MC, it’s important to research the reputation of the content creator as well.

 Reputation research should be performed according to the topic of the page.  For example, if the page contains medical  information, research the reputation of the website and content creator for providing medical information.  It’s possible for  a website to be a go-to source for one type of content (e.g., humorous videos), but an untrustworthy source for a different  type of content (e.g., financial information).

 A website’s or content creator’s reputation can also help you understand what a website or content creator is best known  for.  For example, newspapers may be known for high quality, independent investigative reporting while satire websites  may be known for their humor.  An individual journalist (content creator) may be known for the clarity of their scientific  articles while a food blogger (content creator) may be known for the deliciousness of the recipes they post online.

 Note that a company or person may create content on many different websites.  For example, a newspaper might have  their own website, upload their video content to a video sharing website, post updates on social media, and contribute  content to a TV channel.  An expert on a topic might publish research papers, have a lengthy blog, and share short  updates on social media.  In these cases, you should research the underlying company or content creator.

 Reputation research is especially important for detecting untrustworthy websites and content creators.  Content may look  great on the surface, but reputation research can expose scams, fraud, or other signs of harm. You never know what you  will find unless you look!  Therefore, reputation research is required for all PQ rating tasks.

 3.3.1 Reputation of the Website

 A website’s reputation is based on the experience of real users and the opinions of people who are experts.  Websites  may represent real companies, organizations, and other entities.  Reputation research applies to both the website and the  actual company, organization, or entity that the website is representing.

 Many websites are eager to tell users how great they are.  Your job is to independently evaluate the Page Quality of the  website, not just accept information that appears on one or two pages of the website without further verification.  Be  skeptical of claims that websites make about themselves, particularly when there is a clear conflict of interest.

 Instead, look for independent reviews, references, recommendations by experts, news articles, and other sources of  credible information about the website.   Look for information written by a person or organization, not statistics or other  machine-compiled information.  News articles, Wikipedia articles, blog posts, magazine articles, forum discussions, and  ratings from independent organizations can all be great sources of reputation information.

 For YMYL topics, the reputation of a website should be judged by what experts in the field have to say.

 Recommendations from expert sources, such as professional societies, are strong evidence of a positive reputation.

 Sources of reputation information will also vary according to the topic or type of company/organization/entity that the  website represents.  For example, you might find that a newspaper (with an associated website) has won journalistic  awards.  Prestigious awards or a history of high-quality original reporting are strong evidence of positive reputation for  news websites.

 Note that some types of information about a website is not related to its reputation.  For example, websites like  Similarweb  have information about Internet traffic to the website, but do not provide evidence of positive or negative reputation.  You  can ignore this type of information since it’s not helpful for PQ rating.

 3.3.2 Customer Reviews as Reputation Information

 Customer reviews can be helpful for assessing the reputation of a store, business, or any website that offers products or  services to users.  You may consider a large number of detailed, trustworthy, positive user reviews as evidence of positive  reputation for a store or business.

 However, you should interpret customer reviews with care, particularly if there are only a few.  Keep the following in mind:

• Be skeptical of both positive and negative reviews. Anyone can write them, including the website owner or  someone whom the store or business hires for this purpose.
• Try to find as many reviews as possible. Any store or business can get a few negative reviews—this is  completely normal and expected.  Large stores and companies receive thousands of reviews, and most receive  some negative ones.
• Read the reviews because the content of the reviews matter, not just the number or star rating. Credible,  convincing reports of fraud and financial wrongdoing is evidence of extremely negative reputation.  On the other  hand, a single encounter with a rude clerk or the delayed receipt of a single package should not be considered  negative reputation information.  Please use your judgment.

 3.3.3 How to Search for Reputation Information about a Website

 Here is how to research the reputation of the website:

1. Identify the “homepage” of the website. For example, for the IBM website, ibm.com is the homepage.  You may  need to identify the content creator, if it is different  from that of the overall website.
2. Using ibm.com as an example, try one or more of the following searches on Google:
   • [ibm -site:ibm.com] – A search for IBM that excludes pages on ibm.com.
   • [“ibm.com” -site:ibm.com] – A search for “ibm.com” that excludes pages on ibm.com.
   • [ibm reviews -site:ibm.com] – A search for reviews of IBM that excludes pages on ibm.com.
   • [“ibm.com” reviews -site:ibm.com] – A search for reviews of “ibm.com” that excludes pages on ibm.com.

 Note : When searching for reputation information, try to find sources that were not written or created by the  website or the company itself.  For example, IBM might have official social media pages that it closely maintains,  which would not be considered independent sources of reputation information about the company.  See  here  for a  Wikipedia article on identifying and using independent sources.

3. Look for articles, references, recommendations by experts, and other credible information written by people about the website.

 High quality news articles and informational articles may be good sources of reputation information.  Search for  such articles.  For example, try [ibm site:en.wikipedia.org].  News articles and informational articles can help you  learn about a company and may include information specific to reputation, such as awards and other forms of  recognition, or also controversies and issues.  Note that some informational articles include a message warning  users that there are disagreements on some of the content, or that the content may be outdated.  This may be an  indication that additional research is necessary.

4. Make sure the information you find is appropriate for judging the reputation of the website. For example,  reputation information for YMYL topics should come from sources that have expertise in the associated YMYL 

 Here are some examples of reputation information:

 3.3.4 Reputation of the Content Creators

 For individual authors and content creators, biographical information articles and online discussions can be a good source  of reputation information.  Expect to find more formal reputation information about people who create content in a  journalistic, scientific, academic, or other traditionally professional capacity, as they often need online credibility for  professional success.  Educational degrees, peer validation, expert co-authors, and citations can be evidence of positive  reputation information for professionals who publish their work.  Employment history can also support a positive reputation  for topics where training, credentials, or experience are important.

 Influencers and other individual content creators who earn income on social media platforms often have reputation  information available as well, ranging from from biographical details or news articles to less formal reputation sources  such as comments by other influencers in their fields.

 For non-professional content creators including ordinary people who post on social media or forums, you may find  informal reputation information on the page itself such as comments by other people about the creators.  For example,  you may find comments or posts from other users helpful to see what other people think about a particular content creator.

 3.3.5 What to Do When You Find No Reputation Information

 You should expect to find some reputation information for large websites and well-known content creators.  People or  businesses who create content in a professional capacity typically have some reputation information available.

 However, small websites may have little or no reputation information.  This is not indicative of high or low quality.  Many  small local businesses or community organizations have a small “web presence” and rely on word of mouth.

 Furthermore, many ordinary people participate in forum discussions or post on social media websites in a personal  capacity.  People may use an alias or username to post in order to avoid sharing personally identifiable information online.  There may be no reputation information available for some individuals.  A lack of reputation about people who post  personal content is neither a positive nor a negative sign in your assessment of the page.

 Finally, remember that there are several important considerations involved in PQ rating (refer back to  Section 3.1 ).  Reputation is important, but if reputation information is not available for a website or content creator, pay extra attention to  other PQ considerations, especially when assessing pages on YMYL topics.

 3.4      Experience, Expertise, Authoritativeness, and Trust (E-E-A-T)

 Experience, Expertise, Authoritativeness and Trust (E-E-A-T) are all important considerations in PQ rating.  The most  important member at the center of the E-E-A-T family is Trust.

 Trust : Consider the extent to which the page is accurate, honest, safe, and reliable.

 The type and amount of  Trust  needed depends on the page, for example:

• Online stores need secure online payment systems and reliable customer service.
• Product reviews should be honest and written to help others make informed purchasing decisions (rather than  solely to sell the product).
• Informational pages on clear YMYL topics must be accurate to prevent harm to people and society.
• Social media posts on non-YMYL topics may not need a high level of Trust, such as when the purpose of the  post is to entertain its audience and the content of the post does not risk causing harm.

 Experience ,  Expertise  and  Authoritativeness  are important concepts that can support your assessment of Trust:

 Experience ,  Expertise , and  Authoritativeness  may overlap for some page types and topics (for example, someone may  develop Expertise in a topic due to first-hand Experience accumulated over time), and different combinations of E-E-A  may be relevant to different topics.  You should consider the purpose, type, and topic of the page, then ask yourself what  would make the content creator a trustworthy source in that context.

 Trust  is the most important member of the E-E-A-T family because untrustworthy pages have low E-E-A-T no matter how  Experienced, Expert, or Authoritative they may seem.  For example, a financial scam is untrustworthy, even if the content  creator is a highly experienced and expert scammer who is considered the go-to on running scams!

 When it comes to Page Quality rating, your assessment of E-E-A-T should be informed by one or more of the following:

• What the website or content creators say about themselves: Look at the “About us” page on the website or  profile page of the content creator as a starting point.  Is the website or content creator a trustworthy source  based on this information?
• What others say about the website or content creators: Look for independent reviews, references, news  articles, and other sources of credible information about the website or content creators.  Is there independent,  reliable evidence that the website or content creator is experienced, has expertise, is authoritative, or is otherwise  considered trustworthy?  Is there independent, reliable evidence that the website or creator is untrustworthy?
• What is visible on the page, including the Main Content and sections such as reviews and comments: For  some types of pages, the level of experience and expertise may be clear from the MC itself.  What evidence can  you gather from examining the MC or testing the page out?  For example, you may be able to tell that someone is  an expert in hair styling by watching a video of them in action (styling someone’s hair) and reading others’  comments (commenters often highlight expertise or lack thereof).

 Important : The website or content creator may not be a trustworthy source if there is a clear conflict of interest.  For  example, product reviews by people who own the product and share their experiences can be very valuable and  trustworthy.  However, “reviews” by the product manufacturer ( “Our product is great!” ) or “reviews” from an influencer who  is paid to promote the product are not as trustworthy due to the conflict of interest.

 Finally, there are many aspects of Trust, some which are not captured by Experience, Expertise and Authoritativeness.  Please consider other aspects in your overall Trust assessment, such as customer service information for online stores or  peer-reviewed publications for academic authors.  If a page is untrustworthy for any reason ,  it has low E-E-A-T.

 3.4.1 YMYL Topics: Experience or Expertise?

 Pages on YMYL topics can be created for a wide variety of different purposes.  If the purpose of a page on a clear YMYL  topic is to give information or offer advice, a high level of expertise may be required for the page to be trustworthy.

 However, sometimes pages on YMYL topics are created to share personal experiences, often regarding difficult life  challenges.  People turn to each other in times of need to share their own experience, seek comfort or inspiration, and  learn from others.  Factual information from experts and authoritative sources may not satisfy this need.

 Pages that share first-hand life experience on clear YMYL topics may be considered to have high E-E-A-T as long as the  content is trustworthy, safe, and consistent with well-established expert consensus.   In contrast, some types of YMYL  information and advice must come from experts.

 Here are some examples:

 4.0       Lowest Quality Pages

 Lowest  quality pages are untrustworthy, deceptive, harmful to people or society, or have other highly undesirable  characteristics.  The considerations for  Lowest  quality include quality of the MC, reputation, E-E-A-T, etc., but there are  special checks you will need to complete first.

 As a reminder, here are the steps for doing PQ rating:

1. Assess the true purpose of the page. If the website or page has a harmful purpose or is designed to deceive  people about its true purpose, it should be rated  Lowest .
2. Assess the potential of the page to cause harm as described in these guidelines. Websites or pages that are  harmful to people or society, untrustworthy, or spammy as specified in these guidelines should receive the  Lowest  
3. Otherwise, the PQ rating is based on how well the page achieves its purpose using the criteria outlined in these

 Steps 1 and 2 are a screening process to quickly detect harmful content before other PQ considerations are applied.

 To identify  Lowest  quality pages, start by considering the following:

 Next, assess the page to determine if any of the criteria for  Lowest  apply:

 4.1      Types of Lowest Quality Pages

 There are many different shapes and forms that  Lowest  pages can take.  To help you keep track, the table below  organizes  Lowest  pages into three broad categories: Harmful, Untrustworthy, and Spammy.  These categories can  overlap (e.g., pages that contain Harmfully Misleading Information are also Untrustworthy).

 Important : There is lots of content that many people would find controversial, one-sided, off-putting, or distasteful, yet  does not meet the criteria for  Lowest  as described in these guidelines.  Use your judgment to follow the standards  outlined throughout Section 4.0 rather than relying on personal opinions.

 4.2       Harmful to Self or Other Individuals

 Lowest  rating for pages with content that encourages, depicts, incites, or directly causes harm to self or other

 individuals.

 Harm includes physical, mental, emotional, or financial harm to people.  Pages should be considered Harmful to Self or  Other Individuals  if they directly attempt to harm people; encourage behavior that may result in harm; depict extremely  violent or gory content without a beneficial/educational purpose; or otherwise are severely traumatic to people who view  the page.

 Pages do not have to be harmful to  all  people to be considered Harmful to Self or Other Individuals.  Different people have  different levels of vulnerability to scams, awareness of potential dangers (e.g., dangerous feats depicted in stunt videos),  and tolerance for viewing violent/disturbing content.  If there is a reasonable possibility that viewing a particular page  would cause harm to those who are most vulnerable, it should be considered harmful .

 Pages created with a beneficial purpose that report on, discuss, or inform about harmful actions or events (e.g., fictional  entertainment, reputable news, education) should typically not be considered Harmful to Self or Other Individuals.  For  example, advocacy aimed at drawing attention to harmful, real-world actions or events (such as a webpage describing a  protest against domestic violence) would not be considered Harmful to Self or Other Individuals.

 Here are some examples of content that is vs. is not considered Harmful to Self or Other Individuals:

 4.3       Harmful to Specified Groups

 Lowest  rating for pages that promote, condone, or incite hatred against a Specified Group of people.

 For the purposes of Search Quality rating, a Specified Group is a group of people that can be defined on the basis of:  ●            Age (e.g., older adults)

• Caste (e.g., Dalits)
• Disability (e.g., people who are blind)
• Ethnicity (e.g., Roma)
• Gender Identity and Expression (e.g., transgender people)
• Immigration Status (e.g., student visa holders)
• Nationality (e.g., Italians)
• Race (e.g., Asians)
• Religion (e.g., Christians)
• Sex/Gender (e.g., men)
• Sexual Orientation (e.g., lesbians)
• Veteran Status (e.g., Marines)
• Victims of a major violent event and their kin (e.g., victims of the Holocaust)
• Any other characteristic that is associated with systemic discrimination or marginalization (e.g., refugees, people experiencing homelessness)

 The tone of the harmful content must be either serious (i.e., not joking or satirical) or mean-spirited (i.e., with an intent to  demean or promote intolerance) to be considered Harmful to Specified Groups.  Comedy or artistic expression related to  a Specified Group should not be considered harmful unless it is clearly mean-spirited or has other clear harmful impact.

 Criticism of objects, philosophies, and ideas are generally not considered Harmful to Specified Groups.  For example,  negative criticism of a religious doctrine should not be considered targeted at the Specified Group that follows that  religion.  Remember that the content must promote, condone, or incite hatred of  people  to be considered Harmful to  Specified Groups.

 Educational pages (e.g., definitions, research, academic papers), news stories, or other pages that have a beneficial  purpose of informing society should not be considered Harmful to Specified Groups.  Similarly, historical  documents/videos that aim to capture the beliefs of different eras should not be considered Harmful to Specified Groups.

 Here are some examples of content that is vs. is not considered Harmful to Specified Groups:

 4.4       Harmfully Misleading Information

 Lowest  rating for pages that mislead people in ways that can cause harm to people and society.

 Misleading pages may have been produced with the intent to misinform people, or the content creator may believe that  the inaccurate information they are sharing is true.  There is an especially high standard for accuracy on clear YMYL  topics or other topics where inaccurate information can cause harm.  Be sure to research consequential facts or claims as  necessary and to the extent the task time allows.

 Pages should be considered to contain Harmfully Misleading Information when they contain  at least one of the following:

 Harmfully Misleading Information can occur from any websites or content creators – even seemingly “expert”,  “authoritative” or “official” ones.  Any type of page with Harmfully Misleading Information should be rated Lowest,  regardless of the source.

 However, note that some types of information are subjective, debatable, unverifiable, or inconsequential.  For example,  pages should not be considered to contain Harmfully Misleading Information if they  exclusively  contain:

• Non-YMYL content created with a clear entertainment purpose, containing no hard claims of factual accuracy and are not harmful to people or society.  Examples include many types of fiction, satire or parody, astrology, folklore,  myths, and urban legends.
• Reviews expressing personal preferences, opinions, or value-based judgments about a product, restaurant, book/movie/TV show, etc.
• Claims or statements that are reasonably debatable when there is not a single established correct answer or truth (e.g., discussions about the relative effectiveness of different healthcare systems)
• Insignificant errors or inaccurate information about a trivial topic (e.g., inaccuracies in the height of a celebrity)

 Pages that aim to persuade others that a certain position or perspective is correct are fairly common on the Internet.

 Pages with one-sided/opinionated/controversial/polarizing content should not be considered to contain Harmfully

 Misleading Information  unless they could create harm to individuals or Specified Groups (as described above) and contain  clearly inaccurate information, contradict well-established expert consensus, or are not grounded by reasonable  facts/evidence.

 A webpage may be considered to have Harmfully Misleading Information based on the MC or on other characteristics of  the page or information about the content creator (e.g., the title of an article is harmfully misleading, even if the article on  its own is not; a creator blatantly misrepresenting their medical credentials for a video on medical topics).

 Harmfully Misleading Information can be especially hard to identify because it may require research from outside sources.

 Reputable fact-checking websites can’t always keep up with the volume of unsubstantiated theories/claims produced by

 the Internet, and some theories may even claim that debunking information is inaccurate.  You should attempt to find  high-quality, trustworthy sources to check accuracy and seek out the consensus of experts if you are unsure.  Please  research theories and claims to the extent the task time allows.  If a theory/claim seems wildly improbable and can’t be  verified by independent trustworthy sources, you should consider it unsubstantiated.

 4.5       Untrustworthy Webpages or Websites

 The  Lowest  rating should be used for pages or websites you strongly suspect are engaging in deceptive or malicious  practices.

 Some untrustworthy pages are created to benefit the website or organization rather than helping people.  Some  untrustworthy pages may even exist to cause harm to people who engage with the page, such as scams or malicious  downloads.

 Your assessment of untrustworthiness may be based on the content of the page, information about the website,  information about the content creator, and the reputation of the website or content creator.

 Your assessment may also be based on a  lack  of critically important information.  For example, any website involving  financial transactions or sensitive information should have comprehensive information about who is responsible for the  site and a way to contact the site if something goes wrong.  If some aspect of a page or website makes you suspect  deception or maliciousness, please look for information about the site.  If you cannot find reputation information to confirm  your suspicion, carefully explore the site.

 Sometimes a single page on an unknown website in isolation may seem odd but subsequent exploration shows no  concern.  However, if you see a pattern of what appears to be deception or manipulation or become concerned about your  own safety, please use the  Lowest  rating and leave the website immediately.

 Pages with the following characteristics should be considered Untrustworthy:

• Inadequate information about the website or content creator for its purpose
• Lowest E-E-A-T or Lowest reputation
• Deceptive purpose, deceptive page design, or deceptive intent
• Deliberately obstructed or obscured MC
• Characteristics of scams, malicious downloads, or other harmful behavior
• Any webpage or website designed to manipulate people into actions that benefit the website or other organization while causing harm to self, others, or Specified Groups

 Important : Highly untrustworthy pages should be given the  Lowest  rating even if you are unable to “prove” the webpage  or site is harmful.  Because many people are unwilling to use a highly untrustworthy page, an untrustworthy page or  website fails to achieve its purpose.

 4.5.1 Inadequate Information about the Website or Content Creator for the Purpose of the Page

 For pages that require a high level of trust, information about who created the content and who is responsible for the  content is critical.

 As discussed in  Section 2.5.3 , we expect most websites to have some information about who (e.g., what individual,  company, business, foundation, etc.) is responsible for the website and who created the MC and some contact  information, unless there is a good reason for anonymity.  As you will see in the examples below, many types of Lowest  pages such as malicious downloads and scams typically have no information or fake information about who is responsible  to prevent recourse by people who are harmed.

 Any site that handles personal, private or sensitive data must provide extensive contact information.  This includes sites  that ask users to create passwords, share personal information, or conduct financial transactions.

 YMYL pages or websites that handle sensitive data with absolutely no information about the website or content creator  should be rated  Lowest .

 4.5.2 Lowest E-E-A-T and Lowest Reputation of the Website or Content Creator

 If the E-E-A-T of a page is low enough, people cannot or should not use the MC of the page.  If a page on YMYL topics is  highly inexpert, it should be considered Untrustworthy and rated  Lowest .

 Use the  Lowest  rating if the website and content creator have an extremely negative reputation, to the extent that many  people would consider the webpage or website untrustworthy.

 Here are some examples originally shown in  Section 3.3.3 .  All pages on these websites should be rated  Lowest  because  of the credible extremely negative or malicious reputation of these websites, making them untrustworthy.

 Note : These website examples may be considered Harmful in some form as well.  Sometimes reputation research  exposes information that may not meet the criteria for considering a website harmful but does cause it to be untrustworthy.

 4.5.3 Deceptive Page Purpose and Deceptive MC Design

 Pages or websites are Untrustworthy if they have a deceptive purpose.  These pages or websites superficially appear to  have one purpose, but in fact exist for a different reason.

 All pages with a deceptive purpose and/or deceptive MC should be rated  Lowest  because pages that engage in  deception are Untrustworthy.

 The following list includes some common types of deceptive purpose pages.  However, no list of deceptive page types will  be complete—deceptive websites continue to evolve as people and search engines figure out how they are being tricked.

• A webpage or website that impersonates a different site (e.g., copied logo or branding of an unaffiliated site,  URL that mimics another site’s name, etc.)
• A non-satirical social network profile made by an impersonator
• A webpage or website that looks like a news source or information page, but in fact has articles to manipulate users  in order to benefit a person, business, government, or other organization politically, monetarily,  or otherwise
• A webpage claims to offer an independent review or share other information about a product, but is in fact created to make money for the owner of the website without attempting to help users.  For example,  the MC may contain intentionally misleading or inaccurate information created with the sole purpose of getting  users to click on monetized links or buy the product
• A website claims to be the personal website of a celebrity, but the website is actually created to make money for the owner of the website without the permission of the celebrity.  For example, the page may  have false testimonials for a product and is created for the sole purpose of getting users to click on monetized  links or buy the product
• A webpage with a misleading title or a title that has nothing to do with the content on the page. People  who come to the page expecting content related to the title will feel tricked or deceived
• A webpage or website with deceptive website information. For example, the website may misrepresent who  owns the site, what the website purpose really is, how the content was created, how to contact the site, etc.

 Another form of deception is through the design of the page.  Some pages are deliberately designed to manipulate users  to take an action that will benefit the owner of the website rather than help the user.

 Here are some common types of pages with deceptive MC design:

• Pages that disguise Ads as MC . Actual MC may be minimal or created to encourage users to click on the Ads.  For example, fake search pages ( example ) that have a list of links that look like a page of search results.  If you  click on a few of the links, you will see that the page is just a collection of Ads disguised as search engine results.  A “search box” is present, but submitting a new query just gives you a different page of Ads disguised as search 
• Pages that disguise Ads as website navigation links . For example, fake directory pages ( example ) that look  like a personally curated set of helpful links, possibly with unique descriptions.  In reality, the links are Ads or links  to other similar pages on the site. Sometimes the descriptions of the links are unrelated to the page.

 Take a good look at the page and use your judgment.  If you believe the true purpose of the page is not what it appears to  be, the page should be rated  Lowest.  If you believe the page was deliberately created with the primary purpose to  manipulate users to click on Ads, monetized links, questionable download links, etc., rather than to help people, the page  should be rated  Lowest.  If the page uses deception in any form even if not described here, the page should be rated  Lowest.

 4.5.4 Deliberately Obstructed or Obscured MC

 Pages are untrustworthy if the MC is deliberately obstructed or obscured due to Ads, SC, interstitial pages, download links  or other content that is beneficial to the website owner but not necessarily the website visitor.  Attempts to manipulate or  coerce users away from the MC is evidence of untrustworthiness.

 Here are some examples of pages with deliberately obstructed or obscured MC that should be rated  Lowest :

• Ads that continue to cover the MC as the website visitor scrolls down the page. The Ads are virtually impossible  to close without clicking on the Ad
• Pop-ups that obscure the MC and cannot be closed without taking an action that benefits the website
• An interstitial page that attempts to coerce a download or click that does not benefit the website visitor
• Ads that push the MC down so far that many users would not notice the MC
• MC in white-on-white text or MC presented so that it is difficult for a human being to read

 Logins or paywalls on otherwise trustworthy sites should not be considered untrustworthy, deceptive, or coercive.  Many  high quality sites such as newspapers or magazines cannot exist without monetization through subscriptions and  paywalls.  Some types of MC should be login protected, such as pages with personal information online banking websites  or private pages on social network websites.

 MC may be inaccessible for non-deliberate or non-coercive reasons, such as content that doesn’t load or content that is  unavailable in a country or region.  These are not reasons for  Lowest .

 Use your judgment to assess whether the page is designed to manipulate or coerce users into taking actions that benefit  the website.  If you are unsure, look at other pages on the same website, conduct reputation research, etc.

 Important:  Remember that many websites need monetization to share content with users.  The presence of Ads alone is  not enough for  Lowest .

 4.5.5 Suspected Malicious Behavior

 It may be difficult to prove that a page causes harm without experiencing harm yourself.  For this reason, you may use the  Lowest  rating if you strongly suspect a page is malicious or harmful even without having proof.  Any of the following  should be considered untrustworthy:

• Pages or websites that you strongly suspect are scams (see these links for more information about Internet scams:  Avoid Scams 1 ,  Avoid Scams 2 ,  Internet Fraud ).
• Pages that ask for personal information without a legitimate reason (for example, pages that ask for name, birthdate, address, bank account, government ID number, etc.).
• Pages that appear to “phish” for passwords to Facebook, Gmail, or other popular online services. See  here  for  information about “phishing” fraud.
• Pages with links that you strongly suspect are malware downloads.

 Important : Do not continue to interact with the page if you observe these characteristics.

 If you do click on a link and encounter a malware warning from your browser or antivirus software ( example ), please do  not continue to that page.  You can assume that the page is malicious for the purpose of rating and use  Lowest .

 Note that not all warnings your browser may display are related to malware (e.g., other types of browser warnings include  certificate acceptance requests, content filtering warnings, etc.).  If you are unsure or hesitant to continue to the page for  any reason, you can release the task.

 4.6       Spammy Webpages

 Pages and websites should exist for the benefit of people who visit the website.  There are many types of webpages that  can benefit visitors, such as online shopping or banking services, videos offering entertainment, or personal pages sharing  a perspective or experience.  Some of these pages depend on advertising and monetization to maintain the website and  create high quality content while still benefiting visitors.

 However, some pages are created to benefit the website owner or other organizations and with little to no attention paid to  the experience of the people who visit.  When such pages use deceptive techniques to appear near the top of search  results, it may disincentivize the creation of high quality content by crowding out pages created with effort, originality,  talent, or skill that is helpful for visitors.

 This section describes characteristics of spam or spam-like pages that should be rated  Lowest . The  Google Search  Essentials Guideline  is a good reference for non-website owners to learn more about different spam techniques.  If you  recognize any of these spam techniques on a page, please use the  Lowest  rating.

 You do not need to develop spam recognition expertise or use special spam identification tools for PQ rating. Please  review this section for guidance on what to look for even without having such tools or expertise.

 4.6.1 Cannot Determine a Purpose, No MC, Little MC, or Lowest Quality MC

 All pages should be created with sufficient MC quality so that the page can achieve its purpose.  However, some pages  are so lacking in MC that the purpose of the page itself is unclear or you may suspect the page is not actually created for  people.

 If any of the following are true, the page should be rated  Lowest :

• Despite your best efforts, the purpose of the page cannot truly be understood because the MC is gibberish or otherwise unusable for people
• The page deliberately has no MC
• The page is deliberately created with so little MC that it fails to achieve any purpose
• The page is deliberately created with such low quality MC that it fails to achieve any purpose

 4.6.2 Hacked, Defaced, or Spammed Pages

 A hacked or  defaced website  is a site that has been modified without permission from the website owner(s).

 Websites can become hacked, defaced, or filled with a large amount of distracting and unhelpful content from bad actors.  Bad actors may add unrelated MC or SC or may modify existing MC on webpages to change the purpose of the page.  These pages should be rated  Lowest  because they fail to achieve their original purpose.  Responsible website owners  should regularly check their websites for suspicious behavior and take steps to protect users.

 A common problem on forum pages or in the comments sections of pages is “spammed” comments: posts with unrelated  comments that are not intended to help other people, but rather to advertise a product or create a link to a website.  Frequently these comments are posted by a “bot” rather than a real person.  If a specific page on a website has many  unrelated “spammed” comments, the page should be rated  Lowest .

 4.6.3 Auto-generated MC

 Creating an abundance of content with little effort or originality with no editing or manual curation is often the defining  attribute of spammy websites.  One way to do this is to use “auto-generated” content.

 It is possible to create many pages or even entire websites by designing a basic template from which hundreds or  thousands of pages are created, sometimes using content from freely available sources (such as an RSS feed or API).  These pages are created with very little effort and have little to no editing or manual curation.  There is little value when  pages repackage freely available content with little attention to no attention to the experience of people visiting the page  (see  here  for more information on automatically generated content).

 It is often easiest to recognize auto-generated content by clicking on multiple pages on a website and observing

 “templated” content.  It may be difficult to recognize auto-generated content from a single page, so be sure to explore the  website if you suspect the content is templated.

 Pages and websites made up of auto-generated content with no editing or manual curation, and no original content or  value added for users, should be rated  Lowest .

 4.6.4 Copied MC with No Added Value

 Another way to create an abundance of content with little effort or originality is to copy content from another source.  The  word “copied” refers to the practice of “scraping” content, or copying content from other non-affiliated websites without  adding any original content or value to users (see  here  for more information on copied or scraped content).

 Any type of content may be copied: text, images, video, etc.  Images may be slightly cropped or edited to avoid detection.  People may alter videos as well, or even make a video of a screen playing video!  Copied content with no value added is  Lowest  quality.

 Sometimes content is “embedded” or included on the page from another source.  There is nothing wrong with embedding  content, but the  Lowest  rating applies if there is no significant original content or value added on the page.

 Likewise, social media posts may be reposted by an unrelated account.  There is nothing wrong with reposting content,  but the  Lowest  rating applies if there is no value added in the form of additional content, context, or discussion.

 Pages created solely with copied, embedded, or reposted content with little to no editing or manual curation are lacking in  effort and originality.  As with auto-generated content, these techniques can be used to create many pages or even a  whole website.  For this reason, it is often easiest to recognize pages created solely with copied, embedded, or reposted  content by exploring many pages on the site.

 The  Lowest  rating is appropriate if all or almost all of the MC on the page (including text, images or videos) is copied,  embedded, or reposted with little or no effort, originality, talent, skill, manual curation, or added value for users.  Such  pages should be rated  Lowest , even if the page assigns credit for the content to another source.

 All of the following are considered copied content:

• Content copied, reposted, or embedded exactly from an identifiable source. Sometimes an entire page is  copied, and sometimes just parts of the page are copied.  Sometimes multiple pages are copied and then pasted  together into a single page. Content that has been copied,  reposted, or embedded exactly is usually the easiest  type of copied content to identify.
• Content that is copied, but changed slightly from the original. This type of copying makes it difficult to find  the exact matching original source.  Sometimes just a few words are changed, or whole sentences are changed,  or a “find and replace” modification is made, where one word is replaced with another throughout the text.  Images may be cropped or videos may be segmented into shorter clips.  These types of changes are deliberately  done to make it difficult to find the original source of the content.  We call this kind of content “copied with minimal  ”
• Content copied from a changing source, such as a search results page or news feed. You often won’t be  able to find an exact matching original source if it is a copy of “dynamic” content (content that changes frequently).  However, we will still consider this to be copied content.

 Important : Licensed or syndicated content should not be considered as “copied” (see  here  for more on web syndication).  Examples of syndicated content in the U.S. include news articles by AP or Reuters.

 4.6.5 How to Determine if Content is Copied

 How do you determine whether all or most of the MC is copied?  How do you identify the original source of the content?  These things can be difficult to determine, but the following steps may help.

1. Copy a sentence or phrase in the text. It may be necessary to try a few sentences or phrases from the page just  to be sure.  When deciding what sentence or phrase to copy, try to find a sentence or series of several words  without punctuation, unusual characters, or suspicious words that may have replaced the original text.
2. Search on Google by pasting the sentence or phrase (surrounded by quotation marks to search for an exact match) inside the Google search box.  You may also try without quotation marks to search for more general 

 Here are some sentences and phrases from  this page  about “The Wizard of Oz” film and how to search for  them.

3. Compare the pages you find that match the sentence or phrase. Is most of their MC the same?  If so, does one  clearly come from a highly authoritative source that is known for original content creation (newspaper, magazine,  medical foundation, etc.)?  Does one source seem to reasonably be the original?  Does one source appear to  have the earliest publication date, verified by sources like the Wayback Machine?

 Use your best judgment.  Sometimes it is clear that the content is copied from somewhere, but you cannot tell what the  original source is.  Other times the content found on the original source has changed enough that searches for sentences  or phrases may no longer match the original source.  For example, Wikipedia articles can change dramatically over time.  Text copied from old copies may not match the current content.  If you strongly suspect the page you are evaluating is not  the original source, consider it likely to be copied.

 Based on this  older version of a Wikipedia article  about “The Wizard of Oz,” which can be found using the  Internet Archive  Wayback Machine , we can confirm that the text in the table has been copied.   Please note that searching for the above  sentence and phrases will not lead you to a current Wikipedia article about “The Wizard of Oz,” which demonstrates how  confirming that the MC of a page contains copied content can sometimes be difficult.

 Important : The  Lowest  rating is appropriate if all or almost all of the MC on the page is copied with little or no effort,  manual curation, or added value for users.  Such pages should be rated  Lowest , even if the page assigns credit for the  content to another source.

 4.7       Examples of Lowest Quality Pages

5.0      Low Quality Pages

 Low  quality pages do not achieve their purpose well because they are lacking in an important dimension or have a  problematic aspect.  Low  quality pages do not meet the standards for  Lowest  but may have similar though milder  undesirable characteristics.

 To identify  Low  quality pages, start by considering the following:

 Next, assess the page to determine if any of the criteria for  Low  apply:

 Important : Low quality pages can occur on any type of website, including academic websites, nonprofit websites,  government websites, or any other generally helpful type of website.  Low quality pages may be about any topic.  Pages  on YMYL topics need more scrutiny for signs of Low quality related to the MC and website/content creator because the  page could impact a person’s health, financial stability, or safety, or the welfare or well-being of society.  However, for any  type of page, a single Low quality attribute is enough to use the  Low  quality rating.

 5.1       Lacking E-E-A-T

 Low  quality pages often lack an appropriate level of E-E-A-T for the topic or purpose of the page.  Here are some  examples:

• The content creator lacks adequate experience, e.g. a restaurant review written by someone who has never eaten at the restaurant
• The content creator lacks adequate expertise, e.g. an article about how to skydive written by someone with no expertise in the subject
• The website or content creator is not an authoritative or trustworthy source for the topic of the page, e.g. tax form downloads provided on a cooking website.
• The page or website is not trustworthy for its purpose, e.g. a shopping page with minimal customer service information

 Important:  The  Low  rating should be used if the page lacks appropriate E-E-A-T for its purpose.  No other considerations  such as positive reputation or the type of website can overcome a lack of E-E-A-T for the topic or purpose of the page.

 5.2       Low Quality Main Content

 MC is  Low  quality if it is created without adequate effort, originality, talent, or skill necessary to achieve the purpose of the  page in a satisfying way.  Mild inaccuracies on informational pages are evidence of  Low  quality.  However, if the page has  harmful inaccuracies or so many inaccuracies that it feels untrustworthy, the  Lowest  rating is justified.

 There are many possible ways for MC to be  Low  quality, and  Low  effort  is important to recognize in all forms.

 While you may not have experience creating MC on webpages, think about your experience with something similar:  writing reports for school.  Here are some common shortcuts that students often take to save effort when writing a report:

• Producing content quickly with little attention to important aspects such as clarity or organization.
• Saving time by summarizing or paraphrasing from other readily available sources.
• Filling a report with large pictures or other distracting content rather than clear, helpful information on the topic.
• Using images from other sources (magazines, stock image websites, etc.) to avoid having to create pictures or diagrams to support the material in the report.
• Using commonly known facts that require little research, for example: “Argentina is a country. People live in Argentina has borders.”
• Using “filler” content, for example: “Pandas eat bamboo. Pandas eat a lot of bamboo. Bamboo is the best food for a Panda bear.”

 Of course, webpages are more diverse than school reports, but you will recognize some of these same techniques on  pages you encounter.  Some  Low  quality content is created with little effort in order to have content to support  monetization rather than creating original or effortful content to help users.  “Filler” content may also be added, especially  at the top of the page, forcing users to scroll down to reach the MC.

 Here are some examples of pages with  Low  quality MC:

 The title of the page is also considered part of the MC.  Titles are important because they represent the page in search  results and help users decide what to click on.  A misleading title can lead to a poor search experience.  Exaggerated or  shocking titles are evidence of  Low  quality because of the poor user experience that results when users see the actual  MC after reaching the page.

 Here are some examples of pages with  Low  quality titles:

 The level of talent or skill required for artistic expression by ordinary people on social media or other websites that allow  content sharing should be viewed leniently.  In many cases, people post activities to share with friends, such as a video  documenting a first attempt at a new hobby.  These types of pages should not be considered  Low  on the basis of talent or  skill because the primary purpose is sharing content.  Use the  Low  rating only if the lack of talent or skill prevents the  page from achieving its purpose well.

 Important :  Low  quality MC does not meet the standard for  Lowest  and is not harmful, but it is unsatisfying for the  purpose of the page or causes the page to fail to achieve its purpose well.

 5.3       Distracting Ads/SC

 We expect Ads and SC to be visible.  However, some Ads, SC, or  interstitial pages  (i.e., pages displayed before or after  the content you are expecting) make it difficult to use the MC.  Pages with Ads, SC, or other features that significantly  distract from or interrupt the use of the MC should be given a  Low  rating.  

 A single pop-over Ad or interstitial page with a clear and easy-to-use close button is not terribly distracting, even though it  may annoy some people.  However, difficult-to-close Ads that follow page scrolls, or interstitial pages that require an app  download, can be truly distracting and make the MC difficult to use.  You can see examples of interstitial pages  here .

 The content of the Ads, SC, or other features may be distracting as well: sexually suggestive images such as  here ,  grotesque images such as  here , and sexually suggestive Ads on otherwise non-sexual pages are very distracting.

 Finally, Ads and SC can be distracting if the titles or images of the Ads or SC are shocking or disturbing.  Here  is an  example of a page with shocking and exaggerated titles, images, and text in the Ads and SC.

 Important : The  Low  rating should be used if the page has Ads, SC, or other features that significantly distract from or  interrupt the use of the MC.

 5.4       Mildly Negative Reputation of the Website or Content Creator

 The  Low  rating should be used if the website or the content creator has a mildly negative reputation.

 If the MC was created by a content creator who is using the website to host it, research the reputation of the content  creator.  Expect to find reputation information on well-known journalists, authors, social media influencers, bloggers and  vloggers, professionals such as lawyers and doctors, etc.  Pay attention when there is evidence of mildly  negative—though not malicious or financially fraudulent—reputation.  However, a lack of reputation information for  ordinary people and lesser-known content creators is expected and is not a sign of  Low  quality.

 Exercise care when researching the reputation of businesses.  Try to find enough reviews to understand a range of  customer opinions and experiences, and read the details of negative reviews and low ratings before inferring that the  business overall has a negative reputation.  A few negative customer service reviews are typical for businesses such as  stores or restaurants.

 5.5    Unsatisfying Amount of Information about the Website or Content Creator

 Expect to find some form of website information for many or most websites.  We expect clear information about who (e.g.,  what individual, company, business, foundation, etc.) created the MC, unless there is good reason for anonymity.   A  long-standing Internet alias or username can identify the content creator.  For personal websites or forum discussions on  non-YMYL topics, an email address or social media link alone may be sufficient.  Many people are reasonably hesitant to  share personally identifiable information online!

 Stores and websites that process financial transactions require a high level of user trust.  If a store or financial transaction  website has just an email address and physical address, it may be difficult to get help if there are issues with the  transaction.  Likewise, many other types of websites on clear YMYL topics also require a high level of user trust.

 Important : Pages that offer payment functionality or process other types of financial transactions should receive a  Low  rating if there is an unsatisfying amount of customer service information or contact information.  Pages on YMYL topics  and other pages that require a high level of user trust should receive a  Low  rating if there is an unsatisfying amount of  information about who is responsible for the website or who created the content.  For forums, social media pages, and  other websites where people commonly exchange opinions, a username is an acceptable identifier.

 5.6       Examples of Low Quality Pages

6.0      Medium Quality Pages

 Medium  pages have a beneficial purpose and achieve their purpose.  There is nothing wrong with  Medium  quality pages.  Expect to encounter many  Medium  quality pages in PQ rating tasks.

 To identify  Medium  quality pages, start by considering the following:

 Next, assess the page to determine if the criteria for  Medium  apply:

 6.1      Types of Medium Quality Pages

 There are two types of  Medium  quality pages:

 Important:  For a page to receive a rating  higher than  Medium , it must meet the criteria described in the next sections on

 High  and  Highest  page quality.  For a page to receive a rating  lower than Medium , it must meet the criteria for  Low  or  Lowest  page quality.  Medium  is a rating that applies to many pages you will encounter in PQ rating tasks: many pages  have nothing wrong, yet do not represent the high or highest quality content that’s available online.

 6.2       Examples of Medium Quality Pages

 7.0     High Quality Pages

 High  quality pages serve a beneficial purpose and achieve that purpose well.  High  quality pages exist for almost any  beneficial purpose you can imagine, from providing information to selling products to making people laugh to artistic  expression.

 To identify  High  quality pages, start by considering the following:

 Next, assess the page to determine if the criteria for  High  apply:

 7.1       High Quality Main Content

 High  quality MC should be satisfying for people visiting the page.  High quality MC shows evidence of effort, originality,  talent, or skill.  For informational pages, High quality MC must be accurate and consistent with well-established expert  consensus when such consensus exists.  Have high standards!  If you aren’t sure whether the content is high quality, try  finding other pages on the same topic to help calibrate your assessment: “typical” and “average” pages on a topic  generally have Medium (not High) quality MC.

 To help develop your standards, here are some examples of  High  quality MC in contrast with  Low  quality MC:

 7.2       Positive Reputation

 Positive reputation of the website can be a reason for a  High  rating if the website is responsible for the MC.  If the website  is not responsible for the MC (e.g., posts on social media websites or forums), positive reputation of the content creator(s)  can be a reason for the  High  rating.

 For non-YMYL topics, reputation information may be less formal.  Popularity, user engagement, and user reviews can be  considered evidence of reputation for non-YMYL websites: websites can be considered to have a positive reputation if  they are popular and well-known for their topic or content type.

 Remember:  Many smaller websites and ordinary people have little reputation information available.  A page can still  receive a  High  rating without reputation information.

 7.3       High Level of E-E-A-T

 Depending on the purpose of the page, topic, and type of website, a high level of E-E-A-T may be required for the page to  achieve its purpose well and be considered  High  quality.  Pages with  High  E-E-A-T are trustworthy or very trustworthy.

 Think about the topic of the page.  What kind of E-E-A-T is required for the page to achieve its purpose well?

 7.4       Examples of High Quality Pages

8.0      Highest Quality Pages

 Highest  quality pages serve a beneficial purpose and achieve that purpose very well.  The distinction between  High  and  Highest  is based on the quality of MC, the reputation of the website and content creator, and/or E-E-A-T.

 To identify  Highest  quality pages, start by considering the following:

 Next, assess the page to determine if the criteria for  Highest  apply:

 8.1       Very High Quality MC

 Very high quality MC should be highly satisfying for people visiting the page.  Very high quality MC shows evidence of a  high level of effort, originality, talent, or skill.  For informational pages, very high quality MC must be accurate, clearly  communicated, and consistent with well-established expert consensus when it exists.  Very high quality MC represents  some of the most outstanding content on a topic or type that’s available online.

 The standards for  Highest  quality MC may be very different depending on the purpose, topic, and type of website.  Here  are some examples of  Highest  quality original content:

• For news : Original reporting that provides information that would not otherwise have been known had the article not revealed it.  Accurate, original, in-depth, and investigative reporting requires a high level of skill/talent and    Very high quality news content will include a description of primary sources and other original reporting  referenced during the content creation process.  Very high quality news content must be accurate and should  meet professional journalistic standards.
• For artistic content (videos, images, photography, writing, etc.): Unique and original content created by highly  skilled and talented artists or content creators.  Such artistic content requires a high level of skill/talent and effort.  If the artistic content is related to a YMYL topic (e.g., artistic content with the purpose of informing or swaying  opinion about YMYL topics), YMYL standards should apply.
• For informational content: Original, accurate, comprehensive, clearly communicated, and should reflect expert  consensus as appropriate.  Expectations for different types of information may vary.  For example, scientific  papers have a different set of expectations than a social media post sharing information about a hobby such as  stamp collecting. However, all types of very high quality informational content share common characteristics of  accuracy and clarity of communication, in addition to meeting standards appropriate to the topic or field.

 The  Highest  rating may be justified for pages with very satisfying MC created with a very high level of effort, originality,  talent, or skill.

 8.2       Very Positive Reputation

 Reputation research is important when giving  Highest  ratings.  For YMYL topics, very positive reputation is often based  on recommendations from known experts or professional societies appropriate to the topic of the page.  Prestigious  awards may be evidence as well, depending on the topic and type of content.

 For non-YMYL topics, reputation information may be less formal.  Popularity, user engagement, and user reviews can be  considered evidence of reputation for non-YMYL websites: websites and content creators can be considered to have a  very positive reputation if they are seen as one of the best sources available online for a topic or type of content.

 Remember:  Many smaller websites and ordinary people have little reputation information.  A page can still receive a  Highest  rating without reputation information.

 8.3       Very High Level of E-E-A-T

 Very high E-E-A-T is a distinguishing factor for  Highest  quality pages.  A website or content creator who is the uniquely  authoritative, go-to source for a topic has very high E-E-A-T.  A content creator with a wealth of experience may be  considered to have very high E-E-A-T for topics where experience is the primary factor in trust.  A very high level of  expertise can justify a very high E-E-A-T assessment.  Very high E-E-A-T websites and content creators are the most  trusted sources on the internet for a particular topic.

 Think about what E-E-A-T means for the topic of the page.  How important is first-hand experience?  Who are the  experts?  What makes a source highly authoritative for the topic?  What makes a website or content creator trustworthy  for the topic?  Standards for very high E-E-A-T will differ depending on the topic of the page.

 8.4       Examples of Highest Quality Pages

 9.0     Page Quality Criteria for Specific Types of Pages

 9.1       Ratings for Encyclopedia Pages

 There are many encyclopedia-type websites.  Some are highly-respected publications that are standard references, while  some are websites with content created and edited by anonymous users, with no editorial oversight or fact checking.  We  may not always know the author of the specific encyclopedia article, and therefore must rely on website reputation  research to determine the E-E-A-T of the article. High  and  Highest  quality ratings should only be used for encyclopedias  with very good reputations for accuracy and expertise, where the article itself is well-researched with appropriate  references cited.

 A note about Wikipedia: in general, the website has a good reputation and is a very popular resource that is generally  valued for accuracy.  However, there is no single author or organization that vouches for the accuracy of Wikipedia  articles, and the quality of pages varies.   You should check individual articles to evaluate the quality of the MC.

 A Wikipedia article on a non-YMYL topic ( example ) with a satisfying amount of accurate information and trustworthy  external references  can usually be rated in the  High  range. A PQ rating in the  Medium  range is often appropriate for  pages with less extensive MC and external references.  Naturally, Wikipedia articles with very little MC should get lower  PQ ratings.  Factual inaccuracy is a sufficient reason for a  Low  or  Lowest  rating.

 9.2       Ratings for Pages with Error Messages or No MC

 In PQ rating tasks, you may encounter pages with error messages or other types of “broken” pages.  Please think about  whether the page offers help for users—did the website owner make an effort to ensure that users visiting the page have  a good experience and get help finding what they are looking for?

 Pages with an explicit error (or custom 404) message are often  Medium  quality, since they tend to clearly communicate  that there’s a problem and help users navigate to other/better pages on the website.  A rating of  High  (or occasionally  even  Highest ) may be used for the rare error message page that involves a high level of effort and original content and  provides a truly satisfying experience for users.

 Pages that are “broken” are often  Low  quality, because the broken aspect prevents them from achieving their purpose  even if the problem is temporary and likely to be fixed later on by the website owner.  However, sometimes “broken”  pages should be rated  Lowest  if the problem is not temporary and instead is due to a systemic problem with the website.  To differentiate between  Low  quality (temporarily broken) vs.  Lowest  quality (evidence of a systemic problem) pages,  check other pages on the same website.   If many other pages have the same problem, there is a systemic lack of effort  on the part of the website owner that justifies the  Lowest  rating.  Otherwise, use the  Low  rating to reflect the (hopefully  temporary!) issue with the MC on the page.

 Here are some examples:

 9.3       Ratings for Forums and Q&A Pages

 Ratings for forum and Q&A pages can be challenging.  Here is some guidance on how to approach these pages:

• The Main Content includes the question, the answers/responses, and the resulting discussions.
• Rate from the point of view of a user who visits the page , rather than a participant involved in the discussion.
• Users who post answers/responses or comments are often identified only by a username or alias. A page  can be  High  or  Highest  quality with just usernames or aliases depending on other criteria.
• The E-E-A-T of a discussion among users can often be judged by the posts or comments themselves.

 ○    For some topics, Experience is the most important dimension of Trust.  For other topics, assessing  Expertise through the posts may be important.  In some cases, the posters themselves will highlight either  their own Experience or Expertise, or other people will comment on it.

                       ○       Pages on YMYL topics require more attention to Trust and more care in the assessment of E-E-A-T.

• Highest quality forum/Q&A pages have extremely satisfying conversations, including participation from users  who have put a great deal of effort into their posts and have a wealth of Experience and/or Expertise on the topic.  Such conversations can be very satisfying because of the depth of discussion, the unique insights, or the sharing  of experiences that many would not have access to in their real-world community.
• Low quality forum/Q&A pages often lack effort (few responses, surface-level rather than in-depth discussion),  lack Experience or Expertise, contain mild inaccuracies, or show a significant lack of respect or decorum among  the participants that might deter others from joining the discussion.
• Lowest quality forum/Q&A pages may contain information or advice that is harmfully misleading, contradicts  well-established expert consensus, encourages harm towards self or other individuals/participants, etc.

 One challenge is rating pages with little to no participation, and/or no answer to an open question:

• A new  page or  recently started  discussion that consists only of a question or initial post by a person should  generally be rated  Medium  unless there are other low quality attributes.  These pages may need time to achieve  their purpose.
• Over time, no participation in a discussion (or only superficial discussion) and/or no meaningful answer to an open question may be a reason for a  Low  quality rating, especially in the presence of other low quality attributes.

 Another challenge occurs when the discussion on the page drifts, becomes combative, or becomes dominated by  misleading or spammy content.  When rating, value the insightful, meaningful discussion that exists.  If the page is a mix  of high and low quality characteristics but has insightful, meaningful discussion, the  Medium  rating may be most  appropriate as long as the page is not potentially harmful.

 Here are some examples:

 10.0  Page Quality Rating Tasks

 At first, PQ rating may seem difficult.  There are several aspects of the page and the website to look at and think about.

 Important : Do not struggle with each PQ rating.  Please give your best rating and move on.  If you are having trouble  deciding between two ratings, use the lower rating.  If you are torn between three ratings, choose the one in the middle.

 Do not consider the country or location of the page or website for PQ rating.  For example, English (US) raters should use  the same PQ standards when rating pages from other English language websites (UK websites, Canadian websites, etc.)  as they use when rating pages from U.S. websites.  In other words, English (US) raters should not lower the PQ rating  because the page location (UK, Canada) does not match the task location.

 These guidelines are specific to “regular” webpages.  Occasionally, you may be asked to rate a landing page that is not a  webpage.  For example, you may be asked to rate a PDF file, a PNG or JPEG image file, etc.  When the landing page of  the URL is not a webpage, some of the criteria in these guidelines may not apply.  In this case, please use your judgment.

 Finally, this Page Quality Rating Guideline does not completely cover every aspect of page quality.  If you find pages that  you truly believe to be  High  or  Low  quality, please rate them as such, even if the reason is based on something not  covered in these guidelines.  Please use the comment section to explain your reasoning.  As always, use your judgment.

 10.1    Instructions for Rating Page Quality Tasks

 The Page Quality task page is broken up into several parts:

1. Some initial questions about the task landing page.
2. A “PQ grid” to record your observations about PQ characteristics of the landing page.
3. The Overall PQ rating slider which records your Overall PQ rating.
4. A comment box to explain your rating.

 Some results to the initial questions will end the task early.  If the page is Porn, Foreign Language, Did Not Load, or  consists of restricted or inaccessible MC (e.g., subscription is required to view MC), you will not fill in the PQ grid or assign  an overall rating.

• Foreign Language should not be used when the language on the landing page is in  the task language, a  language that is commonly used by a significant percentage of the population in the task location, or English.
• Did Not Load should be used for pages where there is absolutely no content on the page created by the website.  There is no MC, SC, or Ads on the page.  See this  Wikipedia article  for descriptions of different types of error 

 The PQ grid is designed to be your “note pad.”  It allows you to record your observations about the landing page and the  website it belongs to.

 11.0  Page Quality Rating FAQs

Part 2: Understanding Search User Needs

 12.0  Understanding Search Users, Queries, and Results

 Why do people search the Internet?

 People use Internet search engines to perform many different tasks in different environments using different types of  devices, such as mobile phones, tablets, laptops, or computers.  Keep in mind that searches can be simple or complex,  and the underlying task that a person is trying to accomplish may take multiple steps to complete.  For example, a simple  task may be to find the director of a movie.  A complex task may be to find a movie’s showtimes nearby, purchase tickets,  and get directions to go to the theater.  Overall, search engines should make it easy for people to complete tasks by  surfacing helpful results right away.

 12.1     Important Rating Definitions and Ideas

 User:  A person who is searching.  People search everywhere: people in all countries, speaking all languages; people of  all ages, genders, races, religions, political affiliations, etc.

 Query : The text, image and/or other content that a user enters in a search engine search box or search app to do a  search.  In these guidelines, queries have square brackets around them to look like a search box.  If a user says “coffee  shops near me,” we display: [coffee shops near me].  If a user types “flights to New York” in the search box, we display:  [flights to New York].  Queries may be spoken, typed, copied and pasted, come directly from a camera or other app,  and/or be related searches that appear in search results.

 There are many different types of queries because people use search to do many things, from searching for specific  websites to looking for answers to questions to browsing videos for inspiration to searching with an image to figure out  where to buy a product.

 Query interpretation : The possible meaning(s) of the query or what the query may refer to.  Many queries have more  than one meaning.  For example, the query [apple] might refer to the computer brand or the fruit or something else.

 User Intent :  What the user is trying to accomplish when they search.  For example, if a user searches for a movie, they  may be interested in showtimes, reviews, cast, etc.  For a specific query interpretation, there may be many possible user  intents.  A single user may have many different intents for a single query!

 Locale : All queries have a locale, which is the language and region for the task. Regions are represented by a two-letter  country code, for example, “US” in the locale “English (US)”.  For a current list of country codes,  click here .  We  sometimes refer to the locale as the task location.

 User Location :  Sometimes more specific information about the location of the person searching is provided, usually a  city or state.  In rating tasks, this information may be shown on a map.  User location is very important for understanding  the query and what results are helpful for searches related to nearby places, such as [coffee shops near me].  For other  searches such as [how does gravity work], user location is not important at all, as gravity works the same way  everywhere.

 Search Engine Results Page (SERP):  The page a search engine shows after a user enters a query in the search box.  The SERP is made up of  result blocks .

 Result : We will use the word  result  to refer to the result block  and  the landing page.

• Result Block: This is an individual “block” that appears on the user’s device in response to the query.  The result  block may display information in the block itself or contain links, or may do both.
• The Landing Page  ( LP ) is the page you see after you click a link in the result block.

 12.2     Understanding the Query

 Understanding the query is the first step in evaluating the task.  Remember, a query is what a user types or speaks into  their device.

 Some tasks include a query research link, which you should use if you don’t understand the query or user intent.

 Otherwise, please do web research using Google or an online dictionary or encyclopedia.  If you still don’t understand the  query or user intent, please release the task.

 Important : If you research the query on Google, please do not rely on the top results on the SERP.  A query may have  other meanings not represented on Google’s search results pages.  Do not assign a high rating to a webpage just  because it appears at the top of a list of search results on Google.

 Think about users in your locale typing or speaking the following queries into their phone.

 12.3     Locale and User Location

 All queries have a task language and task location (referred to in rating tasks as the “Locale”).  The locale is important for  understanding the query and user intent.  Users in different locales may have different expectations for the same query.

 Some tasks show a user location in addition to a locale.

 However, for many or most queries in a given locale, the user location does not change our understanding of the query  and user intent.  Here are some examples: [facebook.com], [pictures of kittens], [distance between the earth and the  moon].

 When is the locale or user location important in understanding query interpretation and user intent?  Please use both web  research and your personal judgment to answer this question.  Ask yourself, “Would people in one city or country be  looking for something different than people in another city or country?”

 12.4     Queries with an Explicit Location

 Sometimes people tell search engines exactly what kinds of results they are looking for by adding the desired location in  the query, regardless of their own location.  We’ll call this location inside the query the “explicit location.”  The explicit

 location makes queries much easier to understand and interpret.

p é “My mom is coming

 to visit.  I need to find a  hotel for her nearby”

 Query: [ Dallas  hotels]

 Locale: English (US)

 User Location: Dallas,TX

 Explicit Location:  Dallas

p é “I need a hotel for my  trip to New York”

 Query: [ New York  hotels]  Locale: English (US)

 User Location: Dallas,TX

 Explicit Location:  New York

 Sometimes the explicit location matches the user location or locale, and sometimes it doesn’t.  

 When there is an explicit location in the query, pay attention to it!  People use explicit locations to indicate exactly what  they are looking for.

 12.5     Queries with Multiple Meanings

 Many queries have more than one meaning.  For example, the query [apple] might refer to the computer brand or the fruit.  We will call these possible meanings  query interpretations .

 Dominant Interpretation : The  dominant interpretation  of a query is what most users mean when they type the query.  Not  all queries have a dominant interpretation.  The dominant interpretation should be clear to you, especially after doing a  little web research.

 Common Interpretation : A  common interpretation  of a query is what many or some users mean when they type a query.  A query can have multiple common interpretations.

 Minor Interpretation : Sometimes you will find less common interpretations.  These are interpretations that fewer users  have in mind.  We will call these  minor interpretations .

• Reasonable minor interpretations : Reasonable minor interpretations may help fewer users, but they still are  helpful to have in a set of results because different users may want different things.
• Unlikely minor interpretations : An unlikely minor interpretation is an interpretation that is in theory possible but  is very unlikely – very few users are likely to have this in mind.

 No chance interpretation : An interpretation so unlikely that almost no user would have this in mind.  Results for no  chance interpretations feel off-topic even if they match the query words.  For example, almost no one wants a result about  an overheated pet for the English US query [hot dog].

 Query: [apple]

 Locale: English (US)

 Note: There are many interpretations for this query; only a few are shown for the purposes of this example.

                     Computer company                                      Fruit                          Person’s name                             City

               Dominant Interpretation              Common Interpretations              Unlikely Minor Interpretation

 Query: [mercury]

 Locale: English (US)

 Note: There are many interpretations for this query; only a few are shown for the purposes of this example.

 12.6     Query Meanings Can Change Over Time

 Remember to think about the query and its current meaning as you are rating.  We will assume users are looking for  current information about a topic, the most recent product model, the most recent occurrence of a recurring event, etc.,  unless otherwise specified by the query.

 The interpretation of the query [iphone], English (US) has changed over time as new iPhone models are released.  The  first iPhone was introduced in 2007.  Users searching for [iphone], English (US) at that time were looking for the new (at  the time) first iPhone model.  Most users now are looking for the most recent or upcoming iPhone model.  In the future,  new models will come out and the dominant interpretation will change again.

 12.7     Understanding User Intent

 It can be helpful to think of queries as having one or more of the following intents.

• Know query, some of which are  Know Simple  queries
• Do query, when the user is trying to accomplish a goal or engage in an activity
• Website query, when the user is looking for a specific website or webpage
• Visit-in-person query, some of which are looking for a specific business or organization, some of which are  looking for a category of businesses

 12.7.1 Know and Know Simple Queries

 The intent of a  Know  query is to find information or explore a topic.  Users want to  Know  more about something.

 Know Simple  queries are a special type of  Know  query.  Know Simple  queries seek a very specific answer, like a fact,  diagram, etc.  This answer has to be correct and complete, and can be displayed in a relatively small amount of space.  As a general rule, if most people would agree on a correct answer, and it would fit in 1-2 sentences or a short list of items,  the query can be called a  Know Simple  query.

 Know Simple  queries may be questions such as [how tall is barack obama].  Frequently,  Know Simple  queries do not  have question words.  For example, [barack obama height] has the same user intent as [how tall is barack obama], but is  not in a question format.

 Most queries are  not  Know  Simple  queries, such as:

• Broad, complex, and/or in-depth informational queries that do not have a short answer
• Ambiguous or unclear informational queries
• Informational queries on controversial topics
• Informational queries with no definitive “right answer”
• Queries where different users may want different types of information, or different sources of information
• Queries where users want to browse or explore a topic
• Queries where users want to find inspiration or ideas related to a topic
• Queries where users are seeking personal opinions and perspectives from real people

 Here are some examples where the  Know Simple  query asks for a simple fact, which can be answered correctly and  completely in a small amount of space, and the  Know  query answer is more complex.

 12.7.2 Do Queries

 The intent of a  Do  query is to accomplish a goal or engage in an activity.  The goal or activity may be to download, to buy,  to obtain, to be entertained by, or to interact with a website or app.  Do  queries may have a specific need to be  immediately accomplished or they may be more open ended and involve browsing and subsequent searches to ultimately  accomplish the goal.

 Note that the line between Do and Know queries is sometimes blurry, with some queries fitting into both categories. Don’t  worry about strictly differentiating between these two categories.  They’re given to illustrate some big-picture patterns in  user intents, and many queries do not fit neatly into one and only one of these categories.

 Here are some examples of Do Queries.

 12.7.3 Website Queries

 The intent of a  Website  query is to locate a specific website or webpage that users have requested.  This single webpage  is called the  target  of the query.

 One type of  Website  query is a URL Query, which can be:

• Exact, perfectly-formed, working URLs, such as [http://www.ibm.com] or [www.ibm.com] or [ibm.com].
• Imperfect URL queries: Queries that look like URL queries, but are not “working URLs”. These URLs do not load  if you type or paste them into your browser address bar.  Even so, we believe users have a specific page in mind.

 Here are some examples.

 12.7.4 Visit-in-Person Queries and User Location

 People use search engines to look for  Visit-in-Person  information, such as finding nearby coffee shops, gas stations,  ATMs, restaurants, etc.

 Some queries clearly “ask” for nearby information or nearby results (e.g., businesses, organizations, other nearby places).

 Some queries are not asking for nearby information or nearby results.  Here are some examples.

 And some queries could go either way.  Some users may want nearby results and others may not.

p é “I want to go to a p é “I want to shop online  Walmart near me.”  on the Walmart website”

 Query: [walmart]  Query: [walmart]

 Locale: English (US)  Locale: English (US)

 Here are some examples of queries with both visit-in-person and non-visit-in-person intent:

• [hotels]
• [post office]
• [h&m]
• [library]
• [bank of america]       [the gap]

 Sometimes, the user location can change our understanding of the query.  For users close to Austin, Texas, the query  [verbena] could have two different interpretations: a popular restaurant named Verbena or the plant verbena.

p é “What is the address p é “I want to learn more  of Verbena?”  about the plant verbena.”

 Query: [verbena]  Query: [verbena]

 Locale: English (US)  Locale: English (US)

 User Location:  Austin,TX  User Location:  Austin, TX

 In most other user locations, there is no restaurant (or anything else) named Verbena and there is just one interpretation  of the query [verbena]: the plant.  The Austin restaurant is not well-known outside of Austin, TX.

p é “I want to learn more about  the plant verbena.”

 Query: [verbena]

 Locale: English (US)

 User Location:  Flint, MI

 Use your common sense when thinking about queries and whether they have possible visit-in-person intent.

 12.7.5 Queries with Multiple User Intents

 Many queries have more than one likely user intent.  Please use your judgment when determining if a particular intent is  reasonable for queries with multiple intents.  Because so many people search for so many things, err on the side of  considering interpretations and intents to be reasonable and doing query research as needed.

 12.8     Understanding Result Blocks

 The following sections contain examples of different types of queries and results.

 12.8.1 Web Search Result Block Examples

 Web Search Result Blocks typically have a title link, a URL and a “snippet” of text describing the page. For many queries,  Web Search Result Blocks are the most helpful type of result.

 12.8.2 Special Content Result Block Examples

 Special Content Result Blocks (SCRBs) appear in the search results page, along with Web Search Result Blocks.

 Special Content Result Blocks are designed to show content directly to users on the search results page.  From working  calculators to playable videos to interactive weather information, these results help users immediately get information or  content.  Some SCRBs may also have links to landing pages.

 Important : Please assume that any interactive features work and function properly. Some notes:

• All result blocks are “screenshots” or images of search results with prominent links enabled. Unfortunately, a  screenshot or image of an interactive result block will not function as it would for a real user.   For the purpose of  rating, please assume that interactive result blocks do function as intended . Try to interact with the result  block as some links, buttons, or other features may work.
• There may be a delay between when the rating task is created and when you actually rate the block, causing some information in special content result blocks to be a few hours or even days out of date.  Stock price or  weather informational blocks are designed to give users extremely current and timely information.  However, due  to a delay in rating time, the information may no longer be accurate.  Don’t penalize a special content result block  for being out of date.   Assume that the blocks show current information for users, unless instructed 

 Part 3: Needs Met Rating Guideline

 13.0  Rating Using the Needs Met Scale

 There  are  many  different  kinds  of  queries  and  results,  but  the  process  of  rating  is  the  same:  Needs  Met  rating  tasks  ask  you to focus on user needs and think about how helpful and satisfying the result is for the users .

 This is what the Needs Met rating slider looks like:

 Use  in-between  ratings  if  you  think  the  rating  of  a  result  falls  between  two  labels.  You  can  either  drag  the  slider  or  click  on  the point that you want the slider to land on.

 13.1     Rating Result Blocks: Block Content and Landing Pages

 For Needs Met rating, you will assign a rating to each result.  Each result includes the content inside the result block and  landing pages associated with the result.  

 Which part of the result do you rate?  If a rating task specifically asks you to rate the Landing Page Needs Met or the  Block Content Needs Met of each result, then you should follow the guidance in  Section 13.1.1 .  Unless these specific  versions of Needs Met ratings are requested, you should follow the guidance below:

 For example, think about the query [what does love mean].

 Here are some examples of Special Content Result Blocks where the block should be rated primarily on the content inside  the block itself.  As always, please use your judgment.

 Note: The guidance in this section specifically applies to Needs Met ratings.  For Special Content Result Blocks that have  landing pages, you may or may not also be asked to provide PQ ratings.  In these cases, your PQ rating should always be  based on the landing page.  Please refer to  Section 14.0  for more about the relationship between Needs Met and Page  Quality ratings.

 13.1.1  Landing Page Needs Met Ratings and Block Content Needs Met Ratings

 In most cases, you should follow the instructions above when assigning Needs Met ratings.  In some cases, though, a  rating task may include special instructions that ask you to assign  Landing Page Needs Met  or  Block Content Needs  Met  ratings.  In these cases, follow the instructions below:

 13.2     Fully Meets (FullyM)

 Fully Meets  is a special rating category which only applies when all of the following are true:

• The query interpretation and user intent is specific, clear, and unambiguous.
• There is a specific website, a specific webpage, a specific know-simple fact or other specific result that all or almost all users in the rating locale and user location are looking for.
• All or almost all users in the rating locale and user location would be completely satisfied by the result.

 Most queries cannot have a Fully Meets result.  Most queries do not specify a single website or webpage.  Most queries  are not specific enough to have a single, know-simple fact as a comprehensive answer.  For most queries, different  people may want different types of results.  For many queries, a single user may need to see many different results.

 You will need to use your judgment to decide whether a result block can  Fully Meet  the user need.  Here are some  scenarios when the  Fully Meets  rating is appropriate:

• The user is clearly looking for a specific webpage or website and the result block with the specific webpage or website  Fully Meets  the user’s need.
• All or almost all users in the rating locale or user location who search for this query would be looking for the same, specific fact or piece of information.  The result block provides the information thoroughly, accurately, and clearly,  and is from a trustworthy source acceptable for all or almost all users.  Before using the  Fully Meets  rating for  queries seeking a very specific fact or piece of information, you must check for accuracy and confirm that the  information is supported by expert consensus where such consensus exists.

 Be conservative when using the  Fully Meets  rating.  Even queries that seem to have a single right “answer” may benefit  from more perspectives to satisfy all or almost all users.  When in doubt, use a lower rating.

 Note: If a result block is very close to being fully satisfying, but the block alone may not be sufficient in order to fully satisfy  all or almost all users, a rating of  Highly Meets  or  Highly Meets+  may be appropriate.

 13.2.1 Examples of Fully Meets (FullyM) Result Blocks

13.2.2 Examples of Queries that Cannot Have Fully Meets Results

 There are some queries that  cannot  have a  Fully Meets  result.  Here are some examples.

 13.3     Highly Meets (HM)

 A rating of  Highly Meets  is assigned to very helpful results for any dominant, common, or reasonable minor interpretation  or user intent of the query.

 Highly Meets  results are highly satisfying and a good “fit” for reasonable interpretations and intents.   In addition, they  often have some of the following characteristics (depending on the query):

• Entertaining or enjoyable for queries with a reasonable entertainment intent
• Representative of the opinions, perspectives and experience of real people for queries with no single right answer      For informational queries, very helpful results that, for example:

                       ○    Are easy to understand the content, e.g., helpful images or videos, an easy to understand summary

                       ○    Provide in-depth or insightful content, e.g., a deep dive explanation, an expert analysis

• For queries that benefit from recently created content, fresh results that provide up-to-date content on “what everyone is talking about right now” e.g., breaking news on a topic or the latest viral video or a hot new fashion  trend

 Highly Meets  results may be any type of content: articles, short form or long form video, images, social media posts,  forum discussions, etc.  There is no length requirement for  Highly Meets  results.  Sometimes what makes a result  satisfying is that it is short and easy to understand.  Other results may be very satisfying because of the depth and  richness of content.  For some queries, comprehensiveness may be critically important, such as queries about truly  complex topics or YMYL queries for which a short answer could be misleading.

 Accuracy is required for  Highly Meets  informational results.  Highly Meets  information pages on YMYL topics must be  accurate and trustworthy.   Highly Meets  medical and scientific information pages must represent well-established  scientific/medical consensus unless the user is clearly seeking an alternative viewpoint.

 A  query  can  have  many  Highly  Meets  results.  Search  engines  should  strive  to  provide  a  diverse  set  of  Highly  Meets  results  from  a  variety  of  websites  in  a  variety  of  content  types  from  a  variety  of  people  and  perspectives. T his  is  especially  important  for  queries  with  no  single,  “right  answer”,  subjective  queries,  queries  with  multiple  intents  and  interpretations,  queries seeking opinions and perspectives, and queries simply looking to browse or explore the web.

 For most queries,  Highly Meets  is the highest possible rating a result can receive.  Results rated  Highly Meets  represent  the most helpful content available for a given query interpretation and user intent.  Some queries have several  Highly  Meets  results because there is a variety of very helpful content available on the internet that addresses reasonable query  interpretations and user intents.  Other queries may have no  Highly Meets  results at all because very helpful content  doesn’t exist on the internet or search engines are unable to find it.  The standard for  Highly Meets  is very high.