How are PNCB certification exams developed?
From question writing to validation checkpoints, a lot of time, expertise, and financial resources go into making certification exams possible. But what are all the steps and checkpoints? How does PNCB ensure that exam questions testing your knowledge—and your score itself—are valid?
Use this webpage to understand the full process from idea to test-taking experience and beyond. For questions about this information, contact exam@pncb.org.
About Our Exams
The PNCB provides national pediatric nursing certification exams that are recognized by National Association of Pediatric Nurse Practitioners (NAPNAP), Association of Faculties of Pediatric Nurse Practitioners (AFPNP), the Society of Pediatric Nurses (SPN), and the American Academy of Pediatrics (AAP).
Find a brief history of each of PNCB’s exams, annual statistics, and information about our program’s accreditations on our Exam Overview webpage.
Content, Confidentiality, and PNCB’s Code of Ethics
Before exploring the steps in exam development, it is important to understand how unethical behavior undermines the in-depth process required to create exam questions. Exam integrity is everyone’s concern, whether you are an exam candidate, recent test-taker awaiting official results, already certified, or an influencer/mentor. Exam content must remain confidential to assure healthcare consumers that only those exam candidates who possess the required knowledge are the ones who pass the exam and earn the credential. For CPNPs, passing the exam is also linked to their ability to be licensed to practice as a nurse practitioner.
Certification exams are secure, and many important efforts are made to ensure that security with every part of the process. Test-takers are reminded of their ethical responsibility when applying and testing. Volunteer subject matter experts involved with exam development also agree to a comprehensive non-disclosure of content.
1. When applying for an exam: On each application, candidates agree to PNCB’s Honor Statement Attestation. Within this attestation, everyone must agree to non-disclosure of exam content:
- I understand that all examination materials are copyrighted and are the sole property of the PNCB. I agree not to disclose or communicate about exam questions with my colleagues, supervisors, mentors, or teachers. Violation of this confidentiality may jeopardize my certification or opportunity to become certified in the future.
2. During the exam experience: Candidates must agree to PNCB’s Non-Disclosure Agreement. In this agreement, testers must again acknowledge that they are prohibited from discussing or sharing exam content with colleagues, other candidates, etc.
- Violation of this confidentiality will jeopardize the opportunity to become certified or maintain PNCB certification.
3. After testing: PNCB receives inquiries from test-takers who want to know what questions they missed on their exam. While we understand that people want to learn from their mistakes, releasing questions (with or without answers) exposes exam questions that are still in use on exams. If those questions were to be shared outside of the secure exam environment, the security and integrity of the exam is compromised. When a test-taker is unsuccessful on the exam, an official score report will offer information about strengths and weaknesses in topic areas that may be helpful to the candidate considering future study needs. PNCB will not tell candidates how many questions were answered correctly.
- In the exam application, a notice that exam content will not be available after testing is included in the Honor Statement Attestation.
PNCB has launched the Ethics in Testing: A Personal Responsibility Toolkit to provide specific examples of acceptable and prohibited behaviors.
Exam Development Process
Certification exams are very different from academic or teacher-made exams. Creating high-stakes exams requires the efforts of exam development specialists and psychometricians who have expertise in the science of measurement and assessment. In addition, subject matter experts (SMEs) holding the credentials volunteer to be item writers and members of various exam-related committees.
Diverse Representation: With each volunteer cohort, PNCB is committed to recruiting and selecting certified individuals reflective of applicable pediatric practice in all relevant respects: geographic location, years of experience, educational background, ethnicity, gender, practice setting, and more. This diversity is key because NCCA standards specify and require the following exam development path. PNCB abides by, and must show, evidence of these processes during reaccreditation.
Here are the steps required to create exam questions and exam forms:
| Task | Details |
|---|---|
Job Task Analysis: Routine, Required Research |
All PNCB exams began with a Role Delineation Study to identify the need for an exam and gather evidence to develop exam scope. After initial exam launch, currency of content is validated periodically with a required Job Task Analysis (JTA) study. The JTA research involves soliciting input from those who hold the credential. Their responses validate what is most prevalent and/or most important in practice for the role. Details about the most recent study conducted for each of PNCB’s exams is found on the exam’s Study Resources webpage.
|
Test Blueprint |
Results from the intensive JTA survey are reviewed and used to develop the content outline, also known as the test blueprint. Major domains (categories) of knowledge are identified from the data and used to determine the number of questions in the exam by content area. Find each exam’s content outline on the exam’s Study Resources webpage.
|
Item Writers Develop Questions |
Certified volunteer subject matter experts are trained by PNCB to write and review questions, also known as ‘items.’ Each item must be aligned with a ‘task’ on the content outline, since those are the only tasks validated by the above research to be essential to practice. Item content must proportionately represent all tasks identified as essential. A note about diversity within item text:
Learn about Item Writer qualifications, responsibilities, and term-limits. In-depth confidentiality / non-disclosure forms are required. |
Exam Committee Members Review and Approve Questions |
The Exam Committee, a group of subject matter experts, demonstrate excellence in item writing and editing, and conduct a rigorous review all newly written items. Members of this committee represent credential holders of the respective exam and are diverse in their relevant role, professional experience, practice settings, and collaborative skills among peers. After group consensus for each item is achieved, items are approved for pretesting based on relevance to practice, accuracy, and currency.
Learn about Exam Committee Member qualifications, responsibilities, and term-limits. In-depth confidentiality / non-disclosure forms are required. |
Exam Form Assembly |
PNCB works with a contracted test development vendor who inserts additional layers of review before generating an exam form(s).
By the time the form reaches a candidate, every scored item has already been tested on hundreds of other test candidates. Unscored items are pretested to see how they perform before their possible use as scored items. Candidates will not be able to distinguish between scored and non-scored questions. Before an item achieves scored status, a PhD-prepared psychometric team collaborates with PNCB to confirm performance and validity of pre-test items. All items are based on a reference found on the Exam Reference List found on our website. (PNCB does not endorse nor have any proprietary relationship with textbooks listed or their publishing companies).
|
Form Review Committee |
Before a new exam form is launched, another distinct group of subject matter experts holding the credential review a form in its entirety. Their role is to validate item content, currency and relevancy for practice, and alignment with the new content outline. These experts also look for item duplication or cueing. Cueing is when one test item could inadvertently help a test-taker answer another item on the same form.
Learn about Form Review Committee Member qualifications, responsibilities, and term-limits. In-depth confidentiality / non-disclosure forms are required. |
Standard Setting |
When new exam forms are created following a job task analysis (JTA), another group of volunteer subject matter experts is convened to determine the minimum raw score needed to pass a form. The passing score is decided using standard psychometric methods, such as Item Response Theory and Modified-Angoff scoring. The experts must look at the form as a whole and again at each item. Each item must be re-analyzed and weighted.
Learn about Standard Setting Committee Member qualifications and responsibilities. In-depth confidentiality / non-disclosure forms are required. |
As you can see, it is a time-intensive process to create a new exam form. A certification board with 4 exam programs like PNCB must spend roughly $1.5 million dollars annually for the above exam development activity as well as exam delivery and seat fees for candidates. Other associated costs related to ongoing accreditation, copyright efforts, and PNCB staff time are not included in this number.
Exam Scoring and Validity
The CPN, CPNP-PC, and CPNP-AC exams contain 175 multiple-choice items. Of these, 150 are scored questions and 25 are non-scored pretest questions. The PMHS Exam contains 150 multiple-choice items, 25 of which are pretest questions. On any exam form, pretest questions are dispersed throughout the exam and are not distinguishable from scored questions.
PNCB exams are criterion referenced. This means the passing score is based on predetermined criteria. Your ability to pass the exam depends on the knowledge you display, not on the performance of other candidates.
Passing scores may vary slightly for each version of the exam. To ensure fairness to all candidates, a process of statistical equating is used. This involves:
- Selecting an appropriate mix of individual questions for each version of the examination that meet the content distribution requirements of the content outline.
- Obtaining an overall difficulty level for the form (as each scored question has been pretested, a difficulty level is available); the standard setting process then considers the overall difficulty level of each exam form.
- Slight variations in difficulty level are determined, then addressed by adjusting the passing score up or down, depending on the overall difficulty level statistics for the group of scored questions that appear on a particular form of the exam form.
PNCB uses scaled scores that range from 200 (0 items correct) to 800 (all items correct) with a scaled score of 400 corresponding to the cut-point on the exam. This means that the passing score across all forms is 400. Using a scaled score is how a certification board may put all scores, regardless of which form the candidate takes, on the same scale, or frame of reference.
