National Assessment of Educational Progress (NAEP) Science Assessment Instrument

Evaluates students' knowledge of three fields of science (earth, physical, and life), three elements of knowing and doing science (conceptual understanding, scientific investigation, and practical reasoning), and two overarching domains in science (the nature of science and themes-systems, models, and patterns-present in science).

Average Review: (5.0)

Supplemental Information:

n/a

Assessment Type:

Multiple choice, short constructed response questions, and extended constructed response questions.

Scale:

N/A

Publication Date:

Nov 07, 2008

Respondent:

4th, 8th and 12th grade students

Domain(s) Evaluated:

Attitude / Behavior, Content / Knowledge, Competence, Career Knowledge / Acquisition

Sample items:

4th Grade
List four ways that the Earth is different from the moon.

8th Grade
What property of water is most important for living organisms?
(a) It is odorless.
(b)It does not conduct electricity.
(c)It is tasteless.
(d)It is liquid at most temperatures on Earth.

12th Grade
In the space below, draw a rough sketch (not necessarily to scale) illustrating the simplified model of the Solar System by showing the Sun and the four inner planets with their orbits. Be sure to label the Sun and each planet.

Reliability:

Cohen's > 0.80

Validity:

High consistency across multiple markers.

Frequency:

Frequently

Administration time:

70 minutes

Requires a Computer:

Requires Internet Access:

Primary reference:

Allen, N.L., Carlson, J., & Zelenak, C.A. (1998). The NAEP 1996 Technical Report. Retrieved from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=1999452

Comments:

The full list of available NAEP student questionnaires can be found here: https://nces.ed.gov/nationsreportcard/experience/survey_questionnaires.aspx

To learn about the NAEP Science Assessment, go here: http://nces.ed.gov/nationsreportcard/science/

Publications related to the Science NAEP can be found here: http://nces.ed.gov/pubsearch/getpubcats.asp?sid=031

The content-specific questions pool for science can be accessed here: http://nces.ed.gov/nationsreportcard/itmrlsx/search.aspx?subject=science

The content-specific questions pool for Mathematics can be accessed here: http://nces.ed.gov/nationsreportcard/itmrlsx/search.aspx?subject=mathematics

A study using NAEP:
Schneider, R. M., Krajcik, J., Marx, R. W., & Soloway, E. (2002). Performance of students in project-based science classrooms on a national measure of science achievement. Journal of Research in Science Teaching, 39(5), 410-422.

A report using NAEP:
Campbell, J. R., Hombo, C. M., & Mazzeo, J. (2000). NAEP 1999 trends in academic progress: Three decades of student performance. Washington, D.C.: U.S. Dept. of Education, Office of Educational Research and Improvement.

Other References:
Reilly, D., Neumann, D. L. & Andrews, G. (2014). Sex differences in mathematics and science achievement: A meta-analysis of national assessments of educational progress assessments. Journal of Educational Psychology, 107(3), 645-662.

This study used the NAEP instrument in their paper and analyzed it for reliability. High consistency across multiple markers was found for the response items for mathematics and science. Cohen's was > 0.80. Item response theory was used to measure latent score, ensuring high reliability. NAEP was compared to a linking study finding the trends comparable to international standards.

Pellegrino, J. W. (2013) Proficiency in science: Assessment challenges and opportunities. Grand Challenges in Science and Education, 340, 320-324.

This article does not do any statistical tests on the assessment, however, it does discuss, in detail, the positive attributes of NAEP compared to assessments testing similar aspects. It specifically talks about how NAEP is one of the few assessments that approximates most of the performance expectations discussed in the NRC framework, the survey aligns with the descriptions of proficiency. Also discussed was the malleable and changeable nature of the survey. It undergoes major revisions nearly every decade, keeping it up to date with current research.