Research Projects
Goals
- Create cognitively-principled, simulation-based academic games that assess and promote students’ learning about science systems, inquiry practices, and collaboration.
- Specify game design principles for the foundational topic of ecosystems based on research on cognition, model-based learning, achievement motivation, game design, and evidence-centered assessment design.
- Document the effectiveness, feasibility, and utility of the games for promoting and assessing understanding of science systems and models and strategies for studying them.
- Explore the potential of games to promote and measure collaboration.
- Document the design principles for SimScientists Games that will support and accelerate development of games for additional science systems.
Summary
Educationally sound games have the potential to draw diverse populations of students into science. Publications of the research on principles for the design of effective educational games can provide models for harnessing the power of technology to engage students in deeper science. Rigorously designed learning games can pioneer unobtrusive methods for monitoring learning progress while enhancing motivation. The SimScientists Games project will produce a new genre of games as formative assessment resources designed to collect evidence of learning during gameplay and reinforce middle grades life science concepts and investigation practices for ecosystems. The game designs will draw upon research-based principles for effective learning, game architecture and mechanics, and methods of evidence-centered assessment design to develop simulation-based games to monitor knowledge and skills and to provide additional feedback and instruction to improve learning. The games will build on the designs of the simulation-based curriculum-embedded assessments developed in the SimScientists program that have research results showing improvements in learning, reliability and validity, and feasibility of classroom use. The games will follow the simulation-based curriculum-embedded assessments, differentiate game entry levels, and provide ongoing feedback and individualized coaching and learning reports. The games are intended to provide engaging activities for understanding and investigating the science system components, roles, interactions and population dynamics of all ecosystems.
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the National Science Foundation under Grant No. DRL-1503481
Human Body Systems: Using Simulations to Foster Integrated Understanding of Complex, Dynamic, Interactive Systems Human Body Systems: Using Simulations to Foster Integrated Understanding of Complex, Dynamic, Interactive Systems
Goals
- Develop an integrated simulation of molecules, cells, and human digestive, circulatory and respiratory systems that allows students to develop productive inquiry strategies.
- Embed the simulation in online instructional modules that provide immediate, individualized coaching as students are challenged with a series of investigative tasks.
- Provide reports of students’ performances during the activities to students and teachers.
- Develop follow-up online collaborative investigations that provide differentiated instruction to strengthen students’ understanding and support transfer and opportunities to engage in scientific discourse.
- Develop one benchmark assessment that measures students’ integration of systems and the use of science practices and reports to students and teachers.Develop and deploy professional development to support teachers as they use these materials.
- Provide evidence of the technical quality, feasibility, and usability of the new materials.
- Study the influence of these materials on complex science and inquiry learning of the integrated multilevel systems of the human body.
Summary
The SimScientists program at WestEd, working with Project 2061 at the American Association for the Advancement of Science (AAAS), is designing, developing and testing SimScientists Human Body Systems, a new technology-enhanced learning environment for high school teachers to use during biology instruction and investigating how the environment helps students develop a deeper understanding of how molecules, cells, and organs work together to support life. The project leverages and extends ongoing work by the SimScientists program in which simulations are used to assess and foster students’ knowledge and inquiry skills in middle school science topics such as ecosystems, atom and molecules, and climate. This project focuses on improved instruction and learning about the role of molecules, cells, and three major human body systems (circulation, respiration and digestion) in one essential function they serve for the organism: deriving energy and building materials from food. By using simulations that show how individual components of the systems function, how the components work together, and how the integration of the components creates a dynamic system that reacts to changing inputs, the project will foster a much deeper understanding of the human body.
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the National Science Foundation under Grant No. DRL-1020264.
Goals
- Create and validate a model demonstrating the integration of science simulations at classroom, district and state levels into a state science assessment system,
- Develop and validate a strand of classroom-based simulation assessments for key life science systems at the middle school level (Ecosystems, Cells and Human Body Systems),
- Adapt the simulation environments and evidence-centered designs of the classroom assessments to develop a set of shorter, “signature tasks” with technical quality suitable for administration in a district or state test,
- Study the predictive validity of the classroom-based curriculum-embedded and unit benchmark simulation assessments for performance on a year-end accountability test,
- Cross-validate the set of life science signature tasks as potential accountability measures that could be used by state assessment consortia, and
- Prepare a Policy Brief documenting the validation studies and recommending models for integrating science simulations into multilevel, balanced assessment systems.
Summary
The shockingly low performance of U.S. students on national and international science tests highlights the need for a major transformation of science assessment to measure understanding of complex systems, model-based reasoning, and inquiry practices. A new generation of technology-based assessments is revolutionizing the what, how, when and where of testing. With the Race to the Top Assessment initiatives focusing primarily on mathematics and literacy, the need for the transformation of science assessment is profound and urgent.
WestEd is conducting this Measurement project to create and validate transformative simulation-based science assessments that can become components of multilevel balanced state science assessment systems. This project will build on evidence of technical quality, feasibility, and utility from three other SimScientists projects funded by NSF and IES. The project assessment design includes alignment with national science standards, templates, simulation shells, and external quality reviews by AAAS. Data collection involves student and teacher cognitive labs, pilot tests and field tests to document the technical quality, usability, and feasibility of the assessments.
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the U.S. Department of Education: Institute of Education Sciences (IES) under Grant No. R305A120390.
SimScientists Crosscutting Concepts: Progressions in Earth Systems SimScientists Crosscutting Concepts: Progressions in Earth Systems
Goals
- Build on the ground-breaking SimScientists program at WestEd and leverage computerized, simulation-based instructional modules with embedded formative assessment, to determine how learning of crosscutting concepts of Systems, Scale, and Cycles progresses across three topics in middle school Earth science,
- Create and validate hypothesized learning trajectories for three crosscutting concepts (Scale; Systems; Cycles), and
- Establish the efficacy of the modules in a research study with 13 teachers and 1300 students.
Summary
The performance expectations of the NGSS integrate core ideas with science practices and crosscutting concepts. This project will develop detailed learning trajectories for three crosscutting concepts identified in the Next Generation Science Standards: Scale, Systems, and Cycles. Foundational research on STEM learning of these concepts will be rooted in Earth system curriculum modules about plate tectonics, climate, and ecosystems. Our research studies and iterative development will test claims that Earth systems models will promote different and improved ways of learning three crosscutting concepts and are valid for three different Earth science topics. This project will influence the field in several ways. First, there are few learning progressions available particularly for the crosscutting concepts identified by the recent frameworks. Second, this project will support research efforts that help bring the framework’s vision of integrated learning across the three dimensions of crosscutting concepts, core ideas, and practices to fruition in classrooms. Third, the focus on crosscutting concepts for simulation-based science curriculum activities and assessments are important and necessary enhancements to curriculum programs because they can promote complex science learning and sustained investigation practices.
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the National Science Foundation under Grant No. 1420386.
Goals
- Specify, test, refine, and validate a system model framework and proposed learning trajectories for three core life science systems (genetics, evolution, and ecosystems),
- Promote and assess students’ model-based learning for the three systems,
- Embed the proposed learning trajectories in interactive instruction and assessment modules that use science practices and feature dynamic models representing the components, interactions, and emergent behavior of each system, and
- Document the effectiveness, feasibility, and utility of the simulation-based science environments for promoting and assessing learning about science systems.
Summary
SimScientists Model Progressions is taking the systems model approach advocated in the Framework for K12 Science Education and the Next Generation Science Standards to develop and validate (1) dynamic, interactive simulations that model the components, interactions, and emergent behaviors for three life science systems (Genetics, Evolution, and Ecosystems), and (2) three proposed learning trajectories composed of performance expectations that combine core ideas and science practices spanning the model levels of each system. Simulation-based modules will promote and assess core ideas and science practices in the proposed learning trajectory. The modules are intended to activate and reorganize the typically disconnected, static body of science knowledge transmitted by textbooks, engage students in developing and using system models, promote active investigations of system dynamics, and refute common misconceptions.
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Institute of Education Science. This material is based upon work supported by the Institute of Education Science under Grant No. R305A130160
Goals
- Validate a multilevel model of a balanced state science assessment system demonstrating the integration of simulation-based assessments at classroom, district and state levels,
- Develop and validate a Physical Science strand of classroom-level, simulation-based assessments for four fundamental science systems (Matter, Motion, Energy, Waves) taught in middle school,
- Adapt the simulation environments and evidence-centered designs of the classroom assessments to develop a set of shorter Signature Tasks for use in district or state tests,
- Study the predictive validity of (1) simulation-based, curriculum-embedded assessments for predicting performance on unit interim benchmark assessments, that in turn (2) predict performance on a year-end Physical Science accountability test,
- Cross-validate the Physical Science strand of simulation-based classroom and year-end Signature Tasks as credible accountability measures for state assessment consortia, and
- Prepare a Policy Brief documenting the validation studies and recommending models for integrating simulation-based assessments into multilevel, balanced science assessment systems.
Summary
The new Framework for K-12 Science Education and the forthcoming Next Generation Science Standards are steering the nation toward more challenging, focused science goals. A new generation of technology-powered assessment is being developed; classroom-based formative assessments are seen as essential for monitoring learning; and serious attempts are being made to redress the major disconnect between classroom assessments and state testing. WestEd is conducting a project that builds on the design templates, technical infrastructure, and extensive evidence-base of the technical quality, feasibility, and instructional utility of the NSF-funded Calipers II (DRL-0733345) assessments. The project will develop and validate a physical science strand of simulation-based, curriculum-embedded assessments for formative use, unit benchmark assessments for summative use, and a set of signature tasks useable for district and or state accountability testing. The multi-level assessment designs will be aligned via the new common science standards, based on reusable simulation-environment designs, and most importantly, shaped by common principled assessment design specifications. The simulation-based assessments address disciplinary core science principles and strategies such as understanding the workings of dynamic science systems and how to use the inquiry practices of science, standards that are seldom addressed well or at all in static paper testing formats.
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the National Science Foundation under Grant No. DRL-12211614.
Goals
- Developed assessment designs and exemplars that take advantage of simulation environments to provide benchmark assessments of achievement of science standards that can be used as formative information about conceptual understanding and extended inquiry, as well as evidence for external accountability purposes;
- Documented the assessment task design templates and technical infrastructure and processes employed;
- Provided evidence of the feasibility, usability, and technical quality of the new simulation-based assessments;
- Prepared a plan for development of a pool of complex assessments linked to key strands in the AAAS Atlas of Science Literacy and core National Science Education Standards (NSES).
Summary
The project, Calipers: Using Simulations to Assess Complex Science Learning (Calipers I), was a demonstration project that used simulations to develop a new generation of assessment systems that can provide high-quality evidence of complex science learning for both accountability and formative goals. The prototype assessments address key topics in middle and high school science: forces and motion (physical science) and ecosystems (life science). The assessments engaged students in problem-based investigations designed to test deep science knowledge and extended inquiry skills.
The project developed simulation-based assessments for two fundamental science key ideas, "Forces and Motion" and "Ecosystems." The assessment designs were aligned with National Science Education Standards and the AAAS Benchmarks. Curriculum analyses aligned the assessment prototypes with representative text-based and technology-based curricula. Simulation shells and storyboards guided the programming of the interactive assessments.
Partners
From 2004 to Spring, 2007, the project was led by Principal Investigator, Edys Quellmalz, in a partnership of the Center for Technology in Learning at SRI International and the Concord Consortium. In Spring 2007, Dr. Quellmalz joined WestEd, and the Calipers project was transferred to WestEd for its completion in Winter, 2008.
Disclaimer Text
The Calipers project was supported by the National Science Foundation under Grant No. 0454772 awarded to SRI International, Edys Quellmalz, Principal Investigator, and to WestEd under Grant No 0741029. Any opinions, findings, and conclusions or recommendations expressed in the materials are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.
Calipers II: Using Simulations to Assess Complex Science Learning Calipers II: Using Simulations to Assess Complex Science Learning
Goals
- Develop formative and benchmark simulation-based assessments of science knowledge for key content in physical, life, and earth science and for science inquiry strategies.
- Enhance formative assessment simulation modules with immediate, individualized coaching, diagnostic reports, and follow-up reflection activities.
- Develop and document technology-based assessment designs and exemplars that take advantage of simulation environments to provide assessments of science standards for formative and summative purposes.
- Document the technical infrastructure and re-usable designs and processes employed.
- Provide evidence of the technical quality, feasibility, and usability of the new assessments.
- Study the influence of formative assessments on complex science and inquiry learning.
- Link the enlarged collection of Calipers II benchmark and formative, embedded assessments to national science standards and the AAAS item clusters.
Summary
The Calipers II project is creating a new generation of simulation-based, curriculum-embedded formative assessments with immediate, individualized coaching for students. Students and teachers receive diagnostic reports and teachers use supporting reflection activities to improve student learning during instruction. Calipers II provides evidence that formative, embedded and summative, benchmark classroom assessments with technical quality can gather and document evidence of students’ learning of connected science knowledge and extended inquiry not often or well measured by conventional tests.
Partners
American Association for the Advancement of Science (AAAS/Project 2061) and the Center for Research on Evaluation, Standards & Student Testing (UCLA/CRESST).
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the National Science Foundation under Grant No. 0733345.
Goals
- Investigate the construct validity of three types of technology-enhanced assessment tasks and items designed to measure complex science learning.
- Compare the construct validity of dynamic (active and interactive) assessment task and item designs to those of static formats intended to measure the same complex learning.
- Identify design principles for types of science assessment task design structures that draw on the capabilities of technology to elicit knowledge of science systems and inquiry abilities.
Summary
The Foundations of 21st Century Science Assessments project is extending principles from the fields of cognition and measurement to the design and validation of a new generation of dynamic science assessments. The project will contribute reusable research-based design principles for developing dynamic science assessment tasks that take advantage of the affordances of technology to measure integrated knowledge structures and inquiry strategies. The project is using the evidence-centered assessment framework to develop and validate reusable task designs. By merging advances in cognitive learning with advances in measurement and assessment research, the study will forge a new, principled framework for studying and documenting the construct validity of technology-enhanced science assessments. WestEd is collaborating with the American Association for the Advancement of Science (AAAS) to conduct this research on the design and construct validity of science assessments that use static and dynamic task designs to test challenging science standards.
Integrating Simulation-Based Science Assessments into Balanced State Science Assessment Systems Integrating Simulation-Based Science Assessments into Balanced State Science Assessment Systems
Goals
- Establish the technical qualities of the simulation-based science assessments when used on a large scale.
- Study the feasibility and utility of the assessments for formative, summative, and accountability purposes.
- Study the effects of the simulation-based assessments for all students.
- Study the effects of the assessments for English learners and for students with disabilities.
- Propose alternative models for integrating simulation-based assessments into state science assessment systems.
Summary
The Integrating Simulation-Based Science Assessments into Balanced State Science Assessment Systems project is taking the simulation-based assessment modules developed in the Calipers project and enhancing them by providing accommodations for students with special needs. Accommodations for additional time, large print and text to speech are being added to the Calipers assessment system. Two topics (Ecosystems and Force & Motion) will be pilot tested with thousands of students in four states in the 2009/10 school year. Assessments will be embedded in curriculum units at key points, engaging students in active investigations, and providing immediate, customized coaching and diagnostic reports. The formative assessments also offer guidelines for off-line reflection activities. In addition, the summative, end-of-unit benchmark assessments will report on student achievement using the classifications of the state testing program.
The project investigates how simulation-based assessments can be used as part of a multilevel, balanced state science assessment system and how the results of the assessments could be integrated into existing district and state accountability systems. The project studies how the rich environments and multiple representational formats in the technology-enhanced science assessments benefit the performance of disadvantaged and low-performing students. The project studies how the accommodations for large print, extended time, speech-to-text, and text-to-speech increase accessibility and affect the performance of English learners and students with disabilities.
Goals
- Use systematic design principles to develop formative assessments to be used during (embedded) and at the end of (benchmark) science curriculum units.
- Use systematic design principles to create a coherent, multilevel state science assessment system by aligning (1) the designs of the embedded assessments with the designs of the benchmark assessments (horizontal alignment), and (2) the designs and items of the embedded and benchmark assessments with relevant specifications and items on the state science test (vertical articulation).
- Study the impact of the formative assessments on student learning.
- Validate the use of data from the embedded and benchmark assessments for interpreting student performance on the targeted science standards.
- Describe the components of the formative assessments and their implementation so that they can serve as scalable models.
Summary
The Multilevel Assessment of Science Standards (MASS) project is creating a new generation of technology-enhanced formative assessments that bring best formative assessment practices into classrooms, transforming what, how, when, and where science learning is assessed. Simulation-based tasks with immediate, individualized coaching provide evidence that embedded and benchmark classroom assessments with technical quality can gather, document, and promote students’ learning of connected science knowledge and extended inquiry not measured by large-scale tests.
Simscientists: Interactive Simulation-Based Science Learning Environments Simscientists: Interactive Simulation-Based Science Learning Environments
Goals
- Develop prototype simulation-based curriculum and assessment modules for three topics in middle school science that will supplement and extend the science knowledge and skills typically addressed in static print materials,
- Improve the science learning of connected knowledge structures and inquiry skills,
- Provide professional development aligned with the science content and skills,
- Document the effectiveness, feasibility, and utility of simulation-based science environments for promoting and assessing science standards, and
- Document the design tools for SimScientists and the implementation instruments that will support and accelerate development of a comprehensive set of SimScientists modules.
Summary
WestEd is developing and documenting the feasibility and effectiveness of a set of simulation-based curriculum supplements for teaching complex science concepts to middle school students. The SimScientists project breaks the mold of the prevailing model of static, text-based transmission of science knowledge by creating a powerful, dynamic intervention that addresses the urgent need to promote deep understanding of scientific ideas and the practices of science.
The SimScientists project incorporates design principles for effective instruction and assessment derived from learning research. Simulation-based modules are designed to supplement and extend existing science instructional materials. SimScientists modules contain seven components: (1) a simulation environment modeling age-appropriate key principles of a science system, (2) problem-driven curriculum inquiry activities, (3) embedded formative assessments with an intelligent scoring system that will provide immediate, individualized feedback and worked examples, (4) a coaching system tailored to individual responses and providing multiple re-teaching scaffolds, (5) offline self-assessment and reflection activities to build students’ argumentation and metacognitive skills, (6) technically sound, end-of-unit benchmark assessments of progress, and (7) standards-based professional development.
Disclaimer Text
Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation. This material is based upon work supported by the U.S. Department of Education: Institute of Education Sciences (IES) under Grant No. R305A080614.