NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 45 of 72
STUDENT ASSESSMENTS
AND ASSOCIATED GROWTH MODELS FOR TEACHER AND PRINCIPAL EVALUATION
PUBLICLY AVAILABLE SERVICES SUMMARY
This form will be posted on the New York State Education Department’s Web site and distributed through other means for all applications that are approved in conjunction with this RFQ to allow districts and BOCES to understand proposed offerings in advance of directly contacting Assessment Providers regarding potential further procurements.
Assessment Provider Information
Name of Assessment Provider: Curriculum Associates, LLC
Assessment Provider Contact Information:
Don Masters, Regional Vice President of Sales 315-350-4988 | [emailprotected]
Name of Assessment: i-Ready® Diagnostic for Mathematics i-Ready® Diagnostic for Reading
Nature of Assessment: ASSESSMENT FOR USE WITH STUDENT LEARNING OBJECTIVES WITH A TARGET SETTING MODEL; OR
SUPPLEMENTAL ASSESSMENT WITH AN ASSOCIATED GROWTH MODEL:
GAIN SCORE MODEL GROWTH-TO-PROFICIENCY MODEL STUDENT GROWTH PERCENTILES PROJECTION MODELS VALUE-ADDED MODELS OTHER:
What are the grade(s) for which the assessment can be used to generate a 0-20 APPR score?
K–12
What are the subject area(s) for which the assessment can be used to generate a 0-20 APPR score?
ELA and mathematics
What are the technology requirements associated with the assessment?
i-Ready runs on most standard PC- and Mac-based systems, using common browsers and standard configurations. System requirements can be found at: www.i-Ready.com/support
Is the assessment available, either for free or through purchase, to other districts or BOCES in New York State?
YES
NO
Please provide an overview of the assessment for districts and BOCES. Please include:
A description of the assessment;
A description of how the assessment is administered;
A description of how scores are reported (include links to sample reports as appropriate);
A description of how the Assessment Provider supports implementation of the assessment, including any technical assistance. (3 pages max)
FORM C
mailto:[emailprotected]
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 46 of 72
Previously approved by the NYSED for use as a measure of teacher and principal effectiveness for grades K–12
under Education Law §3012-c, i-Ready Diagnostic for reading and math is an effective, research-based, web-based
diagnostic assessment for students in grades K–12.
Using a computer with internet access and a headset, students take the online diagnostic that assesses down to
the sub-skill level in reading and math. i-Ready’s sophisticated computer-adaptive algorithms ensure learners are
assessed efficiently across a number of knowledge domains. The questioning format adapts as students respond
to each question—getting more or less challenging as needed—to complete the diagnosis and identify each child’s
performance level. The adaptive nature of the assessments meets students at their own skill level, so they
experience success as well as challenge while i-Ready accurately measures their mastery of New York State
Learning Standards.
i-Ready includes a powerful management and reporting suite for delivery of essential performance information at
the district, school, class, and student levels. Actionable, real-time reports guide educators in identifying the
instructional needs and abilities of individual students and instructional groups, and include explicit next steps for
remediating areas of academic weakness. For sample reports, go to
www.curriculumassociates.com/products/iready/i-ready-reports.aspx. For a narrated program tour, go to www.i-
ready.com/tour.
i-Ready Diagnostic is aligned to the New York Learning Standards for reading and math. In an independent study
conducted by the Educational Research Institute of America, i-Ready was found to have strong correlations to the
2013 and 2014 New York State Assessments. In 2013, correlations ranged from .77-.85 across grades and subjects.
In 2014, the correlation for ELA across grades 3-8 was .82 and in mathematics across grades 3-8 was .81. In
addition, i-Ready successfully predicted proficiency on the assessment for 85 percent of students. Plus, i-Ready
accurately identified individual student needs on the standards to drive targeted instruction—both student and
teacher-led.
Scoring and Reporting. The primary function and purpose of i-Ready Diagnostic is to make appropriate
instructional recommendations and placement decisions for students performing at different levels within the K–
12 grade span. A grade-level-ready student has demonstrated sufficient skills at the beginning of the school year
that he or she is considered ready for curriculum at the chronological grade. To determine scale score thresholds
for the performance standard for each grade level, a separate performance standard-setting meeting was held for
each subject.
One of the greatest advantages of using the i-Ready system over traditional paper-based assessments is the fact
that test results are instantly available to administrators once students have completed the test. i-Ready provides
numerous reporting views that make the viewing, sorting, and analysis of data straightforward and fast—and
reports are focused on accuracy and ease of access to a range of meaningful data. Access is secure via unique user
logins and an intuitive interface, interpretation of results is streamlined for educators of all backgrounds and
experience levels, and there is an emphasis on the data that is most likely to inform effective instructional decision
making.
The program is web-based, so all reporting is instantaneous and available at anytime, anywhere the authorized
user has Internet access. Users receive unique logins that enable a customized view of the data. For instance, each
teacher has access only to his or her class(es), while a superintendent has access to all schools, classes, and
individual students in his or her district. Data are also available for individual domains and by teacher, so overall
gains over time may be tracked. All reports may be printed or downloaded in PDF; many data may also be
exported as CSV files.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 47 of 72
New York HEDI Report. This district-level report (see sample in Appendix A) shows teacher performance in
summary and detail form, based on the four levels of teaching effectiveness using district-wide targets—Highly
Effective, Effective, Developing, and Ineffective.
Instructional Modules. Math and ELA instructional modules within i-Ready Instruction are available as an optional
add-on to i-Ready Diagnostic. The instructional component adapts to the student’s performance level to deliver
differentiated instruction. Student Response to Instruction Reports are then immediately available to the teacher
to inform instruction.
Implementation Plan Overview. Curriculum Associates employs a straight-forward account set-up process to get
school districts and BOCES up and running quickly with i-Ready. We support LEA and school staffs in assessment
administration and analysis of results:
1. We assign a primary point of contact (Account Manager) to the LEA.
2. The LEA works with the Account Manager to set up the site accounts prior to training and professional
development.
3. We hold a deployment meeting to determine the LEA’s specific needs and set the training schedule.
4. We offer professional development via customized onsite sessions, on topics such as understanding and
administering i-Ready assessments, accessing and analyzing student results, and using i-Ready data to
make informed instructional decisions.
5. We offer administrator training on topics that include implementing i-Ready and effectively using the
assessment as a measure of student growth for purposes of teacher and principal evaluation.
Our in-house Technical Support and Customer Services teams are available throughout the implementation to
assist users with any ongoing needs. i-Ready users may call, email, or chat with Curriculum Associates’ support
team: 800-225-0248, www.i-Ready.com, or [emailprotected]. Phone support is available Monday
through Thursday from 8:30 AM through 7:00 PM Eastern and Friday from 8:30 AM through 5:00 PM (excluding
holidays). After-hours support is available via email.
Please provide an overview of the student-level growth model or target setting model for SLOs for districts and BOCES, along with how student-level growth scores are aggregated to the create teacher-level scores, and how those teacher-level scores are converted to New York State’s 0-20 metric. In the 2015–2016 school year, Curriculum Associates has recommended using a simple gain score as targets for
student growth for each grade and subject. A simple gain score is the difference of the last i-Ready Diagnostic
assessment and the first i-Ready Diagnostic assessment (the gain). The recommended gain score targets by grade
and subject are presented in the tables below.
Grade
READING/ELA
0.75 Year Ranges
0.75 Year Suggestion
1.0 Year Ranges
1.0 Year Suggestion
1.5 Year Ranges
1.5 Year Suggestion
2.0 Year Ranges
2.0 Year Suggestion
K 34 – 45 34 46 – 60 46 69 – 90 90 92 – 120 120
1 34 – 45 34 46 – 60 46 69 – 90 90 92 – 120 120
2 29 – 39 29 39 – 52 39 59 – 78 78 78 – 104 104
3 22 – 32 22 30 – 44 30 45 – 66 66 60 – 88 88
4 14 – 20 14 19 – 27 19 29 – 41 41 38 – 54 54
5 14 – 20 14 19 – 27 19 29 – 41 41 38 – 54 54
6 11 – 17 11 15 – 23 15 23 – 35 35 30 – 46 46
7 11 – 17 11 15 – 23 15 23 – 35 35 30 – 46 46
8 11 – 17 11 15 – 23 15 23 – 35 35 30 – 46 46
mailto:[emailprotected]
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 48 of 72
Grade
MATHEMATICS
0.75 Year Ranges
0.75 Year Suggestion
1.0 Year Ranges
1.0 Year Suggestion
1.5 Year Ranges
1.5 Year Suggestion
2.0 Year Ranges
2.0 Year Suggestion
K 24 – 30 24 32 – 41 32 48 – 62 62 64 – 82 82
1 24 – 30 24 32 – 41 32 48 – 62 62 64 – 82 82
2 22 – 28 22 30 – 39 30 45 – 59 59 60 – 78 78
3 21 – 27 21 28 – 37 28 42 – 56 56 56 – 74 74
4 16 – 23 16 22 – 31 22 33 – 47 47 44 – 62 62
5 16 – 23 16 22 – 31 22 33 – 47 47 44 – 62 62
6 9 – 17 9 13 – 23 13 20 – 35 35 26 – 46 46
7 9 – 17 9 13 – 23 13 20 – 35 35 26 – 46 46
8 9 – 17 9 13 – 23 13 20 – 35 35 26 – 46 46
These targets should be viewed as estimates to help ensure that students stay on track, and do not fall further
behind. However, some students who start the year behind their peers will need intensive intervention and would
need to grow more than the suggested one year target to close the gap with their peers at the end of the year. For
these situations, the target could be increased to suggest more aggressive growth aspirations.
Target setting should also consider the amount of instructional time between the first and last diagnostic.
Specifically, the above targets are based on 30 weeks between the first and last assessment. However, if
significantly less time is expected between the first and last assessment, a lower target might be set to take into
account proration of the target over the anticipated number of weeks (i.e., a target might be 80 percent of the 1.0
year target if only 24 weeks are planned between the first and last test).
Only one target can be set by subject and grade in an i-Ready account; however, for the purposes of evaluation, a
teacher could develop a target for each student and then take the average of these desired gains and make that
his or her overall target. This target could be tracked outside of the system and translated into a teacher-
effectiveness metric. For example, the following class is a class of 10 fourth-grade mathematics students. The one
year target is 22 scale points:
Name Original Target Multiplier 1 Multiplier 2 Final Target
Anna 22 1 .83 (25/30 weeks) 18.26
Beatrix 22 1.1 .83 20.086
Connor 22 1.1 .83 20.086
DeAndre 22 1 .83 18.26
Elaine 22 1 .83 18.26
Frederick 22 1.1 .83 20.086
George 22 1.1 .83 20.086
Hector 22 1 .83 18.26
Isabella 22 1.1 .83 20.086
Juanita 22 1.1 .83 20.086
In this case, the administrator decided that the target should be reduced to 83 percent of the target, because the
time between the first and last assessments was significantly less than the requisite 30 weeks. Also, in the case of
six students—Beatrix, Connor, Frederick, George, Isabella, and Juanita—the teacher wanted to set a more
aggressive growth target (110 percent of the 1.0 years), because those children are starting the year well below
grade level.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 49 of 72
Adding the numbers in the Final Target column and dividing by the number of students gives the teacher’s average
growth target of 19.3556, which rounds to 19. This target could then be used to determine a teacher rating on the
scale: Highly Effective, Effective, Developing, or Ineffective. The target should be multiplied by .75, 1, and 1.5
respectively, to convert to the rating scale and determine the cut points for the HEDI categories. In this example,
the following scores would result for evaluation:
Highly Effective—29 or higher
Effective—19 to 28
Developing—15 to 18
Ineffective—Under 15*
* To determine how these numbers can be changed into a 20-point scale for HEDI, please refer to the crosswalk
below.
In the end, while the i-Ready system may not capture the personalized ratings for each teacher, the system will
determine the final adjusted average gains for each grade or class, and these gain scores can be used to inform the
teacher’s accountability rating by evaluating against the rating scale above.
Curriculum Associates recommends aggregating scores in the following manner to determine an adjusted average
score for each teacher’s students. First, find the gain for each student by taking the difference in scale scores and
subtracting the student’s first assessment from their last assessment. Then, for students who show score reversals
or negative gains, set these scores equal to zero (Curriculum Associates defaults to considering these situations as
having no evidence of a gain in ability level, and therefore they are counted as zero). Next, determine the
arithmetic mean of all of these values, including the zeroes and positive gains. Finally, take this adjusted average
and divide by the target score. This will provide a percentage gain which can be translated into a HEDI metric and
a score on a 20-point scale.
We created a crosswalk (figure below) that maps scores from the aggregated teacher-level growth score to the
teacher and principal evaluation 20-point metric by taking the target goals and multiplying by the percentages as
shown in the chart below. The blacked-out marks show where the aggregated score may be repeated from rating
to rating. In these cases, we recommend going with the higher rating for the principal and teacher evaluation. The
current scores for Effective reflect the default targets. However, this chart can be recreated with customized
targets in the following manner. First, replace the values in the 15-point column with the new targets; then
multiply the percentage at the top of each column by the new target to get the new minimum value for each cell.
For the column worth 13 points, we recommend multiplying by 75 percent and then rounding down.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 50 of 72
Evidence of Fairness of Proposed Aggregated Teacher Growth Scores
Prior academic history can account for a significant difference in growth rates. In this case, prior academic history
is estimated using the first placement score on the Fall administration of i-Ready Diagnostic. There are four
placement groups to consider:
Group 1: Students who place two or more levels below their chronological grade
Group 2: Students who place one level below their chronological grade
Group 3: Students who place Early on level in their chronological grade
Group 4: Students who place Mid, Late, or Above their chronological grade
These different placements can be added as a multiplier to the methodology shared above. Curriculum Associates
completed analyses on the variations in the score compared with all of the students, and we recommend the
following multipliers for LEAs that want to account for these differences:
Multipliers for Prior Academic History, Mathematics and ELA
Subject Grade Band 2/2+ Below 1 Below Early Mid
Math
K-5 110% 100% 90% 70%
6-8 120% 80% 80% 80%
ELA
K-5 130% 100% 80% 70%
6-8 130% 80% 80% 80%
New York State Next Generation Assessment Priorities Please provide detail on how the proposed supplemental assessment l or assessment to be used with SLOs addresses each of the Next Generation Assessment Priorities below.
Characteristics of Good ELA and Math Assessments (only applicable to ELA and math assessments):
The adaptive i-Ready Diagnostic leverages advanced technology to
provide a deep, customized evaluation of every student and to track
student growth consistently and continuously over the child’s entire
K–12 career. i-Ready also provides valid and reliable growth metrics
across a district and school environment to optimize administrative
decision-making for long-term performance improvements.
Educators frequently choose adaptive assessments for the
instruments’ high precision and efficiency, allowing them to pinpoint
student needs more accurately and in less time than with traditional
fixed-form assessments. By dynamically selecting test items based on
student response patterns, i-Ready’s adaptive assessment is able to
derive large amounts of information from a limited number of test
items and can adapt to students with low and high abilities to obtain
a more precise measurement of student performance.
For administrators, an adaptive assessment has proven to be the
most precise measure of student growth (Growth, Precision, and
CAT: An Examination of Gain Score Conditional SEM by Tony D.
Thompson, Research Report, December 2008). This real-time
visibility enables immediate, effective course corrections.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 51 of 72
Administrators using i-Ready receive real-time, comprehensive
insight into:
Percent of students performing below, on, and above grade
level
Percent of students on track to meet annual growth
expectations
Details by school, grade, class, and student
i-Ready for Reading/ELA
Foundational Skills. i-Ready Diagnostic assesses the foundational
skills of phonological awareness, phonics, and high-frequency words:
Phonological Awareness. In i-Ready Diagnostic, test items
use both audio and visual support to assess children’s ability
to distinguish and manipulate the sounds in spoken
language. The stems, which comprise questions or
directions, are read aloud to children, as are the individual
answer choices. Children can use an audio icon to hear
items and answer choices repeated. Many items are
supported by art. Most items focus on segmenting and
blending, because these skills are the most important
building blocks for phonics instruction. Children are asked to
segment and blend syllables, onset and rime, and individual
phonemes. Other items assess children’s ability to
manipulate phonemes by deleting, adding, or substituting
sounds in spoken words.
Phonics. i-Ready Diagnostic assesses children’s ability to
recognize sound-spelling correspondences. Test items use
both audio and visual support. Some items—which
comprise questions or directions—are read aloud, and
children are asked to choose among written answer choices.
Other items are written, and children are asked to choose
among answer choices that are read aloud. As with
phonological awareness, children can use an audio icon to
hear items and answer choices repeated. Many items are
supported by art. Items focus on a range of high-utility skills,
including: letter recognition; one-to-one letter-sound
correspondences; CVC and CCVC words—as well as other
one-syllable words; consonant digraphs; final e conventions;
r-controlled vowels; inflectional endings; vowel teams
(digraphs and diphthongs); two-syllable words; three-, four-,
and five-syllable words; and words with prefixes/suffixes.
High-Frequency Words. Words assessed and taught in
i-Ready Diagnostic & Instruction are drawn from the Dolch
Basic Word List (Dolch, 1941) and the Fry Instant Word List
(Fry, 1999). Test items in i-Ready Diagnostic assess
children’s ability to recognize high-frequency words. Some
stems—which comprise questions or directions—are read
aloud, and children are asked to choose among written
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 52 of 72
answer choices. Other stems are written, and children are
asked to choose among answer choices that are read aloud.
Children can use an audio icon to hear items and answer
choices repeated.
Vocabulary. Test items in i-Ready Diagnostic assess students’
knowledge of both Tier 2 words (academic or literary words) and Tier
3 words (domain-specific or content-area words). Panels of teachers
and reading specialists selected the words to be assessed, using
research-based lists that included:
Words Worth Teaching (Biemiller, 2010)
The Living Word Vocabulary (Dale & O’Rourke, 1981)
The Educator’s Word Frequency Guide (Zeno, 1995)
The Academic Word List (Coxhead, 2000)
The panels made these selections to reflect the types of words
children learn in various disciplines at different grade levels and in
various stages of their lives. Test items assess knowledge of these
words in context, and those aimed at early readers include visual
support. Because oral vocabulary is a critical part of reading
development, test items at Kindergarten through grade 2 are
supported by audio.
Comprehension. Students’ abilities to understand both literary text
and informational text are evaluated in i-Ready Diagnostic. The focus
in Kindergarten is on listening comprehension. At this grade,
comprehension items are supported by both audio and art. Reading
comprehension is the focus at grade 1 and above. Students are
presented with a passage, and interactive, multiple-choice items are
shown next to the passage. When a passage has more than one
page, students may page back and forth through it while still viewing
the item. This format and process encourages students to find
textual support for their selected answer.
i-Ready for Mathematics
The Common Core State Standards organize mathematical content
within grades by domains—big ideas that connect topics across
grades. A major goal of this grouping is to build understanding of
mathematical concepts within each domain and how they progress
across grades.
i-Ready Diagnostic further organizes the Common Core Domains into
four major groups: Number and Operations, Algebra and Algebraic
Thinking, Measurement and Data, and Geometry.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 53 of 72
Number and Operations. In i-Ready Diagnostic, the items aligned to
the Number and Operations in grades K–2 allow students to
demonstrate proficiency in the skills associated with counting, whole
numbers, the algorithms of the operations, and understanding of
place value.
In these grades, in the least difficult items, virtual manipulatives are
used to help students show conceptual understanding of place value
and the algorithms for adding and subtracting. For example, students
can utilize a virtual base-ten block tool to help with regrouping for
solving subtraction items.
In grades 3–5, the items aligned to the Number and Operations
domain allow students to demonstrate a deeper understanding of
the concepts they learned in the primary grades, while also
demonstrating their understanding of how these concepts expand
into other sets of numbers, such as fractions and decimals. In this
domain, there are technology-enhanced items where students are
able to show conceptual understanding of fractions by plotting the
fractions on a number line tool.
In grades 6–8, the items aligned to the Number and Operations
domain allow students to demonstrate their understanding of how
the concepts they learned earlier in this domain extend to integers
and real numbers. They also demonstrate their facility with
converting among different representations of numbers.
Algebra and Algebraic Thinking. In i-Ready Diagnostic, the grades K–
2 items aligned to Algebra and Algebraic Thinking allow students to
demonstrate their ability to represent problem situations with
number sentences. As in Number and Operations, in these earlier
grades, students use virtual manipulatives to represent these
problem situations. For example, a 10-frame with counters can be
used to represent what is meant by the equation 5 + 2 = 7, and how
that may be manipulated to show understanding that 7 – 2 = 5.
In grades 3–5, the items aligned to Algebra and Algebraic Thinking
expand to include students’ capabilities of modeling problems using
equations. These items allow students to demonstrate their
understanding by asking them to select the equation that best
models a mathematical or real-world problem.
In grades 6–8, the items aligned to Algebra and Algebraic Thinking
expand on students’ understanding of modeling problems to using
different representations to solve the problems, including
expressions and equations and functions. In these grades, students
may represent situations by graphing a line that represents a
situation on a coordinate graphing tool.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 54 of 72
In grades 9-12, the i-Ready Algebra domain expands to include the
high school Common Core domains of Algebra, Functions, and
Numbers and Quantity. In these domains, students extend work with
algebraic relationships to polynomial, exponential, logarithmic, and
other advanced functions; complex number systems, and vectors.
They use equations and inequalities to model real-world and
mathematical situations and to solve non-routine problems.
Measurement and Data. In i-Ready Diagnostic, the items aligned to
Measurement and Data allow students to observe, collect, display,
organize, and interpret measures and data. In grades K–2, the items
focus on measuring using virtual tools such as a ruler, and
interpreting data displayed in simple graphs such as picture and bar
graphs.
In grades 3–5, the items aligned to Measurement and Data provide
opportunities for students to demonstrate their extended
understanding of more complex measurements and data sets. The
items aligned to this domain in these grades also emphasize
conceptual understanding of geometric measurement. For example,
there is a tool that allows students to fill a rectangular prism with
unit cubes to demonstrate an understanding of volume.
In grades 6–8, the items no longer have any focus on geometric
measurement, concentrating solely on the concepts of statistics and
probability. Items ensure that students are given the opportunity to
demonstrate their conceptual understanding of more complex data
sets.
Technology-enhanced items allow students to demonstrate their
understanding of bivariate data by graphing linear functions that
closely represent a data set.
Geometry. In i-Ready Diagnostic, the items aligned to Geometry
allow students to demonstrate proficiency in identifying, analyzing,
and reasoning with shapes and figures. In grades K–2, the items are
concentrated on two areas—students are provided the opportunity
to demonstrate proficiency with the attributes of different shapes,
and they are able to show connections to a conceptual
understanding of fractions as part of a whole. Technology-enhanced
items allow students to sort or identify shapes that have similar
attributes.
In grades 3–5, the items aligned to Geometry expand on students’
understanding of figures and begin to assess student understanding
of the attributes in hierarchies. These items also ask students to
demonstrate a conceptual understanding of two-dimensional figures
in space. Some of the technology-enhanced items have students plot
shapes in the first quadrant of a coordinate grid. Other items may
have them fill in a two-dimensional space with unit squares to help
demonstrate proficiency with a conceptual understanding of area.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 55 of 72
In grades 6–8, there is somewhat of a shift in the domain. In grades
K–5, the only geometric measurement concepts covered in the
Geometry domain are those that deal with conceptual
understanding of area. However, in grades 6–8, with the
Measurement and Data domain focusing on Statistics and
Probability, all of the geometric measurement concepts fall under
the Geometry domain. These include area of composite figures,
surface area, and volume.
In grades 9-12, the Geometry domain expands to include both
Geometry and Statistics and Probability from the high school
Common Core domains. In these domains, students apply and prove
theorems involving lines, angles, and figures to extend their
understanding of geometric properties. They also employ logic and
data to make informed decisions about real world situations.
In addition to these concepts, higher-level geometric concepts are
also assessed in i-Ready in grades 6–8.
These concepts include relating transformations to congruence and
similarity, and analyzing proofs of the Pythagorean Theorem and its
converse. Some of i-Ready Diagnostic’s technology-enhanced items
in this domain at these grade levels use a virtual protractor to allow
students to demonstrate proficiency with rotations.
Assessments Woven Tightly Into the Curriculum:
i-Ready may be administered seamlessly in conjunction with regular
classroom instruction, as the assessment is given entirely online and
the program automatically scores, analyzes, and reports student
results in real-time. As each student works individually and at his or
her own pace on the adaptive test, educators may administer
i-Ready in small groups or to the whole class, for maximum flexibility.
To support the day-to-day academic goals of the teacher, i-Ready’s
comprehensive reports provide explicit next steps for instruction and
point-of-use lesson plan PDFs. Based on each student’s and
instructional group’s identified needs, i-Ready Diagnostic reports
also provide direct connection to optional online lessons via i-Ready
Instruction (cost option) and recommendations for specific lessons in
other Curriculum Associates’ programs (such as Ready®).
In these ways, i-Ready embodies the philosophy that learning is a
continuous cycle of assessment linked to instruction.
Performance Assessment: The i-Ready Diagnostic test bank includes thousands of multiple-
choice and technology-enhanced assessment items, field tested with
more than one million students to ensure they are accurate, valid,
and reliable measures of the intended skills being assessed.
The RFQ defines a performance assessment as one in which students
are required to perform a task, including problem solving.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 56 of 72
i-Ready items emphasize conceptual understanding and procedural
fluency, and many entail word problems/problem solving. For
example, i-Ready Diagnostic contains mathematics items where
students must bisect angles using a virtual compass and straight-
edge or fill-in rectangular prisms with unit cubes to determine
volume. The reading assessment contains items where students
must pull out evidence from passages to support themes, rather
than to just choose them from a limited number as in a selected
response items.
To reflect real-world use of mathematics as well as the Common
Core, students have access to onscreen, interactive tools—including
a calculator, spreadsheet tool, protractor, compass, straight-edge,
and ruler—that may be needed as they answer items.
Efficient Time-Saving Assessments:
i-Ready’s computer-adaptive format maximizes the yield of
actionable data, while optimizing administration efficiency. The
assessment enables educators to pinpoint student needs more
accurately and in less time than with traditional fixed-form
assessments. By dynamically selecting test items based on student
response patterns, i-Ready is able to derive large amounts of
information from a limited number of test items and can adapt to
students with low and high ability to obtain a more precise
measurement of student performance.
When a student fails more difficult items, additional items assessing
less difficult skills are presented to helps to. drive more precise
targeting of instruction.
Students receive 54–72 items per subject, and typically take 30–60
minutes per subject to complete the diagnostic. Testing may be
completed in multiple shorter sessions. Average duration varies by
subject and grade level, with grades K–3 tending toward the shorter
end of the range. Additionally, variability exists in every grade given
different student performance levels.
Technology: i-Ready Diagnostic is a fully web-based, vendor-hosted, Software-as-
a-Service application. This offers numerous benefits to the Board of
Regents, NYSED, and New York educators. All program maintenance,
updates, and upgrades are included in the highly cost-effective
license fee, and we push them automatically to all end users for
immediate implementation upon release—with no need for local
installation or support.
Student responses are automatically and immediately scored by the
program’s sophisticated analytics engine, which presents data
reports in real time.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 57 of 72
Authorized users have secure access to the system 24/7 (with the
exception of system maintenance, scheduled during low usage
periods), from any compatible, internet-enabled device. The web-
based platform gives our development team the flexibility to rollout
new features and enhancements multiple times each year, at no
additional cost to active clients.
By virtue of being an online assessment employing computer-
adaptive algorithms and technology-enhanced items, i-Ready
Diagnostic helps to prepare and familiarize students with needed
21st-Century skills.
Degree to which the growth model must differentiate across New York State’s four levels of teacher effectiveness (only applicable to supplemental assessments):
Our proposed growth model differentiates educators across the
State’s four levels of teacher effectiveness—Highly Effective,
Effective, Developing, and Ineffective—very similarly to the
distribution for New York’s 2013-2014 distribution of teacher
effectiveness scores. For all schools in i-Ready’s New York K–8
population for 2014–2015, these categories are distributed as
follows:
Subject H E D I
ELA 5% 62% 20% 13%
Math 3% 62% 20% 15%
These numbers are similar to the 2013–2014 growth ratings* for
grades 4–8 for New York schools, shown here:
H E D I
School 7% 76% 10% 7%
*Source: https://www.engageny.org/resource/technical-report-
growth-measures-2013-14.
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 63 of 72
not
STUDENT ASSESSMENTS FOR TEACHER AND PRINCIPAL EVALUATION
ATTESTATION OF TECHNICAL CRITERIA – SUPPLEMENTAL ASSESSMENTS
WITH CORRESPONDING GROWTH MODELS
Please read each of the items below and check the corresponding box to ensure the fulfillment of the technical criteria outlined in the Technical Application on “FORM B-2”. PLEASE SUBMIT ONE “FORM G” FOR EACH APPLICANT. CO-APPLICANTS SHOULD SUBMIT SEPARATE FORMS. COMPLETE THIS SECTION:
2.2(A) Narrative Overview of Proposed Supplemental Assessment and Associated Growth Model
This application contains a short overview of the assessment being proposed, including the intended purpose of the assessment, and how the assessment is administered. For supplemental assessments, this application contains a description of the growth model and how it is used in conjunction with the assessment. For K–2 assessments, this application contains evidence that the proposed assessment is consistent with this RFQ’s requirement that the assessment not be a “Traditional Standardized Assessment” as defined above in the section “Definitions of Key Terms Used in this RFQ.”
N/A N/A
2.2(B) Evidence of Capability
This application provides an overview of services provided by the Assessment Provider, including a description of the range of support / technical assistance that the Assessment Provider would provide to an LEA if selected by an LEA for this service. This application contains information as to whether the Applicant or Assessment Provider has been denied approval as a provider of assessment services in another state(s) and the reason(s) for such denial. If denied within New York State, the location and reason are indicated.
N/A
2.2(C): Evidence of Copyright Owner/Assessment Representative History of Assessment Development
This application contains evidence that the Copyright Owner/Assessment Representative has a history of developing assessments of student learning (achievement or growth) for the purpose of making defensible judgments about educator effectiveness.
N/A
FORM G
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 64 of 72
2.2(D)-i: Technical Documentation Related to Assessment and Student Growth Score Properties: RELIABILITY Both “minimum” and “desired” qualifications are listed. For the purposes of this RFQ, applications will only be rated against the “minimum” qualifications; however, NYSED’s aspirational “desired” qualifications are also listed to identify possible future requirements for assessments and associated growth models. For supplemental assessments used in conjunction with growth models: This application contains evidence of the minimum criteria for reliability:
Student test scores have adequate levels of reliability (e.g., coefficient alpha > 0.75).
This application contains evidence of the desired criteria for reliability:
Standard errors provided for students growth scores.
Student growth classifications have adequate decision consistency.
Teacher effectiveness classifications demonstrate adequate consistency.
Examples include agreement statistics (e.g., kappa coefficients) based on simulation studies.
Check all that apply:
2.2(D)-ii: Technical Documentation Related to Assessment and Student Growth Score Properties: VALIDITY – ALIGNMENT Both “minimum” and “desired” qualifications are listed. For the purposes of this RFQ, applications will only be rated against the “minimum” qualifications; however, NYSED’s aspirational “desired” qualifications are also listed to identify possible future requirements for assessments and associated growth models.
For supplemental assessments used in conjunction with growth models: This application contains evidence of the minimum criteria for alignment validity:
Evidence that test content is sufficiently aligned with New York State Learning Standards and covers a range of measurable standards. Documentation that demonstrates that:
(a) at least 80% of the test measures content aligned with NYS learning standards,
(b) no more than 20% of test content is aligned with other learning standards or objectives, and
(c) a range of content from the NYS learning standards is measured Note: Other relevant standards can be proposed if NYS Learning Standards do not apply to subject area.
This application contains evidence of the desired criteria for alignment validity:
100% alignment between NYS Learning Standards and assessment.
Check all that apply:
2.2(D)-iii: Technical Documentation Related to Assessment and Student Growth Score Properties: VALIDITY – RELATIONS TO OTHER VARIABLES Both “minimum” and “desired” qualifications are listed. For the purposes of this RFQ, applications will only be rated against the “minimum” qualifications; however, NYSED’s aspirational “desired” qualifications are also listed to identify possible future requirements for assessments and associated growth models.
For supplemental assessments used in conjunction with growth models: This application contains evidence of the minimum criteria for validity in relation to other variables:
Evidence students’ growth scores are correlated with other measures of student progress (e.g., r > .5 with measures such as the number of objectives mastered by a student over the course of the year, teachers’ ratings of
Check all that apply:
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 65 of 72
students’ progress, or scores from other assessments).
This application contains evidence of the desired criteria for validity in relation to other variables:
Evidence teacher effectiveness ratings are positively correlated (e.g., r > .5) with other measures of teaching effectiveness.
2.2(D)-iv: Technical Documentation Related to Assessment and Student Growth Score Properties: VALIDITY – INTERNAL STRUCTURE Both “minimum” and “desired” qualifications are listed. For the purposes of this RFQ, applications will only be rated against the “minimum” qualifications; however, NYSED’s aspirational “desired” qualifications are also listed to identify possible future requirements for assessments and associated growth models.
For supplemental assessments used in conjunction with growth models: This application contains evidence of the minimum criteria for validity of internal structure:
Scale properties appropriate for growth model used (*see notes*). Total scores and subscores on student assessments should be supported by dimensionality analyses (e.g., IRT residual analyses, factor analyses).
This application contains evidence of the desired criteria for validity of internal structure:
Evidence students' scores are on an interval scale.
*Notes: If gain score model is used, evidence is needed that students' pretest and posttest scores are on the same scale. If student growth percentile model used, justification for the number of years included in the model should be provided. If growth-to-proficiency, projection, or value-added models are used, evidence is needed that the model explains a significant amount of variability in student achievement. Also, models should demonstrate robustness to missing data.
Check all that apply:
2.2(D)-v: Technical Documentation Related to Assessment and Student Growth Score Properties: UTILITY AND COMPREHENSIBILITY Both “minimum” and “desired” qualifications are listed. For the purposes of this RFQ, applications will only be rated against the “minimum” qualifications; however, NYSED’s aspirational “desired” qualifications are also listed to identify possible future requirements for assessments and associated growth models.
For supplemental assessments used in conjunction with growth models: This application contains evidence of the minimum criteria for utility and comprehensibility:
Technical documentation that describes how student growth and educator effectiveness are calculated.
This application contains evidence of the desired criteria for utility and comprehensibility:
Student growth reports support instructional improvement. Resources and supporting materials available to the field.
Check all that apply:
2.2(E)-i: Technical Documentation Related to Aggregating Student-Level Growth Scores to Teacher-Level Scores: CREATION OF TEACHER LEVEL SCORES
For supplemental assessments used in conjunction with growth models: This application includes a narrative description of how student-level scores are aggregated to create a single teacher-level score for each teacher.
N/A
NYSED RFQ: Teacher and Principal Evaluation Service Provider – Assessments (App Period: 2015-16)
Page 66 of 72
2.2(E)-ii: Technical Documentation Related to Aggregating Student-Level Growth Scores to Teacher-Level Scores: EXCLUSION RULES
This application includes a description of any exclusion rules that remove students associated with a given teacher from the teacher’s teacher-level score (either through a growth model or in conjunction with an SLO).
N/A
2.2(F): Technical Documentation Related to Converting Teacher-Level Growth Score to New York State’s 0-20 APPR Scale
This application includes a crosswalk that maps scores on the assessment’s aggregated teacher-level growth score to the required New York State teacher and principal evaluation metric, which ranges from 0-20. This application includes procedures for converting teacher-level growth scores to the 0-20 APPR scale comply with the New York Standards for each evaluation rating category, which are based on the following definitions. For supplemental assessments used in conjunction with growth models: This application includes an explanation of the assignment of HEDI rating categories based on the following ranges:
Highly Effective: results are well-above State average* for similar students
Effective: results meet State average* for similar students
Developing: results are below State average* for similar students
Ineffective: Results are well-below State average* for similar students
N/A
2.2(G)-i: Technical Documentation Related to Fairness: TEST TAKERS Consistent with the new Testing Standards (2014), there is an increased focus in the industry on fairness of assessments and their uses. Please provide evidence of fairness for both the proposed assessment and, if applicable, the proposed growth model.
This application includes evidence that the proposed assessments are fair to all test takers (e.g., Differential Item Functioning [DIF] / bias information, fairness evaluation / sensitivity review plan.)
2.2(G)-ii: Technical Documentation Related to Fairness: TEACHER GROWTH SCORES
This application includes evidence of fairness of the proposed aggregated teacher growth scores (e.g., lack of correlation between aggregated teacher growth scores and student demographics). The evidence of fairness of the proposed aggregated teacher growth scores includes an explanation of how the growth model incorporates (a) prior academic history, (b) poverty, (c) students with disabilities, and (d) English language learners.
N/A