# Technical Report

The SEAM was developed to address the need for psychometrically sound social-emotional assessment tools for young children. The SEAM was designed as a curriculum-based assessment measure to assist in the prevention and early identification of social-emotional difficulties and behavior disorders, as well as to build positive partnerships with families and optimize positive caregiver–child interactions in the first years of life.

SEAM benchmarks and items were identified from the literature on social-emotional development of young children raised in mainly Western cultures; certain concepts repeatedly emerged as those that were deemed essential or critically important to the mental health competence of young children (Squires & Bricker, 2007). These benchmarks and items were reviewed and revised in an iterative process based on feedback from family members and experts in infant mental health, early childhood, early intervention/early childhood special education, psychology, and behavior disorders.

Additional data were also collected on Infant, Toddler, and Preschool Intervals through an online research web site (Squires et al., 2012a, 2012b). Research questions included the following:
- What is the item functioning for the Infant and Toddler Intervals?
- What is the reliability of the Infant, Toddler, and Preschool Intervals, including internal consistency, test–retest, and interrater reliability?
- What is the validity of the Infant, Toddler, and Preschool Intervals, specifically content and congruent validity?

The pencil-and-paper data were gathered from caregivers and practitioners in early childhood programs serving typically developing children and children with developmental delays. Online data were gathered from a variety of caregivers around the United States using a research web site. Paper-and-pencil data, including demographic and utility surveys, were completed one of several ways, including individually by caregivers without practitioner assistance, by caregivers during an interview with a practitioner, and by a practitioner with at least 20 hours of weekly contact with a child (for interrater reliability). Practitioners included early childhood classroom teachers and assistants primarily working with infants and toddlers who were typically developing and early interventionists/early childhood special educators working with families and their children who were at risk or eligible for Part C services.

Data were collected in 49 states across the United States and from Canada. The number of completed SEAMs from each state ranged from 1 to 279, with the largest number coming from Oregon. The sample included a total of 2,201 SEAMs; 1,850 were collected online, and 351 were collected from paper-and-pencil versions. Of the sample, 59% of children were male and 41% were female. The children represented in the sample were predominately Caucasian (76.1%). Other ethnicities included multiracial (6.2%), Hispanic/Latino (4.9%), African American (4.7%), Asian (3.7%), American Indian/Alaskan Native (1.1%), Native Hawaiian/Pacific Islander (0.1%), other nonspecified ethnicity (1.4%), and unknown (1.8%). Fifty-eight percent of children were typically developing, whereas 42% of children were identified with a disability or developmental delay.

Data on family income and education level also were collected. The majority of caregivers reported incomes greater than $50,000 (57%), whereas 43% reported incomes below that level. The greatest percentage of participating caregivers (60%) had a bachelor’s or postgraduate/graduate degree, whereas 19% had some college, 17% had a high school diploma or general equivalency diploma, and 4% had not completed high school.

Data analysis techniques included item response theory (IRT) modeling as well as classical test analyses. IRT modeling was used to examine item order and fit statistics while traditional test analyses were employed to complete validity and reliability studies.

### Table 7.1. Item fit statistics

| Age interval | Benchmark | Infit mean MNSQ(SD) | Infit MNSQ range | Outfit mean MNSQ(SD) | Outfit MNSQ range | Item ordering |
| --- | --- | --- | --- | --- | --- | --- |
| Infant | 1.0 | 0.98(0.19) | 0.84-1.29 | 0.95(0.17) | 0.81-1.23 | a,d,b,c |
| 2.0 | 0.97(0.21) | 0.79-1.26 | 0.97(0.24) | 0.71-1.29 | a,b,c |
| 3.0 | 0.99(0.08) | 0.89-1.09 | 0.97(0.05) | 0.89-1.02 | b,a,c |
| 4.0 | 0.99(0.10) | 0.87-1.14 | 0.93(0.14) | 0.78-1.16 | b,c,d,a |
| 5.0 | 1.00(0.28) | 0.74-1.44 | 1.16(0.52) | 0.74-2.01 | b,a,c,d |
| 6.0 | 0.95(0.16) | 0.73-1.17 | 1.14(0.31) | 0.79-1.45 | b,a,c,d |
| 7.0 | 1.00(0.12) | 0.87-1.16 | 0.95(0.11) | 0.84-1.10 | c,a,b |
| 8.0 | 0.98(0.09) | 0.86-1.08 | 1.01(0.14) | 0.82-1.11 | a,c,b |
| 9.0 | 1.00(0.06) | 0.92-1.06 | 0.92(0.04) | 0.87-0.97 | a,b,c |
| 10.0 | 1.00(0.11) | 0.87-1.16 | 0.94(0.15) | 0.73-1.11 | d,a,c,b |
| Toddler | 1.0 | 1.00(0.22) | 0.73-1.23 | 1.00(0.23) | 0.74-1.25 | d,b,a,c |
| 2.0 | 1.02(0.26) | 0.68-1.35 | 2.70(3.12) | 0.69-8.09 | a,b,c,d |
| 3.0 | 0.99(0.13) | 0.84-1.17 | 1.03(0.17) | 0.84-1.25 | a,b,c |
| 4.0 | 0.97(0.12) | 0.85-1.14 | 0.99(0.12) | 0.89-1.16 | a,b,c |
| 5.0 | 0.99(0.05) | 0.92-1.06 | 1.00(0.07) | 0.92-1.08 | b,a,c,e,d |
| 6.0 | 1.00(0.06) | 0.95-1.08 | 0.98(0.04) | 0.95-1.04 | c,b,a |
| 7.0 | 0.98(0.13) | 0.83-1.14 | 0.98(0.16) | 0.79-1.19 | a,c,b |
| 8.0 | 0.99(0.11) | 0.88-1.15 | 0.98(0.12) | 0.85-1.15 | c,d,a,b |
| 9.0 | 0.98(0.05) | 0.94-1.03 | 0.93(0.02) | 0.92-0.95 | a,b |
| 10.0 | 1.00(0.10) | 0.87-1.15 | 0.99(0.12) | 0.85-1.17 | a,c,b,d |
| Preschool | 1.0 | 0.99(0.14) | 0.76-1.17 | 0.92(0.18) | 0.65-1.20 | a,b,c,e,d |
| 2.0 | 0.98(0.14) | 0.85-1.20 | 1.25(0.43) | 0.84-1.88 | a,b,c,d |
| 3.0 | 0.99(0.13) | 0.83-1.20 | 0.97(0.15) | 0.79-1.15 | c,a,b,d |
| 4.0 | 0.98(0.04) | 0.94-1.02 | 0.98(0.04) | 0.93-1.02 | a,b |
| 5.0 | 0.99(0.14) | 0.80-1.18 | 1.00(0.16) | 0.80-1.24 | a,b,d,c |

Item characteristics (e.g., difficulty, sensitivity) are estimated with the person's responses to the set of measurement items, and each person's ability level is estimated based on his or her set of responses and the estimated item characteristics. The results provide evidence of unidimensionality for each benchmark and support the use of the Rasch 1PL PCM as a means to evaluate item functioning.

### QUESTION 1: WHAT IS THE ITEM FUNCTIONING FOR THE INFANT AND TODDLER SEAM™ INTERVALS?
Item Fit Statistics

PENCIL-AND-PAPER AND ONLINE DATA COLLECTION
Before proceeding to data analysis, a differential item functioning (DIF) analysis using a Rasch one parameter logistic (1PL) partial credit model (PCM) for polytomous scoring was completed with the estimation software Winsteps 3.73 in order to examine whether SEAM items appeared to be functioning differently with different administration methods. The results from the DIF analysis indicated that there were only minor differences in item functioning between administration methods. Evidence for significant DIF was demonstrated in 3 out of 35 items (8.6%) in the Infant Interval and 2 out of 35 items (5.7%) in the Toddler Interval. These results suggested that most of the items functioned invariantly and were not affected by extraneous artifacts inherent in the method of completion by the rater.

### QUESTION 2: WHAT IS THE RELIABILITY OF THE INFANT, TODDLER, AND PRESCHOOL SEAM™ INTERVALS, INCLUDING INTERNAL CONSISTENCY, TEST–RETEST, AND INTER-RATER RELIABILITY?
Interrater Reliability
Interrater reliability data for the Infant and Toddler Intervals were collected from teacher dyads working at a high-quality child care center serving primarily children of University of Oregon faculty and staff. Master teachers and assistant teachers from the infant and toddler classrooms participated. Pearson product moment correlation coefficients and intraclass correlations were computed to examine interrater agreement.

| Classroom | n | r | Intraclass correlation |
| --- | --- | --- | --- |
| Infant 1 | 12 | .776** | .564* |
| Toddler 1 | 7 | .668 | .657* |
| Toddler 2 | 7 | .948** | .932** |
| Toddler 3 | 8 | .640 | .324 |

Test–retest reliability data were collected by online caregiver participants. After completing the SEAM via a research web site, caregivers were immediately given the option to complete a second SEAM.

| SEAM™ interval | n | r |
| --- | --- | --- |
| Infant | 43 | .987** |
| Toddler | 42 | .968** |
| Preschool | 49 | .989** |

### QUESTION 3: WHAT IS THE VALIDITY OF THE INFANT, TODDLER, AND PRESCHOOL SEAM™ INTERVALS, SPECIFICALLY CONTENT AND CONGRUENT VALIDITY?
Correlation of Mean SEAM™ Scores with Age
Two analyses were computed using a subset of the data sample that included children who were known to be typically developing in order to calculate mean SEAM scores across 6-month intervals for all age intervals and to calculate correlation of mean SEAM scores with age for the Infant and Toddler Intervals. There was a consistent increase in mean scores across the 6-month age intervals in both the Infant and Toddler Intervals. Correlations of mean scores with age for the Infant (r = .354) and Toddler (r = .391) Intervals were moderate and significant at p < .01. Correlation of mean scores with age for the Preschool Interval (r = .124) was low and not significant.

|  | DECA | ITSEA Compliance | ITSEA Negative Emotion | ITSEA Pro-social | ASQ:SE |
| --- | --- | --- | --- | --- | --- |
| Infant SEAM scores | r | .754** | .628** | -.415* | .651** | -.557** |
| Toddler SEAM scores | r | NA | .564** | -.261** | .652** | -.516** |
| Preschool SEAM scores | r | NA | NA | NA | NA | -.810** |

### QUESTION 4: WHAT IS THE UTILITY OF THE SEAM™ SYSTEM?
The authors collected utility data from 434 caregivers who completed the SEAM. More than 93% of caregivers felt that the SEAM asked appropriate and useful questions. More than 90% of caregivers felt that items were clearly worded.

Researchers also conducted a written utility survey with practitioners from Part C early intervention programs. The majority of practitioners (91%) completed the SEAM with families during home visits. Ninety-two percent of practitioners agreed or strongly agreed that SEAM items were clear and easy to understand.

### FUTURE RESEARCH EFFORTS
The authors have collected objective data in this initial study of the SEAM system to substantiate the SEAM intervals as well as additional utility data gathered through focus groups. The authors used IRT to investigate item functioning and item ordering and conducted classical test analyses in order to perform the validity and reliability studies. Results suggest the SEAM had robust results related to validity, reliability, and utility. Further study is needed, however, with a stratified, randomized national sample to confirm these results.