Skip to main content

Direct Assessment

Direct assessment involves evaluation of student artifacts in CREW designated courses to determine the extent to which the student learning outcomes were met. CREW courses are designated within the three tiers of general education at NSU and also across the disciplines.  Table 7 displays the curricular assessment plan for the CREW quality enhancement plan.  The direct approach for assessing student learning will use the AAC&U Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics for reading and written communication. 

Table 7.  Curricular Assessment Plan for CREW

 

Curriculum

 

Course Levels1

 

Assessment Method

 

Expected Competency

Levels2

 

Expected Performance Rate3

 

 

General Education

 

Tier 1 Courses

 

 

Direct

 

2

 

50%

 

Tier 2 Courses

 

 

Direct

 

2

 

60%

 

Tier 3 Courses

 

 

Direct

 

3

 

70%

 

Discipline

Research Seminars & Capstones

 

Direct

 

3/4

 

85%

 

  1. NSU’s general education courses are grouped into three tiers. Tier 1 courses include 18 credit hours of mostly 100-level required courses that provide students with basic learning tools.  Tier 2 courses include 100-300 levels of courses that provide students with opportunities to learn how to effectively appraise information.  Tier 3 courses provide students with capstone experiences to develop foundations for intellectual empathy. Disciplinary courses provide students with opportunities to read and write across the disciplines in the form of research seminars, term papers, or capstones.   
  2. Student artifacts related to CREW will be assessed by using the Association of American Colleges and Universities (AAC&U) Valid Assessment of Learning in Undergraduate Education (VALUE) rubrics for reading and written communication.  The respective rubric has four levels of competency: (1) Benchmark; (2-3) Milestones; and (4) Capstone.
  3. It is expected that 50 percent or more students completing CREW courses in Tier 1 will reach milestone competency level 2 or better at the end of the courses. At least, 60 percent of Tier 2 completers will reach milestone competency level 2 or above after completing the courses. Similarly, at least 70 percent of Tier 3 course completers are expected to meet milestone competency level 3 or better.  Within the discipline, the expectation is that, at least, 85 percent of students completing CREW designated courses will be testing at competency level 3 or 4 of the AAC&U VALUE rubrics for reading and written communication.

Table 8.  Assessment Implementation Plan for CREW

 

Academic Year

 

 

Course Levels

Number of CREW Courses

Number of Students

 

Pilot Year:  2018-19

 

 

Tier 1

 

  4

 

   150

 

Year 1:  2019-20

 

 

Tiers 1 & 2

 

  6

 

1,000

 

Year 2:  2020-21

 

 

Tiers 1,2 & 3

 

  9

 

1,500

 

Year 3: 2021-22

 

 

Tiers 1,2 & 3

 

12

 

2,000

 

Year 4:  2022-23

 

 

Tiers 1,2 & 3 + Discipline

 

15

 

2,500

 

Year 5: 2023-24

 

 

Tiers 1,2 & 3 + Discipline

 

18

 

3,000

 

As indicated in Table 8, five class sections in four courses (ENG 101, 102; HIS 101; and SOC 101) will be piloted and assessed in spring 2019 semester to include the two learning outcomes for CREW.

In Year 1, the scope of this QEP will expand to six CREW-designated courses in Tier 1 and Tier 2 and will be assessed by faculty teaching the courses.  Faculty development will be provided on the use of AAC&U VALUE rubrics for reading and written communication.  In the first year, approximately 1,000 students will be assessed with the expectation that, at least, 50 percent and 60 percent of Tier 1 and Tier 2 completers, respectively, will attain milestone competency level 2 or better after completing the courses. 

Year 2 assessment will cover the three tiers of general education courses that are designated as CREW courses.  The scope of this QEP will expand to nine CREW-designated courses in Tier 1, Tier 2, and Tier 3 and will be assessed by faculty teaching those courses.  Faculty development will be provided on the use of AAC&U VALUE rubrics for reading and written communication.  In the second year, approximately 1,500 students will be assessed with the expectation that, at least, 60 percent and 70 percent of Tier 2 and Tier 3 completers, respectively, will attain milestone competency level 2 and level 3 after completing the courses.

In Year 3, the number of CREW courses will be expanded to 12 courses with a number of class sections reaching, at least, 2,000 students within the three general education levels.  Year 3 assessment strategies will be similar to year 2 strategies.

In Year 4, the scope of this QEP will expand to 15 CREW-designated courses within the general education tiers and across various disciplines.  At this time, it is anticipated that no fewer than 2,500 students would benefit from the CREW implementation.  The competency level and expected performance rate of students in each category of courses assessed are as shown in Table 1 above.  It should be noted that not less than 85 percent of students assessed at the disciplinary level are expected to reach competency level 3/or 4, using the AAC&U VALUE rubrics for reading and written communication.

 

Year 5, the last year of the QEP implementation, will embrace 18 courses distributed among the general education courses and various disciplinary courses.  The assessment approach will be similar to Year 4 strategies.

Value-Added Approach

The impact of the CREW quality enhancement plan will be measured through a value-added approach.  Baseline data will be collected in the Pilot Year and Year 1.  Through longitudinal tracking of students at various levels of course completions from Year 1 to Year 5, performance growth will be measured insofar as students’ reading and written competencies are concerned.

Table 9.  Value-Added Analysis for the CREW Learning Outcomes

 

Course

Levels

 

Expected Competency Levels

 

 

Academic Year

Level 2

Level 3

Level 3 or 4

 

Tiers 1 & 2

 

 

X

 

X

 

X

 

2019-20

 

Tiers 1,2 & 3

 

 

 

X

 

X

 

2020-21

 

Tiers 1,2 & 3

 

 

 

X

 

X

 

2021-22

 

Tiers 1,2 & 3 + Discipline

 

 

 

 

X

 

2022-23

 

Tiers 1,2 & 3 + Discipline

 

 

 

 

X

 

2023-24

 

As shown in Table 9, participating students’ knowledge and skills in the two CREW learning outcomes will be incrementally measured.  Completers of Tiers 1 and 2 CREW courses will be assessed to determine the percentage of students meeting the VALUE rubrics competency levels 2 and above during academic year 2019-20.  The 2019-20 assessment will provide the baseline data for comparisons in the subsequent years.  In both academic years 2020-21 and 2021-22, students who have completed the three tiers of CREW general education courses will be assessed to estimate the percentage of students testing at levels 3 and above of the VALUE rubrics in reading and written communications. In academic years 2022-23 and 2023-24, performances of students who have completed the CREW designated courses in general education and their respective disciplines will be assessed for their competency achievement levels.  Comparisons will be made with respect to students who had not completed all levels of the CREW courses.  Expected competency achievement levels (in percentages) are indicated in Table 9, according to the depth of CREW courses completed by students at the time of measuring their skill levels. 

 

Supplementary Direct Assessment

The HEIghten test administered by Educational Testing Service (ETS) will be used to supplement the internal assessment processes as an external validation of the achievement of students’ competencies in reading and writing.

A pilot HEIghten assessment of written communication and critical thinking was conducted in spring 2018 semester to establish baseline performance of NSU freshmen students. The students’ results were compared with the comparison group’s proficiency level on the HEIghten Outcomes Assessment.  The table below presents NSU students’ results by proficiency level.

Table 10. HEIghten Outcomes Assessment Results by Proficiency Level, Spring Semester 2018
Comparison Group Scaled Score NSU Students Scaled Score Proficiency Level Proficiency Level Scale Percent of NSU Students by Proficiency Level
Written Communication
162.9 158.7 Developing 150-160 67%
Proficient 161-171 32%
Advanced 172-180 1%
        Critical Thinking
162.2 156.4 Developing 150-160 81%
Proficient 161-171 19%
Advanced 172-180 0%


The HEIghten results showed that about two-thirds of NSU freshmen tested at the developing level of proficiency.  In subsequent years, students will be tested at different level of studies to determine the value-added gains resulting from the CREW QEP.  The expectation is that, at least, two-thirds of NSU graduating seniors will test at proficient or advanced level in the HEIghten written communication assessment by the end of CREW implementation.

In academic years 2020-21 and 2021-22, the HEIghten test will be administered to a random sample of students who will have completed the CREW courses in general education tiers 1-3.  It is expected that 50 percent or more of the students will be testing at Proficient or Advanced levels.  Again, in academic years 2022-23 and 2023-24, the HEIghten test will be administered to a random sample of students who will have completed the CREW courses in general education tiers 1-3 and disciplinary courses.  At this stage, it is expected that three-quarters (75 percent) or more of the students will test at proficient or higher level. 

Table 11.  Assessment Timeline for CREW
Assessment Academic Year
2018-2019 2019-2020 2020-2021 2021-2022 2022-2023 2023-2024
Fall Spring Fall Spring Fall Spring Fall Spring Fall Spring Fall Spring
Written Communication X X X X X X X X X X X X
Reading X X X X X X X X X X X X