STATE OF HAWAII
BOARD OF EDUCATION
SPECIAL MEETING

MINUTES

Queen Liliuokalani Building
1390 Miller Street, Room 404
Honolulu, Hawaii 96813
Thursday, March 23, 2023

PRESENT:
Bruce Voss, Chairperson
Kaimana Barcarse, Vice Chairperson
Bill Arakaki
Shanty Asher
Lynn Fallin
Ken Kuraya
Makana McClellan
Lauren Moriarty
Kili Namauʻu

EXCUSED:
None

ALSO PRESENT:
Maverick Yoshida, Student Representative
Colonel Wouldiam Palmer, Military Representative
Keith Hayashi, Superintendent, Department of Education
Tammi Oyadomari-Chun, Deputy Superintendent of Strategy, Department of Education
Damian Betebenner, Senior Associate, Center for Assessment
Tony Alpert, Executive Director, Smarter Balanced
Alison Kunishige, Executive Director
Kenyon Tam, Board Analyst
Regina Pascua, Executive Secretary
Lady Garrett, Secretary


I. Call to Order

Board Chairperson Bruce Voss called the Board of Education (“Board”) Special Meeting to order at 11:20 a.m. Board Chairperson Voss, Board Vice Chairperson Kaimana Barcarse, and Board Members Bill Arakaki, Shanty Asher, Lynn Fallin, Ken Kuraya, Makana McClellan, Lauren Moriarty, and Kili Namauʻu were present.


II. Discussion Items

Board Chairperson Voss called for public testimony.

David Miyashiro, HawaiiKidsCAN, testified on agenda item II.A, entitled “Presentation on measuring student achievement: pandemic impacts and recovery.” He referenced HawaiiKidsCAN’s written testimony, which outlined questions for the Board to consider and expressed interest in learning about what similar actions other states are taking to regain momentum and concerns based on Hawaii data.

Cheri Nakamura, HEʻE Coalition, testified on agenda item II.A, entitled “Presentation on measuring student achievement: pandemic impacts and recovery.” She commended the academic gains during the 2021-2022 school year but noted that the data for the last eight years does not show an increase in student achievement or reduction of the achievement gap and expressed disappointment that the Department used resources to hire an outside consultant for an analysis that only concluded that adaptive and resourceful leadership are contributing factors to improvement and did not provide more Hawaii-specific details.

Susan Pcola-Davis, PTSA, testified on agenda item II.A, entitled “Presentation on- measuring student achievement: pandemic impacts and recovery.” She stated that the Department hiring contractors to analyze data may not be the wisest use of funds, that the data do not represent the reality, and that the Department needs a Hawaii-based analysis done by the individuals closest to the system

Board members received written testimony before the meeting. (A listing of the people who submitted written testimony before the meeting is included at the end of these minutes.)

Board Chairperson Voss called on Keith Hayashi, Superintendent, and Tammi Oyadomari-Chun, Deputy Superintendent of Strategy, to provide a presentation on measuring student achievement related to pandemic impacts and recovery.

Chun shared that everyone is concerned about the COVID-19 pandemic’s impact on student learning, which shut down schools three years ago and affected students’ social-emotional and mental health status. She outlined the context for the student achievement data, stating that students have had years of COVID-19 pandemic related learning disruptions, which affected educational learning opportunities and social-emotional learning, and that the system continues to struggle with challenges relating to student attendance. Chun stated that schools are reporting that more students are absent because behaviors have changed since the COVID-19 pandemic. She also explained that the smarter balanced assessment (“SBA”) is the statewide assessment for English language arts (“ELA”) and mathematics for Grades 3-8 and Grade 11, which provides the Department with an indicator of how students are doing academically statewide and how their performance compares with students in other states that use the same assessment. Chun explained that SBA data showed drops then improvements for every grade level, student groups, and in every complex area, with some areas experiencing more severe impacts.

Chun stated that Hawaii’s initial recovery was higher for ELA than mathematics and noted that the data is good although not where the Department wants to be. She stated that the data show increased movement and that the Department would inform the Board about schools, strategic impact, and what would happen in the next year.

Chun introduced Damian Betebenner, Senior Associate, Center for Assessment and Tony Alpert, Executive Director, Smarter Balanced. She stated that the Department engaged national experts to provide an in-depth analysis.

Alpert shared that he is employed by the University of California, Santa Cruz and that the Smarter Balance consortium comprises of 12 states, including Hawaii. He noted the SBA measurement is common across schools and establishes progress, which helps prioritize where resources need to be added. Alpert explained that the prioritizing of resources is different from day-to-day resources for students, which are an understanding of whether students are learning and whether adjustments to instruction are needed. He explained that educators help with the development of the assessment to measure the right content for consistency on what is taught in schools. Alpert noted that Smarter Balanced makes available the largest array of accessibility resources, such as American Sign Language videos and audio support for multiple languages. Alpert stated that Smarter Balanced wants to ensure it is measuring what needs to be measured and one way to make sure that the assessment is accruate is that the summative assessment is an adaptive test to get information from each student.

Alpert explained that data can be used for awareness since a summative assessment lays the foundation for questions and requires an understanding of the larger context and cascading consequences. He shared that comparing current performance to prior performance may not explain what changes need to be made to help students learn. Alpert stated that data are reviewed by individual years, but there is a focus on trend data when making future projections. He shared that students learned at a faster rate after the pandemic and that mathematics would have a slower recovery compared to ELA.

Alpert stated that Chun provided Hawaii SBA results compared to other states in the Smarter Balanced consortium. He noted that there was an odd trend in high school students, since higher performing students opted to take the assessment based on an agreement with the University of Hawaii. Alpert outlined the math results, noted that math performance dropped more than ELA, which follows nationwide pattern, and that one hypothesis is that ELA is more conducive to online learning. He also shared that Hawaii has made substantial progress in earlier grade levels, which is important since there are critical stages of understanding to progress.

Alpert noted that Hawaii participates in the National Assessment of Educational Progress (“NAEP”) assessment, which provides statewide results but does not provide any individual student data. He stated that SBA trends are consistent with NAEP scores, which can help policy-makers evaluate what students can do and helps provide the foundation of good policy-making.

Betebenner provided conceptual foundations of academic impact. He described learning as having velocity with tailwinds (which increases progress) and headwinds (which impede progress), so learning slows and there are drops in attainment. He stated that academic recovery is learning acceleration, or a tailwind where the velocity picks up so students are making up for lost ground due to the headwinds of the COVID-19 pandemic.

Betebenner shared an analogy of a road trip where construction slows you down, so in order to reach your destination at the originally scheduled time, you need to increase your pace. He stated that the data looked at the extent of learning velocity decreases and a scale, which provides the timeframe to allow students to catch up. Betebenner stated that they examined the impact on all grades, student subgroups, and content tested by the SBA. He shared that there were concerns about non-participation that may bias results and referenced a statistical technique called multiple imputation to observe prior achievement results. Betebenner stated that based on the results, there are larger impacts to mathematics than ELA, but generally the impacts were moderate. He stated that the takeaway is that student recovery would be a long-term process. Betebenner described the spectrum of academic recovery, which ranged from deterioration to stabilization, then to recovery and provided a description of each.

Betebenner emphasized that Hawaii has a lot to be proud of and has come through better than before the COVID-19 pandemic to help students catch up. He stated that there is an interest in understanding what schools are doing in large recovery areas.

Betebenner explained the process to find exemplary schools and the placements of these schools fell into the following categories: large impact/large recovery (9 schools), minimal impact/large recovery (4 schools), and large impact/poor recovery (3 schools). He explained that the goal was to get a diverse set of socio-economic and demographic characteristics with the goal of finding replicable strategies that can be used by Hawaii schools to help students recover.

Betebenner provided some of the observations from exemplary schools and noted that the schools with large recovery had strong leadership which drove decision-making throughout the COVID-19 pandemic.

Board Chairperson Voss expressed appreciation for the information and noted that people drive recovery and not just programs.

Board Member McClellan asked for more information in terms of strategies around the online platforms in relation to long-term learning loss. She noted that she feels like her children are out of school more than before the COVID-19 pandemic and expressed interest in understanding what online platforms or tools, if talking about learning loss, have been effective besides in-person learning.

Bettebenner stated that the difficulty is the notion of an online platform and that what is being used can be different at each school.

Board Member Moriarty asked what school activities Bettebenner investigated and what was the theory relating to these activities as it relates to learning loss. She also asked Bettebenner investigated and whether there were other trends relating to the activities definition and if it includes curriculum instructional practices.

Bettebenner replied that his analysis was based on the spectrum of activities developed with principals in 90-minute interviews to describe the different types of remote learning, which could lead to different learning outcomes based on how it was implemented. He stated that terms such as high dosage tutoring may not mean the same thing to practitioners, which could lead to nonsensical results so at this time work is being done to identify vocabulary to describe what state level activities schools participated in.

Board Member Moriarty expressed excitement for the data and noted until this type of data is gathered the Board does not now what is actionable and working. She noted that the Department tried to address four strategies during the COVID-19 pandemic and that going forward the Board needs to know if there has been recovery in these areas.

Board Member Moriarty stated that results for South Dakota, Washington, and Connecticut were better than Hawaii. She asked for more information on what these states did different. Bettebenner asked if by better, whether Board Member Moriarty is referring to higher percent proficiency. Board Member Moriarty confirmed this was correct and asked what Bettebenner’s hypothesis is based on the data. Bettebenner replied that he would not conclude that these other states recovered better than Hawaii based on higher proficiency. He stated that the rates of learning velocity are different for different states, so the data structure does not exist to determine which states recovered better.

Board Chairperson Voss noted that achieving math proficiency by Grade 8 is a desired outcome in the Board’s strategic plan. He noted that all states showed a drop in math for Grades 4 to 6 and asked why there are large and alarming drops in math proficiency in these grades.

Bettebenner shared the differences between ELA and math and noted that ELA is based on cumulative knowledge, which is not discrete by grade level, but students can build on what they know and then fill in the gaps. He contrasted this to mathematics where each year teachers introduce different concepts. Bettebenner added that there are other factors involved, like teacher preparation, adult capacity, student motivation, that impact student performance. He stated that students not performing as well in mathematics as the students go up in grades is an issue across the nation and one that would likely require more investment to end up with different outcomes.

Board Member Asher expressed happiness with the findings, but shared that she cannot help but wonder if the COVID-19 pandemic had a more drastic effect on communities without access to the internet or additional information for English learner students. She asked if Bettebenner did any analysis to determine whether there were other factors that could have contributed to learning loss for students.

Bettebenner stated that the achievement gap is a nationwide issue and that the most disadvantaged students experienced the most learning loss. He stated that it seemed like the trends for improvements were parallel between marginalized and non-marginalized students, but clearly the rate of improvement needs to be greater for the marginalized students because there is still an achievement gap between these student groups. Bettebenner said that the Board would need to push for greater investment, focus, and efforts to address this issue.

Alpert stated that the images reflect averages, which masks heterogeneity.

Board Member McClellan and Board Member Kuraya left the meeting at 12:26 p.m.

Alpert stated that after the COVID-19 pandemic the data show that some student groups recovered and others did not, which is referred to as a K-Shape Economic Recovery.

Board Chairperson Voss emphasized that none of the disadvantaged groups show recovery, just stabilization.

Board Member Fallin asked, where does the Board go from here and asked about a methodology for setting targets that are realistic, but include higher expectations. Bettebenner replied that the Board needs to understand that establishing unrealistic targets for the system, would lead to unrealistic learning expectations for students.

Board Member Kuraya returned to the meeting at 12:30 p.m.

Bettebenner explained that it is important to review learning gains and look at the data to understand how students learn.

Board Member Namauʻu asked, as the Department gathers data for exemplary schools, what is the Department going to do with information about what kinds of things are working. She stated that she would like to know who the outliers are and why only a few schools were able to move ahead.

Hayashi stated that the Department has engaged people across the system and is committed to identifying best practices and having educational leaders meet with each other to share ideas to determine how to use those ideas in schools. He stated that the Department is committed to the continuous improvement of the system.


Board Chairperson Voss called for public testimony.

Stephen Schatz, Hawaii P-20 Partnerships for Education, testified on agenda item II.B, entitled “Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update.” He expressed support for alignment between the Department, Hawaii P-20, and higher learning and emphasized that KPI 1.3 mentions student achievement while in school and continuing to degree seeking or workforce training programs.

David Miyashiro, HawaiiKidsCAN, testified on agenda item II.B, entitled “Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update.” He referenced the written testimony provided by HawaiiKidsCAN, asked how would the Department maintain buy-in throughout the plan, noted that legislators and community partners are not mentioned, and asked about the proposed timeline and process to review the Board policies.

Danson Honda, member of the public, testified on agenda item II.B, entitled “Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update.” He urged the Board to implement a financial literacy program and to reevaluate the goal of the education system, whether it is to prepare students for life or college.

Cheri Nakamura, HEʻE Coalition, testified on agenda item II.B, entitled “Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update.” She stated that baseline growth should start from 2014, that English and Hawaiian language proficiency should not be lumped together, requested adding facilities to Goal 2.3 because there is over $2 billion in unexpended funds, and asked what direction the Department is providing to schools.

Susan Pcola-Davis, PTSA, testified on agenda item II.B, entitled “Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update.” She shared an example of key data points, which would require new data collection and noted that an honest evaluation would lead to breakthroughs.

Board Chairperson Voss called on Hayashi and Oyadomari-Chun to provide an update on the second phase of the Hawaii Public Education 2023-2029 Strategic Plan.

Hayashi stated that Chun has engaged with the stakeholders in the complex areas and would provide the update from the Department.

Chun stated that the Department is presenting on stakeholder engagement after the Board asked for broader engagement. She stated that the Department is also asking for feedback on the draft KPIs, strategies, and performance measures. Chun reported that as of this morning, the Department received 1,300 responses to its request for feedback on the draft implementation plan. Chun reported that she would present a summary of the feedback on April 6, 2023.

Chun explained that the draft template of the implementation plan identifies the KPIs that would measure goals annually, identifies targets for improvement, and would be used to track targets overall at the state level. She shared that Hayashi’s memorandum describes the process and that the Department anticipates changes to the draft KPIs, because there are 10 goals and 18 performance indicators which were selected for feedback.

Chun outlined that there are areas which require further work and that discussions have identified concerns, like encouraging teaching to the test and whether it is possible to consider different data to measure student achievement which would help to move the system and behaviors forward.

Board Member McClellan returned to the meeting at 12:56 p.m.

Board Chairperson Voss asked for an explanation of the differences between KPIs and performance measures.

Board Vice Chairperson Barcarse left the meeting at 12:57 p.m.

Chun stated that a strategic plan desired outcome is reading proficiency by Grade 3. She noted that it is important to have KPIs and target setting, but expressed concern that all performance measures need to be approved by the Board with targets for each item. She asked that the Board consider delegating authority to the Department for these performance measures.

Board Vice Chairperson Barcarse returned to the meeting at 1:01 p.m.

Board Member Fallin stated that a desired outcome regarding early learning was added to the strategic plan, Priority 1, because of the financial investment in the Kindergarten Entry Assessment (“KEA”), but the performance measure focuses on being assessed and not proficiency. She shared that she thought that this KEA desired outcome was the Board was making a bold policy statement about investing in students early and asked how the Board would ensure that students get necessary and timely support.

Chun asked if Board Member Fallin is referring to assessments prior to kindergarten or after students enter kindergarten. Board Member Fallin replied that there is no explanation of how performance on the KEA would be monitored. Chun stated that the Board identified the KEA the Department would administer, but it was not chosen to be a KPI because there is no data system available to track this information.

Board Member Fallin stated that if students are assessed at Kindergarten there would be baseline data but there is no way to know what the baseline is and what the students are missing. Chun replied that activities do not rise to the same level as strategies but the Department would report on Goal 1.1 and the strategies that are approved.

Board Member Fallin emphasized that the investment in early learning is an opportunity and that it is the Board’s role to guide this data. She reiterated that the Board has made a policy investment, so there needs to be an understanding of how to keep monitor KEA assessment data.

Board Chairperson Voss stated that what Board Member Fallin raises is critical, that the Board needs to be able to determine the progress being made towards a desired outcome. He asked if this can be addressed by the subject matter committee for key priorities or if the Department can provide initiative reports on what can be done or approved by the Board. Chun stated that the Department is monitoring and tracking more data than what it reports to the Board.

Board Member Arakaki stated that he participated in a principal meeting on Kauai and had the opportunity to share the strategic plan process. He shared his experience in the meeting and the various reactions from administrators. Board Member Arakaki referenced prior strategic plan goals, such as student and staff success, which directly relate to the current version of the strategic plan. He emphasized that schools and complex areas currently collect data, but the schools need to show what they are doing and what is working or not working going forward.

Board Chairperson Voss asked if the Department is using some of the formative assessment data already being collected to measure whether progress is being made towards the desired outcomes. Chun replied that the Department is in discussions on whether to use the formative or summative assessments for reporting to the Board, she noted that schools all need to use the formative assessments but can select assessments on their own. She explained that the Department is not advocating having schools use the same formative assessments.

Board Chairperson Voss asked whether data from schools who used various programs during the COVID-19 pandemic could be used to show progress and provide important data. Chun replied that there are four assessments commonly used across schools, but not all schools use these commonly used assessments, so the Department would need to investigate further if it needs to exclude schools that do not use commonly used assessments or ask all schools to use commonly used assessments so the Department can collect data. Board Chairperson Voss emphasized that the Board would focus on whether progress is happening or not.

Board Member Kili Namauʻu stated that when highly qualified teachers are in a classroom it is important to review how effective these teachers are going to be with students. She stated that the Department can hire effective principals, CAS, and staff based on qualifications and asked, once in these positions is there a way to monitor these employees to ensure effectiveness.

Hayashi stated that (tenured) teachers previously rated as "effective" undergo performance evaluations every five years and principals regularly visit classrooms, do walkthroughs, and observe the teaching and learning. He explained that if there are issues that raise to the level of concern, principals talk to teachers and provide the rationale for the decisions, including thoughts about moving forward. Hayashi shared that principals work with teachers and supervisors of classified employees who also have an annual evaluation cycle.

Board Member Namauʻu expressed appreciation for the evaluation process, but asked if there is a measure that can capture the effectiveness of teachers so that the Board can see this information. Chun stated that the most appropriate metric would be relating to the percent of teachers that have been found to be rated as effective through the Educator Effectiveness Evaluation system and suggested that the Department provide a presentation on this system and the various components to measure this information from a metrics point of view.

Board Member Moriarty stated that the following four changes are needed: (1) using the base year of COVID-19 pandemic data is not acceptable, since this data can could be unreliable; (2) a student who is near reading proficiency should not be considered proficient; (3) an understanding needs to be provided on the effectiveness of teachers who are qualified; and, (4) the three civic engagement opportunities need to be civic.

Board Member Moriarty also asked that the Department add a strategy for math proficiency because what is currently being done has not gotten the proficiency results that the state needs. She also suggested adding a measure to track growth of students and their proficiency or growth with a transition matrix model or something similar to track whether students are progressing.

Board Member Moriarty also stated that there needs to be an understanding on what changes are needed, what strategies would be used to make changes, and what progress is being made. She asked how the Department creates a principal pipeline to ensure effective principals would be able to continue to get the desired outcomes. Board Member Moriarty stated that key administrative systems and initiatives for KPI 17 may not produce integrated statewide improvement results. Board Member Moriarty noted that the Board may need a joint meeting with the Hawaii Teachers Standards Board to understand if the Department is bringing in qualified teachers and how certifications are being maintained.

Chun stated that this is one component of the plan and in the draft on the website there are proposed statewide strategies. She also stated that there would be reporting to the Board and assured that this is a piece to the puzzle that would be further developed.

III. Adjournment

Board Chairperson Voss adjourned the meeting at 1:39 p.m.

List of the people who submitted written testimony before the meeting
NAME
Organization
Agenda Item
David Miyashiro
HawaiiKidsCAN
II.A. Presentation on measuring student achievement: pandemic impacts and recovery; II.B. Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update
Susan Pcola-DavisII.A. Presentation on measuring student achievement: pandemic impacts and recovery; II.B. Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update
Cheri Nakamura
HEʻE Coalition
II.A. Presentation on measuring student achievement: pandemic impacts and recovery; II.B. Update on Hawaii Public Education 2023-2029 Strategic Plan (Phase II), Implementation Plan: Key Performance Indicators and stakeholder engagement update