Overview

The goal of this competition is to develop computer systems capable of automatically evaluating essays to assist teachers in the classroom by enhancing formative feedback strategies, enabling them to focus on specific areas of writing that require improvement among their students. Specifically, the emphasis is on assessing narrative essays written in the Portuguese language by students within the Brazilian basic education system. As such, participants are invited to develop a computational system capable of estimating a grade for an input essay for each specified competency of interest following the established grading rubric.

Dataset and Competition settings

All essays used in this competition were manually digitized and anonymized. Afterward, the essays were analyzed by two human evaluators who assessed different aspects of the essay based on a pre-defined correction rubric. This rubric provides instructive guidance for educators to consider four required competencies:

Formal Register: Appropriate use of the Portuguese language. Aspects such as misspelling words, inadequate use of nominal/verbal agreement and nominal/verbal regency, and inappropriate usage of punctuation symbols are considered.

Thematic Coherence: Adequate understanding of the text production proposal and its development associated with knowledge from different areas, according to the requested proposal, i.e., the plausibility of the text developed concerning the motivating text.

Textual Typology:  Conformity of the text production proposal regarding a Narrative textual typology, articulating ideas, facts, and information in a sequenced and logical way, presenting the constituent elements of this type of textual structure: narrator, place/space, temporal organization, multiple or single characters performing actions.

Textual Cohesion: Correct use of linguistic mechanisms to interconnect text elements, such as words, sentences, and paragraphs.

Each dimension was assessed using integer levels ranging from 1 to 5, with higher levels indicating better text quality and language proficiency and lower levels demonstrating a lack of proficiency.

Important Dates

• Nov. 27, 2023   Release of the training and validation dataset
• Nov. 27, 2023   Competition opens to the participants
• Jan. 20, 2024    Deadline  for enter the competition. 
• Feb. 01, 2024   Paper submission to the PROPOR 2024 Program Committee describing the contest and the obtained results.
• Mar. 14, 2024   Final contest results to be announced at the PROPOR 2024 conference.

For further information, please visit https://sites.google.com/view/propor2024-portuguese-aes