KenSwQuAD – A Question Answering Dataset for Swahili Low Resource Language
View/ Open
Publication Date
2022Author
Wanjawa, Barack; Wanzare, Lilian ; Indede, Florence ; McOnyango, Owen ; Muchemi, Lawrence ; Ombui, Edward ;
Metadata
Show full item recordAbstract/ Overview
This research developed a Kencorpus Swahili Question Answering Dataset KenSwQuAD from
raw data of Swahili language, which is a low resource language predominantly spoken in
Eastern African and also has speakers in other parts of the world. Question Answering
datasets are important for machine comprehension of natural language processing tasks such
as internet search and dialog systems. However, before such machine learning systems can
perform these tasks, they need training data such as the gold standard Question Answering
(QA) set that is developed in this research. The research engaged annotators to formulate
question answer pairs from Swahili texts that had been collected by the Kencorpus project, a
Kenyan languages corpus that collected data from three Kenyan languages. The total Swahili
data collection had 2,585 texts, out of which we annotated 1,445 story texts with at least 5
QA pairs each, resulting into a final dataset of 7,526 QA pairs. A quality assurance set of 12.5%
of the annotated texts was subjected to re-evaluation by different annotators who confirmed
that the QA pairs were all correctly annotated. A proof of concept on applying the set to
machine learning on the question answering task confirmed that the dataset can be used for
such practical tasks. The research therefore developed KenSwQuAD, a question-answer
dataset for Swahili that is useful to the natural language processing community who need
training and gold standard sets for their machine learning applications. The research also
contributed to the resourcing of the Swahili language which is important for communication
around the globe. Updating this set and providing similar sets for other low resource
languages is an important research area that is worthy of further research.