Scholarly QALD at ISWC 2024
Task for Hybrid Question Answering over Scholarly Knowledge Graphs and Text

UPDATE

The challenge is now live at https://codalab.lisn.upsaclay.fr/competitions/19140. Due to the initial delay in releasing the dataset, we have adjusted the remaining deadlines accordingly.

Task Description

In the previous version of the Scholarly QALD Challenge at ISWC 2023, 7 teams took part in the tasks of entity linking and question answering over the DBLP and ORKG Knowledge Graphs (KG). We releasd two datasets, each catering to one KG, where participants were required to fetch the answer for a given question from the corresponding KG.

This year, we introduce a more challenging task of Hybrid-QA, which will require participants to query multiple sources of knowledge to fetch the right answer for a given question. The domain of the challenge remains scholarly data, however unlike last year, participants will have to find the answers from multiple textual sources (e.g., Wikipedia) and Knowledge Graphs (e.g., DBLP).

By taking part in the challenges sub-tasks, participants can explore and will experience a versatile range of academic and industrial approaches and applica- tions through the multi-faceted workshop format. We will try to bridge the gap between academia and industry to attract junior as well as senior researchers from both worlds leading to a memorable experience. We target to publish system descriptions as proceedings published by it - Information Technology Journal.


In our second iteration at ISWC 2024, we have the following task:

Task: Hybrid-QA — Hybrid Question Answering over multiple Knowledge Graphs and Text Sources: For a given question, participants have to consider multiple scholarly sources for finding the correct answer, e.g., DBLP KG, Wikipedia text, OpenaAlex KG etc.

Evaluation

For the task, we evaluate the participants' approaches using CodaLab. Further details regarding evaluation, like procedure to upload system results on CodaLab, and the evaluation scripts, are available here.

Important Dates (Tentative)

The tentative timeline template for the Scholarly QALD Challenge is as follows. All deadlines are 23:59 AoE (anywhere on earth).
Date Description
2024-03-16 Website & first call for participation ( https://kgqa.github.io/scholarly-QALD-challenge/2024/ )
2024-05-26 Release of dataset
2024-05-26 Codalab is open for submissions
2024-07-31 Codalab stops accepting submissions
2024-08-21 Submission of System Papers
2024-09-18 Notification of Acceptance
11th to 15th November 2024 ISWC Conference, Visit iswc2024.semanticweb.org

People