Replication Challenge

Replication Challenge is an exciting opportunity to get in touch with researchers from prominent research institutions and work together on a replication study of a selected work.

The activity is designated for early stage researchers (PhD students, research-oriented master students or other research enthusiasts) and focuses on topics falling within the scope of the DisAI project, namely:

• multilingual language technologies
• multimodal natural language processing
• trustworthy artificial intelligence

while preferably covering (but not limited to) the domain of:

• disinformation combating 

Each involved participant will be assigned a mentor from the leading partners. Together they will select a scientific paper (related to project topics) and replicate the research described therein. This way, the early-stage researchers will get better acquainted with the state-of-the-art in their field of study and gain practical research experience. The involved participants will have 3 months to replicate the selected research work. 

Each researcher will prepare a final report on the achieved results and submit it according to the instructions below (please scroll down for more info about the submission process).

The activity will culminate with a 1-day workshop, where the participants will present the results of their work. In the replication study, the participants and their mentors are encouraged to extend the replicated work with some novel interesting ideas, so their results can be published to the scientific community as well.

To participate in the replication study, please apply according to the instructions below. Approximately 10 early stage researchers will be selected after balancing the interest and mentors capacity.

Mentors and Research Works to Replicate

Note that the list of papers for replication is not exhaustive. You can propose your own paper when applying for the Replication Challenge, which can be considered by the mentors (UPDATED).

• Dr. Simon Ostermann
  Lab Manager, Senior Researcher, Group Lead “Data and Resources”, DFKI:
  • ATTEMPT: Parameter-Efficient Multi-task Tuning via Attentional Mixtures of Soft Prompts 
  • Parameter-Efficient Transfer Learning for NLP
  • SPoT: Better Frozen Model Adaptation through Soft Prompt Transfer
  • The Power of Scale for Parameter-Efficient Prompt Tuning
  • other
• Dr. Stefanos Papadopoulos
  AI Researcher at Information Technologies Institute, CERTH:
  • BCMF: A bidirectional cross-modal fusion model for fake news detection
  • Self-Supervised Distilled Learning for Multi-modal Misinformation Identification
  • Findings of Factify 2: Multimodal Fake News Detection – Reproduce one or more of the top methods from the FACTIFY2 challenge 
  • INO at Factify 2: Structure Coherence based Multi-Modal Fact Verification
  • Team Triple-Check at Factify 2: Parameter-Efficient Large Foundation Models with Feature Representations for Multi-Modal Fact Verification
  • SpotFake: A Multi-modal Framework for Fake News Detection
  • End-to-End Multimodal Fact-Checking and Explanation Generation: A Challenging Dataset and Models
  • other
• Dr. Matúš Pikuliak
  Senior researcher, NLP Team, KInIT:
  • Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models
  • CONCRETE: Improving Cross-lingual Fact-checking with Cross-lingual Retrieval
  • other
• Assoc. prof. Michal Gregor
  Expert researcher, NLP Team, KInIT:
  • Scaling sentence embeddings with large language models
  • MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks

Application

Fill in this form and indicate your interests. We will contact you with the decision, eventually containing information about mentor and research work assignment.

Results submission

After finalising the replication study, participants are obliged to submit a report summarising the achieved results.  

We will provide more information about the results submission process soon.

Schedule

• October 4th – Call for Participation
• Run I
  • October 4th – October 20th – Application and ESR-Mentor Matching
  • November 2nd – January 31th – Replication Study Realisation
  • January 31th – Report submission
  • February/March 2024 (to be specified) – Review, Camera-ready preparation
• Run II
  • December 11th – April XXth (exact day TBS) – Replication Study Realisation
  • April XXth 2024 (to be specified) – Review, Camera-ready preparation
• April 2024 (to be specified) – Workshop in Bratislava

Workshop

The participants of the replication challenge will present their findings at the 1-day workshop that will be organised in March or early April 2024 in Bratislava as an hybrid event.

More details about workshop will be provided later.