A Shared Task on Understanding Figurative Language
Tuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, Smaranda Muresan
Rank | Model | Acc@0 |
Acc@50 | Acc@60 | |
---|---|---|---|---|---|
|
T5-3B (resource-rich baseline) |
76.72 | 69.05 | 44.27 | |
Submit to this leaderboard: You can submit your prediction by going to CodaLab.You will have to provide details about your underlying model and the number of parameters. All details on submission file format and a evaluation script will be provided there
We use Accuracy@60 for ranking all methods because it means the explanations are of superior quality.We will also conduct human evaluation of top 5 performing systems. When accounting for top 5 sytems we will take into account both systems with good Accuracy measures but also systems that are competitive but typically use smaller models (in terms of number of parameters)