Shared task

Due to many requests, we have extended the task registration and preliminary run submission deadline to August 23rd!

Please check Important dates for deadlines of shared task registration, preliminary run, and formal run.

Background

We have been annually holding the AI werewolf contests under the AI Werewolf project. The AI werewolf contest has two divisions, the protocol division and the natural language division. The protocol division asks participants to implement an AI werewolf player agent that communicates in a middle language called the AI werewolf protocol. The natural language division asks participants to implement an AI werewolf agent that communicates in natural language. We follow the previous configurations of the natural language division in our AIWolfDial2019 shared task.

The Werewolf Game in this Shared Task

Agents of the AIWolfDial2019 shared task will play the werewolf game of five players, including roles of a seer, a werewolf, a possessed, and two villagers. Players do not know other players' roles. All players, other than the werewolf, are humans.

A game consists of a couple of days, continuing until the human team or the werewolf team survives. A werewolf can specify and attack another player in the end of the day; the attached player will be eliminated from the game. All of players are required to vote to another player, and a player voted most will be eliminated from the game. When humans survive, the villager team wins. When a werewolf survives, the werewolf team wins. A possessed is a human but belongs to the werewolf team. A seer can specify another player in the end of the day, then either human or werewolf is notified.

Agent Technical Specifications

Language Requirement

A shared task participant of AIWolfDial2019 is required to implement an AI werewolf agent that communicate either in English or Japanese. Agents of the Japanese language are required to make an English version, at least by using machine translation internally.

AIWolf Agent APIs

A shared task participant of AIWolfDial2019 is required to implement an AI werewolf agent that connects to our AIWolf server via network. Therefore, there is no limitation on the client side hardware/software other than the network I/O specifications.

We recommend to use our sample agent code with runtime environment (java), distributed from our Github repository prepared for this shared task . Please clone the repository to your Eclipse environment and run the org.aiwolf.ui.bin.AutoStarter class with specifying AutoStarter.ini as its program argument (here is the run configuration). This configuration runs the same five random talk agents. You will see "java.lang.ClassNotFoundException: com.sun.java.swing.plaf.windows.WindowsClassicLookAndFeel" but please ignore. We provide our test server to connect and test your agent as same as the preliminary/formal run, you can try connecting by changing the .ini file.

The AIWolf project provides APIs to implement your agent. http://aiwolf.org/en/server describes how to implement your agent in Java. Other libraries include .NET and Python. Please note that these descriptions assume the protocol version, not the natural language version.

Agent Specification

• Day 0 has greetings only.
• The end of Day 0 has an inspection by a seer, and the game starts from Day 1.
• After Day 1, the end of the days have votes by all players and an attack by a werewolf. Vote, attack, inspection are made via specific APIs (network communications).
• A day consists of a couple turns, where all of agents can make a talk for each turn, receiving talks of previous turns.
• An agent should make a talk within a specified periods after a talk request is sent.
• During days, Agents can communicate anything in natural language. A talk should consists of normal letters and punctuations only. An agent returns "Skip" when nothing to talk, returns "Over" if nothing to talk anymore in that day.
• Use Agent[0x] (e.g. Agent[05], x is 1-5) to mention other agents．
• An anchor e.g. ">>Agent[0x]" could be inserted at the beginning of a talk to refer to another agent, to whom your agent with to talk with. That agent is assumed to respond something to your agent by using an anchor.

Resources

• A sample agent (remote connect version) in Java for this shared task
  • Our sample agent Github repository (recommended, you can create your own agent by this package only if your implementation is in Java)
• AIWolf platform (please note that there is a "protocol" version and a "natural language" version, either in local run or remote connect run)
  • .jar library downloads and tutorial http://aiwolf.org/en/server
  • source code repository https://github.com/aiwolf/
• Corpus and logs
  • Previous game logs (in Japanese) https://kanolab.net/aiwolf/
  • Mafiascum https://www.mafiascum.net/ An online Mafia game forum
  • WolfBBS annotated corpus (in Japanese) https://github.com/aiwolf/wolfbbs_annotations
• Open source previous agents (communicates in the Japanese language)
  • Keldic https://github.com/1never/aiwolf_keldic_2018 https://github.com/1never/aiwolf_keldic_gat2017
  • m_cre https://github.com/mcre/aiwolf-4th-nlp

From Registration to Formal Run

Registration

A team should send a mail to aiwolf at kanolab.net (replace at by @) to register the shared task, describing your team name, a contact e-mail address, names and affiliations of your members (please mark a contact person when a team consists of multiple members), communication language (English and/or Japanese) of your agent. There is no fee required to register/participate the shared task. You will be notified our preliminary and formal run server location after registration.

Testing Your Agent System Beforehand

A shared task participant of AIWolfDial2019 is required to implement an AI werewolf agent that connects to our AIWolf server at the specified timing. We will provide an AIWolf server running, where participants can try connecting with dummy agents to check their system behavior. Participants are required to check their systems certainly work before the shared task run.

Preliminary Run

Participants should run your five agents connected to our server, running 50 games. Then submit your game logs to the organisers via e-mail. If there are too many participants to run the formal run, organisers might select formal run teams depending on these logs. In either case, these logs will be used in the final evaluations.

Formal Run

A participant team is required to connect an agent to our server at a specified timing to play games with other participants. If your team cannot remotely attend the formal run, you should prepare your system where the organisers can somehow trigger your remote system to connect our server and run, e.g. by via a web browser.

System Evaluation

Participants should submit a system design description document to the organizers. This document and logs of the games might be used for research purpose and included and published in our overview paper without any further permission. Participants are encouraged to submit a paper to the workshop.

Games will be between the same agents, different agents, and/or human players. Reviewers will perform subjective evaluations on the game logs, using following criteria:

A Natural utterance expressions

B Contextually natural conversation

C Coherent (not contradictory) conversation

D Coherent game actions (vote, attack, divine) with conversation contents

E Diverse utterance expressions, including coherent characterization

Please not that vague utterances, that can be used regardless of contexts, are not always natural in the werewolf game.

System Descriptions

https://kanolab.net/aiwolf/AIWolfDial2019/AIWolfDial2019SharedTaskSystemDescriptions.pdf