A Wizard of Oz experiment is a research experiment in which subjects interact with a computer system that subjects believe to be autonomous, but which is actually being operated or partially operated by an unseen human being
As you see on the attached screenshot, the WoZ tool displays a screen full of buttons, where each button corresponds to a character response. A WoZ operator (wizard) pushes a button and triggers the response that can either be a playback of an audio recording, a video clip, or an animation sequence handled Smartbody and rendering engine. The WoZ tool does not do any media playback itself. It relies on an external component to handle the media. From the technical point, each button has an associated vhmsg that is triggered when the button is pushed. For example, it can be a vrExpress message with a video clip file name supplied as a parameter. We have a separate VideoPlayer tool that listens for vrExpress messages, parses the message content, finds the video file with the right name, and plays it on the screen.
The buttons are arranged into rows, where every row has a title. The rows are arranged into screens, where every screen has a title. You can specify buttons that trigger transitions between the screens. The same button can appear in multiple rows and the same row can appear on multiple screens.
TBD
TBD
You can pass WoZ parameters via the query arguments in the URL. Here are some of the arguments:
true or false whether to automatically
generate an extra row for switching between individual screens. The
default value is true.
true or false whether to show the chat
transcript panel. The
default value is true for connector=ADConnector
and false
in other cases.
VHMSG,
ConsoleConnector, and ADConnector
localhost.
DEFAULT_SCOPE.
true or false token
to specify whether
the connection is
secure. The default value is true.
To help an operator to locate relevant buttons WoZ has... TBD
The data for the wizard comes from several tables (as in database tables) and can be created as an Excel or Google Sheet spreadsheet. Each table should be defined as separate sheet in the spreadsheet. The following sheets can be defined. 3 of the sheets are required and 1 is optional.
Multiple WoZ datasets are allowed per a single Excel document. Each dataset must have its own button, row, and screen sheets. The row and button sheets are mandatory.
The name of the sheets are important. For example, "foo.buttons" specifies the button sheet for dataset "foo".
A sheet that uses the plain "buttons", "rows", or "screens" name will be merged with the corresponding sheet from each dataset. For example, "foo.buttons" will be merged with the content of the sheet "buttons".
A sheet with the name "buttons" specifies all the buttons in the interface. Each row corresponds to a single button and the columns map to button's attributes. The first row contains the column headers. The following column (and the corresponding headers) are required:
The following attributes are optional:
The following screenshot shows a regular button definition
The following screenshot shows how transition buttons can be defined in the sheet.
If the "woz.vhmsg" column is omitted, WoZ will send a vrExpress populated with id and tooltip. If columns "woz.sender" and "woz.addressee" are present, WoZ will use those values to fill the appropriate parts of the message. If the columns are omitted WoZ uses values "visitor" and "all". The message content can be specified explicitly or it can be defined as a template using the values in other columns as variables. For example, the default behavior can be reproduced by using
vrExpress $$sender$$ $$addressee$$ $$sender$$$$messageCount$$ <?xml version="1.0" encoding="UTF-8" standalone="no" ?><act><participant id="$$XML(sender)$$" role="actor" /><fml><turn start="take" end="give" /><affect type="neutral" target="addressee"></affect><culture type="neutral"></culture><personality type="neutral"></personality></fml> <bml><speech id="sp1" ref="$$XML(id)$$" type="application/ssml+xml"> $$XML(tooltip)$$ </speech></bml></act>
Here the content of column "woz.sender" will be the second token in the message and so on. Note that the variable name omits "woz."from the column title.
Note that in case when the message contains XML code, you need to make the content of the variables is encoded appropriately when inserted into the message. We introduce XML(•) to indicate those cases. For example, $$XML(sender)$$ will take the content of woz.sender column and encode is as valid XML (e.g., replace "<" with "<") before inserting it into the message.
A button can trigger a dialog box with popup buttons and text fields for
the operator to fill. For example, if the tooltip column contains a token
that looks like ##token##, when the operator clicks on the
button, she will be presented with a dialog box, with the text of the
tooltip where each ##token## is replaced by a text entry
field. After entering information into the field, the user can press the Send
button on the dialog to send the modified text in the vhmsg. In the following
example the tooltip column contains "Let's start. ##input##".
The token expression in the ##token## expression
serves as
the variable name that can be used in message generation. (See the previous
section.) You can define as many variables as you want, the variable names
are case-sensitive. If the variable name matches a column name in the
button sheet, the entered value will overwrite the value in the column.
The ##token## expression results in a text entry field added
to the dialog. If you want to limit the set of possible values, you can
instruct WoZ to generate popup menu instead of the text entry. In this case
use an expression like this: ##token;value1;value2;value3##.
Here the variable with the name "token" can take one of the 3 values:
value1, value2, or value3. And the dialog box will include a popup menu
with 3 items.
If you want the items in the menu be different from the actual values
assigned to the variable, use both the value and the item text separated by
colon ":". For example, ##token;value1:Item
1;value2:Another Item;
value3:And one more item##. The menu will have three items: "Item
1", "Another Item", and "And one more item."
You can specify the buttons for transitions between screens by hand as described in the previous section, but sometimes it would be better to generate them automatically from the content of the screens sheet (see the Screen Sheet section for the details). You can do something like this:
| J | K (woz.id) | L (woz.label) | M (woz.color) | O (woz.transition) | |
|---|---|---|---|---|---|
| 2900 | =ROW() | ||||
| 2901 | =FLOOR((ROW()-J$2900-1)/2,1)+1 | =IF(J2901<=COUNTA(screens!$1:$1), CONCATENATE("transition.",N2901),"") |
=INDEX(screens!$1:$2,2,J2901) | color.tab | =INDEX(screens!$1:$2,1,J2901) |
| 2902 | =FLOOR((ROW()-J$2900-1)/2,1)+1 | =IF(J2902<=COUNTA(screens!$1:$1), CONCATENATE("transition.",N2902,".selected"),"") |
=INDEX(screens!$1:$2,2,J2902) | color.tab.selected | =INDEX(screens!$1:$2,1,J2902) |
Repeat the last two lines for every screen on the screen sheet (copy the last two rows and keep pasting them until the cells in the woz.id column starts showing up empty. Make sure you adjust the column and row references appropriately.
You should get something like this:
A sheet with the name "rows" specifies all the button rows in the interface. Each Excel table row corresponds to a single row in the interface. The first column contains the row ID, the second column contains the row title, the rest of the columns contain the row's buttons, one button ID per cell. The order of the buttons in the WoZ interface will be the same as the order of the buttons in the Excel table. Note that the first row in the table is an actual row in the WoZ interface. It specifies buttons that transition between different screens. Also note the different number of buttons in each row. For example, row 11 contains just one button, while row 3 references at least three buttons.
You can specify the tab row (the row with buttons that transition between screens) by hand as described in the previous section, but sometimes it would be better to generate them automatically from the content of the screens sheet (see the Screen Sheet section for the details). You can do something like this:
| A | B | C | |
|---|---|---|---|
| 230 | =ROW() | ||
| 231 | =IF(ROW()-$B$230<=COUNTA(screens!$1:$1), CONCATENATE("tab.",INDEX(screens!$1:$1,1,ROW()-$B$230)),"") |
Screens | =IF(OR(ROW()-$B$230>COUNTA(screens!$1:$1),COLUMN()-2>COUNTA(screens!$1:$1)),"", CONCATENATE("transition.",INDEX(screens!$1:$1,1,COLUMN()-2),IF(ROW()-$B$230=COLUMN()-2,".selected",""))) |
You can have the same tab row for every screen. You can also elect to build tab rows that are different for individual screens. For example, you may want to highlight the button corresponding to the current screen with special color. That what the table illustrates. This will generate a tab row for every screen.
Suppose N is the number of screens, you need to copy the row (231) N times. And, then copy the column (C) N times. You should get something like this:
A sheet with the name "screens" specifies all the button screens in the interface. Each Excel table column corresponds to a single screen in the interface. The first row in the column contains the screen ID, the second row contains the screen title, the rest of the rows contain the screen's rows, one row ID per cell. The order of the rows in the WoZ interface will be the same as the order of the rows in the Excel table. Note how the first row (id "tab.row") is the same across multiple screens.
If you are generating the tab rows using Excel formulas as described in the previous section, you can insert the tab row into the screen sheet as shown in the table.
| A | B | |
|---|---|---|
| 1 | Intro | Pre-War |
| 2 | Intro | Pre-War |
| 3 | =CONCATENATE("tab.",A1) | =CONCATENATE("tab.",B1) |
A sheet with the name "colors" specifies all the button colors in the interface. Each Excel table row corresponds to a single color in the interface. The first column contains the color ID, the other three columns specify the color component values either as RGB or HSB tuples. The first row in the table contains the column headers. The attached screenshot shows the colors defined by the hue, saturation, and brightness values. If you want to use RGB, put "red", "green", and "blue" in the corresponding column titles.
Some suggested to use Excel cell styles and background color to specify the colors for the buttons. Due to the extremely archaic way Excel stores and handles colors (don't ask) extracting the color component values that would result in colors that look anywhere close to the cell background proved extremely difficult, nigh impossible.
