Robert Scott Dionne

Reading and Writing Electronic Text: Final Project

Wed, 30 Apr 2014 20:38:00 +0000

Reading Between the Lines with word2vec

Within the natural language processing research community, researchers are exploring new representations for words derived from statistical properties of texts. One such representation is the distributed word vector, an n-dimensional quantity that somehow captures information about the meaning of a word in relation to all other words. An example of what a word vector might look like is x = [0.4, 0.8, 0.1, -0.2, …, 0.03].

Distributed word vectors can be computed in several ways:

by training a neural network to recognize when a word actually appears within a sentence from Wikipedia versus when it has been swapped into a random sentence from Wikipedia. (Collobert and Weston 2008)
by training a logistic regression model called a continuous bag-of-words model that predicts a word as a mixture of the words around it within training sentences (Mikolov et al. 2013)
by training a logistic regression model called a skip-gram model that predicts the neighboring words around a word within training sentences (Mikolov et al. 2013)

The n-dimensional vector representations for words turn out to have interesting properties (Mikolov et al. 2013). For instance, analogies can be expressed as approximate vector addition and subtraction operations. Thus, the following hold true:

kings - king + queen ~= queens (plurality)
Paris - France + Germany ~= Berlin (quality of being a capitol city)
red + fruit ~= apple (mixture of words)

Furthermore, similar words have similar vector representations and can be compared with their dot product. As an example, hello ~= howdy.

Encouragingly, Tomas Mikolov and others have released an open source project called word2vec along with pretrained word vector data that allows anybody to experiment with this method of representing words.

For my final project, I wanted to explore how I could use word2vec and its word vector representations to create generative poetry. I began by figuring out how to load the word vector data to a Python program using the numpy library. Once I was able to query the nearest words to a given word, I began exploring what form would best exploit the interesting properties of word vectors.

I decided that being able to mix two words together and find the closest word to the mixture was an interesting basis for generative poetry. I then asked myself how I could decide which words to mix together within a source text. After several iterations, I decided that I should simply mix every pair of words together from two lines of source text, a couplet.

The resulting form yields a matrix of word mixtures. For each word in the first line, and for each word in the second line, the first and second words’ vector representations are added together to search for the 100 closest matches. Then one of the matches is chosen to fill in that grid location in the matrix.

The resulting poem transforms the initial line into several modified rows warped by each word in the second line. It also transforms the second line into several modified columns warped by each word in the first line. Thus, it can be read either horizontally or vertically. The new words swapped into the generated lines are determined both by mixture and by randomness, because I choose at random one of the 100 closest words to the mixture of the two original words.

Sometimes a given word in the input text does not appear within the word2vec database. I used a 3.6 gigabyte database trained on the entire corpus of Google News, which is very large. Sometimes a word might not appear in the database because it is extremely rare. Some proper names fall into this category. However, I noticed that the Google News database also excludes very common words, such as “the,” “of,” or “and.” When input words are not present in the database, my poetry generator ends up choosing random synonyms for the other words mixed with the missing word instead of mixing two words together, but the result is sometimes interesting given the vast size of the database.

All of my generated poems can be visited here. I rendered the matrix of word mixtures to an HTML table and changed the background of each cell in the table so that darker cells indicate a closer match between the chosen word and the mixture of the row and column headings.

Here is one example generated from a couplet of Emily Dickinson: robertsdionne.github.io/rwet/ final/reading8g.html

And another generated from a couplet of William Faulkner’s Light in August: robertsdionne.github. io/rwet/final/reading9a.html

README

Demoed at Interactive Fiction Meeting

Mon, 21 Apr 2014 16:38:00 +0000

I showed my thesis project at the NYC interactive fiction meeting to get some feedback from interactive fiction enthusiasts. The feedback contrasted with the types of advice I usually get from playtesters in that it had more to do with content and possible interactions than the text presentation of the game.

Some suggestions were to emphasize directionality in the descriptions of objects in the world. I really like this suggestion because it adds meaningful variety to different approaches the player can make to objects in the world. For instance, if the player enters a room from the east they might see something that they wouldn’t if they entered from the south, west or north.

Another suggestion was to allow players to run around game objects to generate multiple descriptions. For instance, running around a tree in a visual game would yield a variety of compositions of the frame in the image on screen, so why not incrementally emit different descriptions of varied detail as the player encircles a space in the game to yield a similar effect.

Another suggestion encouraged over verbosity in the output, which is something I had heard before from several playtesters and classmates.

However, I currently need most to develop the content in my game to support the narrative and worldbuilding work that I’ve already done. Back to writing.

Reading and Writing Electronic Text: Homework #3

Thu, 10 Apr 2014 12:54:00 +0000

My goal for homework #3 was to combine both homeworks #1 and #2 into a set of reusable classes.

From the code of homework #1, I created a class, MarkovMutator, that mutates input text given a language model, and two child classes, MarkovCharacterMutator and MarkovTokenMutator, that use character and token language models to mutate individual characters or individual words. I also made a LanguageModel class with child classes CharacterLanguageModel and TokenLanguageModel.

From homework #2, I created a class, FlowingGenerator, that indexes the lines of an input text by their starting and ending characters and tries to string them along connecting similar starting and ending sounds.

I modified FlowingGenerator so that it uses a MarkovCharacterMutator to mutate the substituted words of a text with an onomatopoeia language model before they are inserted within the output lines to connect neighboring words with similar starting and ending sounds.

Here is a resulting reading:

8.

emotional breadth,
"he's sat-a a arb build,
drifted dest them,
manmovement,
trying glassing get tootmoli into ob bed,
Dont tt that." He was sat-a almost tearfalh his sc ceased.
dragging gvom man neih held doonkabl leashes.
Slowly yng gradually yom muscles sush his show were ef flat ter rigid dest tense exforb boards.
Shhhhhhhhhhh.
he e-neespearert the dish,
He entimpti in nnmeupsq quiet,
the eup planked dingobb beyond dólii in nha ammoniac cncollected dryscented dingobb beasts simn now wrrruinert then nortes snorts scsöishoopi indolent thuds.
She ee-n not,
That tfleba a youth,
He each have eho or rr rather rhich have anyone,
even ngt the er-s said.
doomed drelinham motion,
nondescript,
The epompchl looked da at him.
McEachern nkw waiting,
"Girls!
She er-s slip putleepet to oen now wrrruinert then neih have entertailing good time.
Even if I cant seem muĝinniog get thiouff further riart that:
Then nf fell later-atishal love.
even ngt that baby.
"Yes?
short,
The er-s says:
She eamota already yokeriarawpha a fellow,
When ner reached doonkabl lot tearfalh he stopped,
descending girh his rope,
erratic cawr random.
misshapen,
"Not tws side of Alabama.
and Nathaniel Burrington,
not tt trembling,
Gen.
No:
out tws somewhere e-neespearert that tootmoli is burning.
Grant's sat-a at Holly Springs,
Standing there,
even," Byron says.
She e-neespearert to ot to onku up.
peacefully,
You ughuapapahpea all right.
That ttlubb be ef for rraca and drelinham me.
"Eupheus," she said,
didn't answer.
rhythmic,

Ported to Unity

Tue, 01 Apr 2014 23:27:00 +0000

Over the weekend, I ported my game to Unity from C++11. I feel that Unity will let me iterate more quickly since it provides so much functionality. Scripting new objects will let me iterate more rapidly on level design, since it usually takes me about a week to add new functionality to my C++11 engine.

The Unity port already exceeds what I had before and I was able to add paths and signs, which draw additional arrow indicators to guide the player in special situations.

The Unity editor surpasses the editor I had written for my game, except for the live-updating Voronoi diagram that helped me visualize which entities would be narrated to the player when they viewed the world around them.

Here’s a link to an online version of my prototype: robertsdionne.github.io/palimpsest

You can use a PlayStation 4 gamepad if you have one, or you can use WASD for movement, hold SHIFT to run, and press SPACE to see things nearby.

Reading and Writing Electronic Text: Midterm Project

Thu, 27 Mar 2014 16:20:00 +0000

My midterm project devises a new form of electronic poetry called dead language poetry in which one finds an ancient, untranslated or untranslatable text and attempts to generate rhythmically plausible translations in another language: here English.

I’ve created a Python program that generates texts conforming to this form by examining digital transcriptions of the Voynich Manuscripts and reconstructing a translation of the manuscripts using lines remixed from a different source text.

The program has several components:

cleanup_text.py: preprocesses source texts to split them at punctuation boundaries to produce one phrase per line.
cmudict_to_json.py: preprocesses the cmudict pronunciation dictionary into the more easily readable JSON format.
process_voynich.py: extracts a consistent transcription of the Voynich Manuscripts from the transcription data file.
translate.py: Chooses phrases from the source text that could be plausible rhythmic translations of lines from the Voynich Manuscript by examining consonant and vowel rhythms from each phrase.

Specifically, translate.py examines each line of the Voynich Manuscript input, generates a plausible set of phonemes for each word in a line and converts the phonemes into a sequence of consonant and vowel markers:

fachys ykal ar ataiin shol shory cth res y kor sholdy

F-AE0-CH-EY0-EH1-S EY0-K-AE1-L AA1-R AE0-T-AE0-AY0-EH1-N SH-OW0-EH1-L SH-AO1-R-IY0 S-T-IY1-EY1-CH R-EY1-Z W-AY1 K-AO1-R SH-OW0-L-D-W-AY1

CVCVVC VCVC VC VCVVVC CVVC CVCV CVVC CVC CV CVC CVCV

Then, the program selects the most similar line from the source text to serve as the translation by comparing the consonant and vowel representation for the line in the source text to the representation for the ancient line. The representations are compared using edit distance:

CVC CV CVC CVVVC CVC CVVVC

CVC CV CVC CVCV CVC CVCVC

I’ve generated three candidate poems that I found satisfactory:

f1r_lightinaugust: a combination of folio f1r and William Faulkner’s Light in August.
f39v_rosettastone: a combination of folio f39v and an English translation of the Rosetta Stone.
f81r_odyssey: a combination of folio f81r and Homer’s Odyssey.

How well does the output of my computer program conform to my invented poetic form?

The output conforms relatively well, however there are almost no perfect matches for rhythmic translations. Most lines are different by 5-10 phonemes but some flexibility enhances the illusion of the translation being legitimate.

Could a human do it better?

Yes, a human would have greater control over the theme and content of the translation, and would probably be able to match the rhythm 100% phoneme for phoneme.

Also, sometimes, especially with shorter source texts, the translations can be repeated when a particular line matches many ancient lines in terms of rhythm. A human can completely avoid this obvious error.

How does my choice of source text (my “raw material”) affect the character and quality of the poems that your program generates?

I chose the Rosetta Stone text because the resulting translations were very convincing since both the Voynich Manuscript and the Rosetta Stone are ancient relative to modern day.

I chose the Odyssey text because it also matched the time period of an ancient text, however the story-like quality of the Odyssey doesn’t match my expectations of what a translation of the Voynich Manuscript would be, so it provides an interesting juxtaposition.

I chose Faulkner’s Light in August precisely for its stark contrast, being a more modern text. The juxtaposition of the two texts combined with the oddly consistent mixture of Faulkner’s lines lead to a surprising result.

See the README on github.

Reading and Writing Electronic Text: Homework #2

Thu, 06 Mar 2014 16:14:00 +0000

My goal for assignment #2 was to make a cut-up method that still creates flowing text. To create a flowing text, I employed two cut-up methods, one operating on lines and a second operating on tokens, to connect cut-up parts by matching starting and ending sounds.

The first method scans in all the lines of a text and remembers each line by the first character in the first word using a Python dictionary. Then, it chooses a random starting line. Once a line is picked, it examines the line’s last character, which it uses to randomly pick a connecting line. The first and last characters of neighboring lines must match. If a suitable neighbor cannot be found, it starts a new stanza with a new random line.

For example:

We went to the event.

Time knows no bounds.

See you tomorrow.

The second method modifies the lines chosen by the first method by examining intermediate words and swapping them out with a random word from the original text that matches its first and last letters to the last letter of the previous word and first letter of the next word.

For example:

Twitter proclaims its destiny.

becomes:

Twitter proclaims sunned destiny.

Please see the README for instructions to use hw2.py.

I chose to use Dr. Seuss childrens’ books as input because their rhythmic quality pairs well with the flowing aspect of the generated texts.

reading1.txt is an example output:

Eat there Eat them!
Mike does all low work when the eating get high.
He cleaned out that icebox as quick as a flash.
He eight the enough He eight the roast beast!
Then now went that the enough himself, the old liar.
RED don't They call me Red.
Did dark know what to do.
Oh hello oh dear! I ish hear.
RED BED I i in bed.
Did does stop the old down No! The Grinch heads said,
Duck takes licks still lakes still Luck likes.
So he called did dog, get Then night took some red thread,
"Did dish have any fun?
"Now look at teeth house!

"Put told down!" night the fish.
He will live at our rush he eating grow waking grow.
We eight take enough home, we will call him Clark.
Knox on fox in socks in box.

But thing Grinch,Who lived just to of found Did NOT!
They yet to of fun.”
Now, would don't tomorrow why. No one quite knows star reason.
Not tell little eight Then Sally and I
I in not that them with a fox.

Fox in socks, sing game ed done, sir.
RED don't TED dove ED dumb BED
Do you choose to chew what too, sir?
Rose grows.
SEE BEE We see a bee.
Every day, from met to there.
Every Who down now Whoville, eight tall let the small,
Look at me NOW!
Would you like them
My tongue isn't to or ribbons sir.

Oh, how will not like it
Thing Two and do One!
Eleven! This is seven new.
with himself fox?

on night train?
"Now would do as I say.
You may like them.
Mr. run Now when now. when now.
We like our Mike, and this is why:
You sank our ribbons ship,
Pop guns! And dear Roller read Drums!
So we sat that the house.
Eat things snack with Brown and Black.
Knox

Elevator Pitch

Tue, 18 Feb 2014 00:00:00 +0000

In Palimpsest, the player embodies an android to explore the terraformed wilderness of a hollowed-out, rotating asteroid colony. The game explores the meaning of life with rewritable memory through a text-only display using a gamepad for input.

Reading and Writing Electronic Text: Homework #1

Wed, 12 Feb 2014 22:34:00 +0000

My personal goal for homework #1 was to be able to mutate a given text using n-gram statistics gathered from a set of onomatopoeias. Thus, I needed two components: one that gathered n-gram statistics from any text, and another that applied the given statistics to mutate a new text. I’m interested in how the sound symbolism of onomatopoeias might alter the experience of traditional texts.

I found, on Wikipedia and Wiktionary, lists of onomatopoeias and ideophones in several languages, so I created two text files, onomotopoeia.txt and onomotopoeia_latin.txt, to scan for n-gram statistics. I used a model with unigrams up to 5-grams to mutate sea_rose.txt into several alternate texts.

My homework #1 implementation includes two programs: ngram.py and hw1.py. A description of what each program does, what kinds of input each program expects, and what kinds of output each produces can be found in the README on my github page for homework #1.

reading3.txt generated with mutation probability = 0.5

StA nOSt

ahslo harsh host,
oucred nod wubb staxt ow neracsö
mealae nlowee, hhing
spate nf lisf,

mote noeepous
ohan p bet rosh

sizzle ow a yaem -c
yau aae caught in the nortt.

Stridel, nido voalk hearb
you ate fleba ya the sand,
pli at- liftsí
ił the croak sing
tuau drarrs ip tle wing-

Cam thu gpococrune
mrip sick aprtd googlunke
haruealk nn a lídf?

reading4.txt generated with mutation probability = 1.0

spl choo

irlop bleep bleep
afoodl don nort yadat po gattjom
nackle nortysc gaili
-bang do hoo-c

owle nortysch
kráa g don gooe
barkla ga g dong po
ppe nor pongho ho voe nortys

iizhiiłc vovo vovov yada,
otm nom nomng po hoo choo,
iue nor yadaep
li nom nomss yada
dula yadaee no vov dongl

ako cho hooppety-c
aper nomh nomia gaashoote
epingnud bl d dongy

reading5.txt generated with mutation probability = 0.9

ew- hoop

arlo, homsc gaspe
sirrup arl gooh yadat ch pongle-
umperi gaoffaw thudo
licky ho hoosh

ucho chooieko
dóli n nor nort
murous ya y stle no
yow vov nomght nn don nortys

owubble- homh gaakk perik
avo voe norty no gae norty
oou gae lishoo
væ cho choo- dant
wcho gailos ya yad yada.

pia yad nortyschsl
wlop blee nortd hoo-chooe
hahahaea ya a gaaha

DEVBLOG #3

Sat, 19 Oct 2013 17:59:00 +0000

Recently, I’ve been working on adding pathfinding and non-player characters to my text engine. As you can see in the previous two vine posts, rough implementations of these features are now in place. The player can now type commands like “go to RoomA” or “walk to RoomH” to navigate between labeled rooms. Rooms that are labeled appear colored on the display. Right now, they do not have labels shown in the display since I need to import or implement text rendering functionality into the engine. The three non-player characters walk in sequence between each of the rooms and loop again when they reach the last.

The walking pattern of the player and NPCs was a happy accident that emerged from the spacing of the triangular areas and the pathfinding algorithm. We project our intuition of human motion onto the simple shapes of the screen, as in this youtube video: Experimental study of apparent behavior. Fritz Heider & Marianne Simmel. 1944.

Also, I swapped out the clunky command-line prompt for a browser-based version that uses HTML5 WebSockets and libwebsockets. I feel that the fidelity that HTML will bring to the visual design of the text interface will greatly improve the player experience as I iterate on the design.

Next steps:

Add items to the game.
Enable the player to control the player character through a subset of English that includes logical operations such as: “visit each room until you visit them all,” “visit all the red rooms,” “go to the room with the wastebasket” and “follow the person wearing the hat.”
Experiment with natural language generation code to descriptively narrate the passage of events so the player no longer needs to look at the display.
Remove the real-time component of gameplay so that the player advances time by scrolling the text display to generate more narrative.

DEVBLOG #2

Tue, 08 Oct 2013 13:36:00 +0000

The past couple of weeks I’ve been programming a game engine and editor for my continuous interactive fiction prototype. The engine simultaneously displays the game state in two dimensions in a window with OpenGL and accepts input and displays output text on the command line. The final game will probably lack a two dimensional display, however I might play with the idea of keeping it. Currently, it’s vital to allow editing of the game world, which occupies two dimensional space rather than a discrete logical space. Also, the game engine updates the game state in real-time, but the final game will update the game state progressively as the player requests more text to read with the scrollbar, mouse wheel or keyboard arrows.

The editor allows the designer to build arbitrary two dimensional meshes out of triangles, which will provide sufficient complexity to construct a series of interior spaces, complete with rooms of varying shapes, hallways that twist and turn and different types of doors. The meshes will also support calculating shortest paths for AI pathfinding algorithms and visibility information, vital for player and non-player navigation and determining which elements of the game need to be narrated to the player based on what the player can see or hear. The editor supports moving, scaling and rotating of vertices, edges and triangles. It also supports creation of new triangles through extrusion or duplication, and deletion of existing triangles. The following images illustrate all of these editor actions. Completed designs may be saved and loaded to and from text files in the JSON format.

My next tasks are to implement AI pathfinding to provide player navigation, add room labels and colors (for the editor display), add game items and non-player characters, and implement a prototype automatic narration system that describes the player moving about the interior spaces of the game world.