Ouya

I'm just giddy

Land of Lisp chapter 7

So a while back I mentioned needing a drip-feed of neurochemical triggers in the form of a productivity chart. I'm now 2 weeks late on a promise. Call it supporting evidence. So moving right along.

Chapter 7 covered more interesting list structures than have been used so far. It also comes across as quite dry so I'm liable to not spend much time in spots.


We start with dotted lists and pairs. A dotted list is simply a list that isn't terminated with nil. If we go back to thinking of Lisp lists as similar to a linked list structure a dotted list would be the same as plugging a data value into the last pointer in the chain. Which would be fugly in a linked list but works out alright for Lisp. Incidentally the dot notation used to print these lists

can also be used as a replacement for cons as in (1 . ( 2 . 3)) or for a common list as in (1 . (2 . (3 . nil))). I would assume using one form over the other would be a matter of taste and convention. The use of dotted lists regularly in code is supposedly uncommon however the book does state that running across such lists happens often which is quite contradictory and I'm not sure which side of the tale to believe. Either way I'll know what I'm looking at when I meet one. There is a relatively standard usage though in concisely describing pairs of things.

Additionally circular lists are discussed; these are simple enough to me that I'll gloss over them. Basically, the cdr of the final cons cell points back to one of the prior cells in the list. You could, if you felt like it, craft a very complex list using a combination of circular references. You'd also confuse the hell out of the interpreter unless you warned it that you were going to by issuing (setf *print-circle* t) which basically tells it to take a little longer in the print routine to make sure it doesn't get stuck printing in a loop.

Association lists (or alists) are also discussed which is, predictably, a list of pairs though presumably it's a list of keys consed to values which could themselves be further lists. Discussed here is a list of dot pairs. This is a convenient structure but apparently becomes inefficient when the list grows beyond a few items. Finally it discusses tree structures in data which we've already been using to some extent. If you've been watching you can see that Lisp lists are adept at handling trees as each branch basically has a symbol by which it can be referenced in the car of its cons cell. If I rip an example straight from the book

we can see this clearly. The *house* is composed of walls, windows, and roof. Each of which is a symbol identifying a list of its component parts.

The last half of the chapter covers Graphviz which takes formatted text input and generates image graphs based on the data. This is largely just data format manipulation code that takes the node and edge parameters defined for the wizard game and converts it to text output properly formatted for Graphviz. I'm not going to discuss this because it's largely boring; feel free to dissect the code linked below. Just as a clarification, data conversion is at the root of the beginning and ending of every program. It is vitally important, clearly, and a topic worthy of understanding. However the majority of the code I'm skipping is just string manipulation. Not only is it a thoroughly understood form of data manipulation it is, in my opinion, extremely dull; like a video game played 4 or 5 times already becomes dull in a vaguely interesting way. I'm going to cover new topics and functions introduced here but I won't be bothered with the boring details because I want my life to be more than vaguely interesting.

What I will discuss is new functionality. The substitute-if function

feeds each item in the target through the predicate-function and when true it substitutes the subvalue and returns the final result. This is demonstrated to work on lists and strings though I suspect it works on lists and a string is simply silently converted to a list of chars and back again.

The complement higher order function returns a function that basically inverts a predicate function. So

returns a function (which could be fed to another higher order function or presumably used in lambda) that returns the opposite of the alphanumericp function (which appropriately enough returns true when given an alphanumeric character). So in short, the function provided by complement above would return false when given [a-zA-Z0-9] and true for any other character.

A noteworthy concept on display in this chapter is the use of side effects. In each of the functions created to reformat the data we aren't really concerned about the output of the function itself. Instead the functions print the data to the console using princ and the output is ignored. This allows us to have greater control of what gets generated and easily review and debug that output. In the final function we'll simply capture the console output to a file. This runs against functional programming style as it relies on side effects. I honestly don't care; it works, it's clean, it facilitates code reuse (which we'll see in a minute) and I'm not religious about programming technique.

Once we've created all the functions required to reformat the data into a Gaphviz format we need to run that data through a Graphviz converter. The following function handles that task.

This gets interesting. Firstly, there's that thunk parameter. What a funny name for a symbol. According to Wikipedia (clearly the authority in computer science studies >.> ) thunks were invented by Peter Zilahy Ingerman and the term was arrived at because it's 'the past tense of think at two in the morning.' Which I can entirely believe. I'm not a hardcore compsci major but I'll do my best to take a whack at describing these, if only to help solidify my own understanding. Also called suspensions these functions offer a means of controlling the timing of evaluation of computations. Why this might be important (beyond deep efficiency mining) isn't readily apparent to me. The book makes the case that wrapping the graph->dot function inside a thunk delays the computation and allows the dot->png function to capture its output but I would argue that the same effect would be had if you simply plopped the function directly into the dot->png function as in this code

That gets you precisely the same output via the same methods. So it seems to me that what the thunk amounts to in this usage is to blackbox the creation of the data processed by the dot->png function. This is something I definitely understand and it allows us to plug any function name we want into the code above provided that the function outputs to console and (preferably) generates data parseable by the Graphviz 'dot' command. This also gets us off the hook for not actually outputting the correct data from the function directly. We use the thunk as a nullary (parameterless) function and ignore its output relying solely on the side effects of the function and the use of global values to perform the computations required.

Now, some less crunchy aspects of the function. The with-open-file function is very similar to let in that it defines a symbol to be used in the following forms.

The symbol becomes a stream variable (glossed over here) which is then available within the with-open-file form to be written to. In the usage above we very cleverly redefine the *standard-output* global stream variable (which is coincidentally the default stream for Lisp printing functions) to point to a file which is opened in write mode (:direction :output) and overwrites any existing file (:if-exists :supersede). Thus all the prints done from within the thunk (which we execute with funcall) are written to the file filename due to the dynamic value of *standard-output* from within the with-open-file form.

Now would be a good time for a quick interlude on keywords. These are symbols prepended with a colon as in :if-exists. These might be thought of as constants (though I'm unsure of the accuracy of the comparison to a C constant). They have an inherent value and that value is themselves and cannot be changed.

These are used in a number of functions to define various options such as file open modes. Options that might otherwise be incomprehensible strings of apparently arbitrary characters (yes, I'm looking at you Perl).

The last part of the chapter discusses the functions for generating an undirected graph. The siginifcant difference here is that the uedges->dot function uses maplist instead of mapcar which (as you might expect from the mnemonic) executes a function across a list (map) but instead of sending each car it sends the remaining list at each iteration (car vs list). Then the lambda function uses unless to skip any edges that are repeated later. The end result is that only the last direction of an edge gets printed but since the different directions of travel are meaningless in this context that's what we want.

The code for this chapter is at:
grapher.lisp
TheWizardAdventureGame.lisp


You'll want to do the following:

to get things running (using the correct paths of course).

Possessions obsessions

So listening to Papa Roach, "Between Angels and Insects".

"There's no money. There's no possessions, only obsession."

I'm struck with the thought that my obsession, yes it is, with video games and more particularly the kind where you accumulate 'stuff' is a sublimation of materialism to assuage the pain of not being able to accumulate real world 'stuff' for so long. I wouldn't call myself deprived but during my childhood we never 'kept up with the Jonses' nor made even a cursory attempt to and I was given a general impression that we were broke. However I went to a school district where 'the Jonses' were effectively idolized. I felt my family's purported lack, whether 'real' or not, and now I think it has perhaps had a lasting impact on my psyche. I feel compelled (yes, that's the right word) to play certain games. Not puzzle games, not shooters. I used to enjoy enforcing mastery on a difficult system (for a long time Gradius II was the best thing in the universe when it was new) but I'm not so good at twitch mechanics anymore. All that's left is really the grind for more ingame 'stuff' which, actually, I end up finding quite boring and not fun

So recognizing this drive, can I control it? Harness it? Expunge it as poison? Games have inherent value, yes, but not the way I tend to consume them.


Coming back and cleaning up the above into something approaching coherence I had the thought that I shouldn't blame my father (that's where the majority of my impression of destitution came from I suspect) as his parents were, my my admittedly faulty estimation, poor. I imagine it takes a solid generation to expunge that from the family's psyche.
I've also read the full lyrics, instead of just listening to them, and am struck by ... well, by the whole thing. I was going to look at a couple of lines but nevermind. Here's the whole thing.

Last word. I'm amused how the close-mindedness of Religion would deprive me of such insight. Rejecting things that don't immediately fall in line with your beliefs is idiocy no matter what terms you couch it in or justifications you give. If scripture is your excuse to reject something without considering its value then you devalue scripture.

Land of Lisp chapter 6

Whoops. Naughty me, I did the 6.5 post before the chapter 6 post. Well, 6.5 was more interesting and easier to write at the same time. So it got done first. Onward; pretend all the lambda navel-gazing hasn't happened yet.

Now this adventure game is pretty ugly, and very user unfriendly. Time to clean it up a bit. That's going to require some better I/O functions.

The basic screen printing functions are:
print: Prints one output per line, strings in quotes. Perfectly machine readable later.
print1: Dispenses with the newline
princ: Human readable output.
For reading input:
read: Just the basic Lisp input, parens and quotes required.
read-line: Reads until the user hits enter and takes the whole input verbatim as a string.
read-from-string: Just like read except it takes a string input instead of reading from the console.

With that it's time to learn one of the more powerful functions. eval takes input and parses it as code. So

executes the form with the (princ "bar") output that you'd expect. This is quite powerful but also dangerous. And it's dangerous in ways that are not necessarily immediately obvious. I'll get to that.

At this point we have all the capabilities to build our very own REPL. Rather like opening a DOS shell within a DOS shell. You might compare it to running Windows in a VM except that the VM handles a lot of background operations and that would be more akin to the next step in this chapter. To the point however, here's the dead simple code for a basic, nofrills, 'custom' REPL (custom in quotes because it's about as bog standard as it gets even though it's a REPL in a REPL).

That's all well and good but when we execute (my-repl) nothing really interesting happens. Visibly at least. Because, of course, the functions we've used have no functionality beyond the basic REPL. We haven't written any. So that's clearly the next step. Now is a good time to note that I'm storing the code for this custom REPL in a separate file because it becomes essentially a general use engine for running simple text adventures. The file will be linked at the bottom along with loading instructions (such as they are).

I'd also like to note in advance that I think the way the game-repl is implemented is a bad paradigm. It basically uses recursion to perform the 'loop' structure. I added a printout to the book's code to make this obvious; basically when you 'quit' you'll get a playback of all the commands entered. Now while this implementation is tiny compared to modern memory sizes it's important to note that recursion has some overhead (which is also made obvious by the printout). It literally does not matter here due to the scales involved, I want to stress that, but it's important to note the point. Maybe this small structural complaint is fixed later in the book, I dunno. But on with it.

First we want to be able to take input. So we build game-read.

Take a moment to look at the flet function quote-it. What does (list 'quote x) do? A heretofore undescribed complexity is that we've been using a shorthand when quoting. 'foo is the same as using the quote function as in (quote foo). So what quote-it does is create a list that is a form of the quote function. It literally quotes its input and returns it. It's important to note that this case doesn't require #'quote because it isn't being passed as a parameter to a higher order function; the full special form is created using list so it will get evaluated normally. So to describe the game-read function fully, first we read a raw line of input, then we create a string (because we requested it by specifying 'string) by concatenating parenthesis around the input. Then we read that string into the cmd variable. Then we quote the cdr of the entered command (if it's more than 1 list item) using mapcar and quote-it, cons the thing back together, and return that result.

Next, we need to evaluate what we just read.


Two things to note. First, I added the case expression. It's completely missing in the book. But I'm afraid 'pickup' is just awkward to me and since get is its own special form it needed to be handled specially by the evaluation function. So I select it with case and convert it to a 'pickup' verb instead. Additional verbs can be added to the case form as needed, though if there are many then a flet might be in order . Second the *allowed-commands* global I'm placing in the game data file. It's specific to the game so doesn't need to be in the custom REPL code. It's defined as (again, I added 'get' myself to get it past the member filter):

So this is pretty straightforward. The only complexity is what I added. An if test is performed to determine if the input verb (the car of the input expression) is allowed and it's either evaled or an error is generated.

Finally there's the output. This is the most complicated. As usual.

Some of the lines probably break in the wrong places which makes it boogers to read. I really need to find a decent scroll block to plug in here, but I'm lazy and have other things to do. Look it up in the code file if it bothers you. (Edit: Should look much better now that it's in gist!)

Basically what happens here is the input list to game-print is printed to a string value by prin1-to-string which works the same as prin1 but returns the string instead of printing it to screen. That string has parenthesis and spaces hacked off the ends by the obvious string-trim function and the output from that is coerced to a list of characters. Then tweak-text is called which handles capitalization (there's nothing new in the function so I'll leave figuring out the logic as an exercise... oh wait, char-upcase and char-downcase... surely those are completely obvious...) and the resultant list of character values is then coerced to a string, princed and a fresh-line is output. Phew.

Now, to plug this all together (with my aforementioned rudimentary stack trace):

So, if you input 'quit' the recursion ends and all the input cmds are re-output.. Dead simple.

Land of Lisp chapter 6.5 - Lambda

Okay, the chapter (sort of) on lambda. Perhaps I'm dense, or inexperienced, or just plain stupid but I'm not sure I see what all the hype is. This (half) chapter, for all its hyperbole, does little to dispel the mist surrounding the importance of the lambda.

It begins with a discussion about how functions are first class values in Lisp. You can pass them around the same way you do any other variable (presumably you can edit them as well... (cdr #'myfunc) being the list of parameters perhaps? But now I'm talking about things I don't know...). This is well and good and useful. You can pass them on to higher order functions all over the place if need be.

To come back to lambda, the special form is

lambda is a macro, not a function, and returns an anonymous function (not a macro, or a llama). Because lambda is a macro it doesn't evaluate all of its parameters prior to execution, so the {code} passes in unaltered (I assume anyway) by the parser.

I suppose what I'm failing to understand is how this is more useful than, say, constructing an actual function (if you need it more than once) or simply placing code inline. For example:

Is functionally equivalent to

and

(I suppose both of those are really the same solution, one just explicitly recurses where the other lets mapcar deal with that complexity) Once I've defuned a function I can use it in multiple places. A lambda function is, by virtue of its anonymity, incapable of such feats (unless, again, I'm missing something?).

Now that I try to think about it though, there doesn't seem to be a C-like loop structure in Lisp. Not that I've gotten to yet anyway. If there is you should be able to simply put the (/ x 2) inside the loop directly. (Edit: There is, it's in the next chapter!)

Now, is the lambda code above 'cleaner'? That depends. A neophyte would probably say 'no'. The mapcar snippet is nearly as concise and more human readable (which is desirable) to a beginner. A programmer steeped in Lisp will, apparently, disagree.

Then of course there's the near-mysticism that the author gets into about Lisp growing out of lambda calculus and how it's really sort of the only function in Lisp, etc. Which is great if you live in an ivory tower and get a stipend for getting other people's brains in knots (especially if you still appear to know what you're talking about). Unfortunately, I don't see (yet?) how this makes Lisp superior to other languages. But. This is the first brush with it and clearly there's something here if the amount of ink spilled on it is any indication. So hopefully understanding will come shortly.

Land of Lisp chapter 5

Chapter 5 is exciting. We get to actually write a bit of an engine (if that's not glorifying it too much) for a text adventure game. Okay, it's 3 rooms and 4 objects and no interactions to speak of yet. That's not the good part. More content can be added easily enough by extending the global variables. But I'm getting ahead of myself.

One language syntax that was introduced is quasiquoting. Basically this is a syntax that allows you to embed bits of code inside a data list. This is familiar to me since I work with a scripting language that does this by default. In Lisp

notice the backtick instead of single quote. That indicates quasiquoting 'mode' (not sure mode is the right term in the same way it applies to data/code mode but whatever). The comma indicates the beginning of a code block and Lisp automatically returns to data mode once the list closes. In this case I could have left the parentheses around foo off, but I left them in to make the end of code mode more explicit.

In the builtin scripting language for UC4

So this might have been a concept unique to Lisp(s) once upon a time, but no longer.

The second bit of syntax is the use of #' to indicate that the following symbol is a function name. This is necessary because Common Lisp maintains separate namespaces for variables and functions. Apparently this means it is called a Lisp-2 language. This in comparison to Scheme which, as a Lisp-1 language, maintains a single namespace containing both functions and variables. So you couldn't have a variable named car in Scheme (car is a standard builtin function in Lisp, I assume it is in Scheme as well?).

There is also some more discussion of functional programming but that isn't why I'm working through this so I'm not going to spend more time on it.

This chapter also introduces the conceptual architecture for the Wizard's Adventure Game and constructs a good bit of basic functionality. We view the game as a mathematical graph; each room is a node and the paths between them are edges in the graph. Why this particularly matters enough to bleed ink on I don't rightly know but I'm not a game designer (yet?). It's as good as any visualization and probably better than some.Using the parlance of nodes and edges we proceed to build the tables for rooms and the paths between those rooms as global variables.

I'll link to the full (and developing, I'm not going to keep versions for each chapter) code for the game at the bottom. In the code above we have a pair of associative array... er, lists describing the nodes and the vital information for the paths. We also build a series of functions to pull out the specific room information. These functions (which I'm not going to bother copying in here, download the source if you're curious) use the assoc function to retrieve a specific element from the associative list.

which we can then pull the description from. We also created a list of objects and a list defining where those objects are located.

We also used the append function to join lists together. This is conceptually equivalent to string concatenation except it applies to lists. We used append in conjunction with apply basically to flatten out nested lists created by mapcar. mapcar is what's called a higher-order function because it takes another function as a parameter and does something with it. In this case, the function is executed against each item in a list

and the returns are consed together into a unified list.

The apply function is also higher-order and it takes a list of inputs and feeds them one piece at a time to the function parameter. This is a convenient way to remove a layer of nesting from a list, but I don't entirely grok the logic. I'll need to feel this one out some more.

In this case we apply-ed the append function to the nested lists created by mapcar to get a flat list of symbols that reads like text. One thing I forgot to discuss at the top is the deliberate use of symbols instead of strings. This isn't imposed by the language, but the language makes it much easier to work with lists of symbols instead of strings, apparently. We could certainly use the string data type for this, but then functions like cadr would work differently on them and would require more code.

We then proceed to take all these things and build a rudimentary user interface. Functions are defined for look, walk, and pickup. Of these, only pickup is really interesting; it uses push to add a variable to the front of the object locations associative list with the value of body. Since assoc only returns the first item in the list that matches this effectively hides the original entry and the item now is only visible in inventory (also a function built at the end of the chapter).

Here's the link to the code as it currently exists. Bear in mind that this will be built upon further in later chapters and I'm not going to bother keeping multiple copies. If you want chapter-by-chapter code buy the book.

Land of Lisp Chp 4

Chapter 4 covers exclusively boolean operations. It starts with a discussion of the only false data value in Lisp which is the empty set '(). This differs from Scheme, though the details of that aren't discussed as they're clearly outside the scope of the book. Note that it is represented in data mode because it is a data value, not a form.

To complicate matters (which seems to happen a lot in this chapter) '() isn't the only way to represent falsehood. This is a careful distinction which I'll come back to. The following are all false in Lisp

The first is just the empty list. The second resolves to an empty list due to the way Lisp parses forms. nil is actually a constant that resolves to 'nil and part of the Common Lisp spec is that 'nil and '() are equivalent. Why are all 4 deliberately set up this way? The book doesn't say. I suspect backwards or cross-dialect compatibility, but to be honest I don't really care right now. It is what it is, as my boss is fond of saying. The important thing is that an empty list is false, and any list that is not empty is true by definition (though in practice the 'nil can present some gotchas that will come up later). This lends Lisp to easy recursive processing of lists like so.

The above recursively cdrs to the end of the list and adds 1 for each symbol; the false branch of the if form returns 0 for the empty list at the end

Next up is if. Which has already been used but not discussed. It's hard to talk about boolean operations without 'if' in seemingly any language. Lisp's if is a bit stunted compared to something like a dialect of C. if only does one thing. Because of the way the command is structured

you can only evaluate one list. Which is a tad misleading since that one list could be a function form that does a million things or, as the book suggests, a call to progn which is glossed over here but apparently behaves similarly to an eval function. Still, it's an important thing to remember and it would tend to reinforce functional programming if that's your thing.

A key feature of if is that, like many other languages, it performs short circuit evaluation. The book spends a lot of ink on this, but I shan't bother except to say that if can do this because it is a special form as opposed to other forms which evaluate all their parameters. This is glossed over here as well as macros which are apparently similar. This will come up again in chapter 16 it seems.

There are a number of alternatives to if. when and unless are mirror siblings.

when executes its commands if {boolean} is true and, as might be guessed, unless executes its commands if {boolean} is false. These each will take an command list of arbitrary length. Both return nil and do nothing if {boolean} is not the correct value.

Finally there are 2 conditional functions that are fully flexible. I almost said functional but while the prior conditionals have been somewhat less flexible than the conditionals of other languages it's worth noting that they're easy to understand and read, and are largely self documenting. These two, on the other hand, are more like another language's switch/case statement, are complex to read (lots of syntax which, in Lisp, is parentheses and hard for an unpracticed human eye to track) and not very self-obvious. They can be formatted to be easier to read and written to be, or the surrounding code architected to be, easier to grok but there's a limit to how much that can accomplish. The book mentions none of that and perhaps I'm just suffering from a neophyte's inadequacies; it is mentioned that seasoned Lispers prefer cond because of it's generality. I don't know, but without further ado however here is the cond and case functions.

Clearly these two can quickly become hard to read. Nevertheless they're probably called for in some circumstances. cond takes a what is basically a list of when forms. The first matching {boolean} is executed and the remainder ignored. As one might expect the final {boolean} is often a true value to drive default behavior. case is nearly a C style switch statement. It takes a {symbol} which it will compare to the various {value}s defined in its body using (specifically) the eq function (this will have more meaning in a minute) and executes only the matching branch. It isn't stated explicitly but the otherwise branch is default behavior.

There are 2 boolean operator functions of note. and ... and... or. These each take a list of parameters and return a boolean based on the boolean evaluation of those parameters much like you would expect. There is an interesting side effect of the short circuit operation of these functions. Take the following example (ripped straight from the book).

In the above form the 3 parameters are fed to and. You could imagine this being called when an editing program is closed. First the *file-modified* variable is tested and if it's false and immediately returns nil. If it's true then the function (ask-user-about-saving) is tested. Presumably this takes input from the user. If the user inputs a value that is evaluated to true then the (save-file) function is executed. In theory this would return a success/failure variable which would then determine whether and returns nil or true. All of that in one little line. That's some pretty dense semantics. It might be better to use a more verbose if structure if the code needs to be maintained by neophytes since all of the relies on some subtle effects. Personally, I like the density and this case isn't terribly difficult to figure out. Maybe it's just because I grok short circuit evaluation, I dunno.

In a slight tangent the book interjects that one subtlety of the Lisp definition of false is that functions get a freebie return value. Anywhere that a function could return truth or falsehood it would be better to find some more useful value to return than just true; if you're testing whether a variable is defined you might as well return the value (unless it's nil!) since it will be evaluated as true regardless of the actual value. The book uses the member and find-if functions to discuss this but I'm not going to get into them since this is already long and I want to move on to chapter 5.

Finally we come to the equality functions. The book discusses 6 commonly used equality functions. Yes, 6! equal, eql, eq, =, string-equal, and equalp. The rule of thumb is to use eq to compare symbols (it's fast, efficient, and doesn't work well on things that aren't symbols) and to use equal for everything else. I don't have the gumption to dig far into equality comparison right now but will point out that equalp will handle case insensitive string comparisons as well as comparing integers to floating point values.