Change the name

So I changed the name of my Blog.

I really wanted to have something more generic to cover both programming and writing Fiction but nothing came to mind so I settled for Lisp, the AI language, the cool language. It is really more my language of choice even if I use Python for practical reasons.

Still newLisp shows some promise in being a practical lisp despite its warts and if I'm motivated enough Warts can be removed and replaced with something better.

New Lisp warts

Well I haven't need to dig too deeply to find warts in NewLisp. and some of them do seem to be by design.

1. No tail call optimization (and nver will be or so the creator has said) seems a little antagonistic to functional programming. And pass by value is not your friend here.
   
2. Contexts are inadequate for any of the jobs they are supposed to do, and they have been overloaded with too many jobs.
   
3. Macros facilites are inadaquate, as there are no gensyms and there is no macroexpand function.
   
4. Memory management seems to be a dogma. The only problem I have with pass by value is that it seems to go against the grain of functional programming. where you pass in functions expecting them to not modify their arguments, so why copy them? I think copy on write would be a better strategy. to adopt, perhaps a little more complex but favarable to functional programming.
   
5. No Type system. THere simply is no facility for user defined types. and hence no way to do anything interesting like generic functions
   

Why contexts are inadaquate

Context take the place of Module, hash tables, objects (closures really) and pass by reference mechanism.

They fail as a packaging system as they have a flat namespace so that all modules must be global. Every scripting language that I know of supports hiarchical modules simply because it works better and reduces the changes of two differnt packages defining the same module. Without hiarchical contexts developing an extensive standard library for newLisp is going to be impractical. Note that all of the successfull scripting languages have large standard libraries, it is just one of the signs that a language is a success.

As hashtables they sort of will do but are inadaquate simply because you must stick all your hashtables in MAIN, where they share a namespace with any loaded modules. So I can't have one without exposing it to the world and the chances of name collision have increased again. If I could create an anonymous context and bind it to a local variable that would be better.

As a method of passing by referance. Sorry but that is a plain ugly hack, which forces the proliferation og global objects. Again if this must be the only way to do it then anonymous context are what is needed.

As Objects. Well no what we have here are not objects but what every other Lisp would call a closure. Except that again while in all other Lisps closures are anonymous here they must be named, in the same namespace as Modules and Hash tables (boy its getting crowded in there). Yet again anonymous context seem to be what is missing.

Unfortunately neither Hiarchical Contexts or Anonymous contexts are likely to be added. Form what I have seen the Project maintainer has already expressed the opinion that he does not like the idea of hierarchical Contexts. Anonymous ones meanwhile would I suspect play havoc with the whole ORO memory management Idea. As once they existed people would use them (quite Heavily I imagine).

Back to Lisp

Looks like I'll be playing wtih a little Lisp again rather than Python. May even need to chagne the name of this blog. This time in the form of newLisp this is a very small and seeming agile implementation of lisp.

It's not a Common Lisp, but instead does its own thing in what appears to be a very light weight manner, which still feels like lisp. All the While focusing on practical things like providing Network (indlucing HTTP) database, Threading and even GUI API's out of the box. As well as what seems like near seamless integration with C librarys, though only time will tell on that one.

Some other features are a little unusual such as a return to older scoping rules. And a namespace implementation called contexts.

Some of the design decisions have been critisised hevily, but others areue along the lines of try it and see it actually works out without being insane. So I'll go down this route.

As a first remark it is a small distibution at 1.1MB of source Tarball and is very easy to compile out of the box. For me on two differnt machines (with different Distos) It just worked.

I have two projects in mind:

1) See how much of the stuff in 'On Lisp' I can get working in newLisp
2) write a Rouge like console game using libcaca

Back to Basics

I seem to be seing stories about how software is getting slowe all the time. Part of the reason is that we expect it to do a lot more stuff, but do we really needed. The most entertaining Benchmark was comparing an Apple plus I beliefve wit ha Duel core Athlon 64. The Apple seemed to complete most common tasks faster than the more modern computer.

The biggest difference was in terms of boot time. Recently I have been dissatisfied at how long my somewhat dated (Pentium III) laptop was taking to boot. My most common use case for the laptop is

1. power up
   
2. start a terminal
   
3. Navigate to a file
   
4. edit the file with VIM
   
5. save & shut down

with that use case, why am i waiting so long for the thing to boot. Granted I didn't time it but it was definetly longer than I needed to do what I wanted to do. So I decided to try something

I made some space and installed a minimal Debian, added the programs I really use (but no X11)
and away I went. After a little bit of tweeking (and some helP) I got a framebuffered console and a nice large font (which still gives me just over 80 characters accross.

end result is that I now have a 25 second wait from pressing the power button to login prompt, and a system which does exactly what I need, and nothing I don't.

and on those rare occasions when I would like to do something more I still have Ubuntu installed in a separate partition and can ask grub to boot into it for me. All it all it was a reasonably simple process, and it just goes to show that the console should not be discounted.

In many instances it is stil the tight interface for the job and it is a pity that things like getting autoconfguring the framebuffer and getting a good font selected are not automateded in any way.

Python 3000

I don't know If I like some of the changes that Python 3000 will bring. Introducing a destinction between string and binary data, seems like a backwards step. It seems to be more Windows style than Unix stile and harks back to the bad old days of breaking files when reading them via FTP.

Is this the cost of using Unicode encodings, and if so is it worth it. Granted I do all my writing and coding in english so I won't really gain anything from being able to use Unicode Identifiers. But all in all it seems clunky.

Dropping the reduce function entierly also seems a little extreme, like many people have already said, just move it to join all the other functional programming functions.

On the other hand Generic Functions, which are a great paradime (and would make my multiple dispatch post oboslete, stand a danger of being droped due to lack of activity. PEP 3124, which describes generic functions actually covers almost the entire CLOS (Common Lisp Object System) specification.

On balance I have a feeling that Python 3000 is just not going to be the same language, the number of differences seems so significant that best practices and general approaches will have to change. Porting to Pythong 3000 looks like it will not be a trivial task. All In all it will be interesting to see what hte adoption rate will be. A lot of the new features will also be available in Python 2.6, It may end up that everyone moves to python 2.6 and stops there preferring to keep the old style ways of doing things, such as havig a single unifed string/ binary type, and does not go any further.

Implementing the Visitor Pattern

The visitor pattern is a very interesting way to go if you have a lot of operations
which need to be applied to one data structure. Especially if this is a recursive data structure such as a tree.

Python provides a very interesting way to implement the visitor patter and get
around some of its inherent problems. Granted when done this way you are really getting something closer to double dispatch then the visitor pattern.

Allow me to take a short deter into CLOS (the common lisp object system). Objects
in CLOS bear more of a resemblance to Structures in C then to Objects in a language Like Java or Python in that a CLOS object has no methods. The methods are defined separately as generic functions. One feature of generic
fumigates is that they can select an implementation based on multiple arguments, where's in most OO languages only the first (the calling object) is considered. This idea is really quite powerful.

back on the python front. Here is an interesting way to implement the visitor
pattern using the available introspection tools.

class Node(object):

    def __init__(self):
        self.MethodNames = [ cls.__name__ for cls in self.__class__.__mro__]

   def accepts(self, visitor, prefix='visit_'):
       for name in self.MethodNames:
           vName = prefix+name
           if hasattr(visitor, vName):
               # we could have allowed getattr to raise an exception but this is generally slower
               # then an explicit check. In this case we assume that the exception would be 
               # thrown regularly under normal operation and would hurt performance.
               getattr(visitor, vName)(self)
               #do any cleanup activity
               return
       raise AttributeException("Visitor %s does not contain a suitable method." %  visitor)


class Visitor(object):

    def visit_Node(self, node):
        #do something
        pass

The important thing about these code is the use of __class__.__mro__
__mro__ stands for Method Resolution Order and is a list of class objects in the
order they are tried when a definition is needed for a method. In Older versions of python this is a simple (and fundamentally broken) depth first search. The problem with simple depth first search is that it doesn't handle
multiple inheritance corectly. Since Python 2.4 a more sophisticated algorithm is used (incidentally the same Algorithm as used in CLOS).

We have in effect created a Dispatch method which is polymorphic on the type of the current class. And moreover has the same method resolution order as the python language its self. The main advantage of this approach over a more
traditional one is that we are free to extend the Node class hiarchy at any time
without breaking existing visitors. Granted the dynamic dispatch will be slightly
slower than accept methods which explicitly call a particular method.

Other side of the Coin

There is always a counter example. Having just saying that dynamically typed languages are better then statically typed ones, I find my own counter example in the kind of bug that static typing would have caught easily and simply.

Recently I have been looking at two things, Databases (and especially SQLObject) and cross platform GUI libraries. I seem to have settled on wxPython for this. wxPython comes with a great Python shell called PyCrust, this once was it's own project but the maintainer agreed to fold into the main wxPython distribution.

PyCrust has a namespace browser (which I thought was quite a nifty idea) except every time I tried to brows SQLObject it would hang. And the problem was name collision. The Namespace inspector in PyCrust checks for objects that expose a method called _getMethodNames, which will assist it in displaying the correct attributes. It just so happens that SQLObject has a class which will happily return a callable object for almost any attribute you ask it for. PyCrust is expecting a list of strings and gets an instance of a class it knows nothing about, to make things worse this class overloads the + operator for its own purpus (it is used
to dynamically construct SQL queries from python expressions).

An error is only generated 20 lines of code later when PyCrust attempts to iterate over what a local variable which should contain a list (but dosn't). Has this changed my mind about dynamic typing... Well No. In the end every language feature has benefits and drawbacks. Dynamic typing opens the door for bugs of this type which can be quite hard to locate. It also gives flexibility, a lot of flexibility, this can make the programming effort a lot easier, as it promotes decoupled code.
In python it is relatively easy to create an objects which are interface compatible with other objects without having a dependency on them.

So what's so good about Python

So what's so good about Python? All things being equal I would program in Lisp, because on Language features I think it rules, but there are problems including lack of libraries, lack of portability and difficulty of installation, the fact that I am yet to meet a full implementation of the Common Lisp standard.

A few years back I tried Franz Common Lisp and it was the best tool I have ever used for developing a GUI (you can make changes to the app while it is running, find an error, fix it and try again (and this includes the GUI), the only problem is the cost, I couldn't afford it, even if I managed to convince my employer to use a Language which almost no-one (who isn't a programmer) has ever heard off, they could afford it either.

When I found Python it was really the next best thing (in the interim I had messed about with Haskell and Prolog). Now I'm not the only Lisper who has ended up in Python, the lambda expression was only ever included because so many ex Lisp programmers where asking for it. Strangle I've never really used it (lambda's work great in Lisp but they just don't seem to fit in python, its an element which doesn't fit).


The other good points in my opinion:

• Interfaces are implicit. Nothing annoyed me more in java that exceptions based on the compiler telling me this is the wrong type of object. I don't care if it implements a next() method I will not call it. With dynamic typing the system will happily call my next method.
• meaningfully whitespace. Using a return to denote lines of code is an excellent idea for so many reasons:

1. Lines of Code can have a definite meaning, there is no point in arguing about this, what constitutes one logical line of Python code is defined in the Language reference.
2. How many times has the average programmer forgotten to press return at the end of a line? How many times has the same programmer forgotten a semicolon ? Most C text books will have a paragraph which goes something like this, because semicolons delimit statements it is possible to put more than one statement on the same line ... But this can make your code harder to read and is considered bad style. These days when I write JavaScript I don't put the semicolons either.

In short the dynamic nature of the language means that you can take shortcuts and avoid the need to write boilerplate code. The SQLObject library
takes this a step further be implicitly creating properties based on the existence of _get_ and -set_ methods this is a very pythonic approach.

what's missing:
There are also a couple of features I would like to see:

Delegation: this is a neat feature of Delphi (I will go out on a limb and say the only neat feature) claim to the world that you have implement interface X and then delegate it to a member variable. In essence this is a tool for avoiding typing which will generate a whole set of stubs for you such as

def method(self, *args, **kwargs):
return self.member.method(*args, **kwargs)

I have some Ideas about how to add something like this to SQLObject, which I'll be talking about in my next post.

Macros: Someone coming from C will have a long list of why macros are bad and dangerous. The problem is that the C macro pre-processor is not a true part of the language and is text based. Comings from Lisp I am familiar with an integrated macro expander which is part of the language and understands the language, hence it can solve all of the problems, associated with macros. And why would you want this... For speed optimization, and writing programs that write programs, if you can predict a head of time that a loop will execute exactly 20 times then why not unwrap it ... Because writhing the same
lines with minor variations is annoying and error prone (even with copy and past), so let the macro expander do it.

The first problem however is one of syntax? In Lisp macros fit, they look don't require any major contortions to express. I'm not the only one who has raised the idea of adding Lisp like macros to Python but so far no-one is come up with a model of what said macros should actually look like.

OK That will do for this post.