abu@software-lab.de

Monk: "If I have nothing in my mind, what shall I do?"
Joshu: "Throw it out."
Monk: "But if there is nothing, how can I throw it out?"
Joshu: "Well, then carry it out."
(Zen koan)

Pico Lisp Frequently Asked Questions

(c) Software Lab. Alexander Burger


Why did you write yet another Lisp?

Because other Lisps are not the way I'd like them to be. They concentrate on efficient compilation, and lost the one-to-one relationship of language and virtual machine of an interpreted system, gave up power and flexibility, and impose unnecessary limitations on the freedom of the programmer. Other reasons are the case-insensitivity and complexity of current Lisp systems.


Who can use Pico Lisp?

Pico Lisp is for programmers who want to control their programming environment, at all levels, from the application domain down to the bare metal. Who want use a transparent and simple - yet universal - programming model, and want to know exactly what is going on. This is an aspect influenced by Forth.

It does not pretend to be easy to learn. There are already plenty of languages that do so. It is not for people who don't care what's under the hood, who just want to get their application running. They are better served with some standard, "safe" black-box, which may be easier to learn, and which allegedly better protects them from their own mistakes.


What are the advantages over other Lisp systems?

Simplicity

Pico Lisp is easy to understand and adapt. There is no compiler enforcing special rules, and the interpreter is simple and straightforward. There are only three data types: Numbers, symbols and lists ("LISP" means "List-, Integer- and Symbol Processing" after all ;-). The memory footprint is minimal, and the tarball size of the whole system is just a few hundred kilobytes.

A Clear Model

Most other systems define the language, and leave it up to the implementation to follow the specifications. Therefore, language designers try to be as abstract and general as possible, leaving many questions and ambiguities to the users of the language.

Pico Lisp does the opposite. Initially, only the single-cell data structure was defined, and then the structure of numbers, symbols and lists as they are composed of these cells. Everything else in the whole system follows from these axioms. This is documented in the chapter about the The Pico Machine in the reference manual.

Orthogonality

There is only one symbolic data type, no distinction (confusion) between symbols, strings, variables, special variables and identifiers.

Most data-manipulation functions operate on the value cells of symbols as well as the CARs of list cells:


: (let (N 7  L (7 7 7)) (inc 'N) (inc (cdr L)) (cons N L))
-> (8 7 8 7)

There is only a single functional type, no "special forms". As there is no compiler, functions can be used instead of macros. No special "syntax" constructs are needed. This allows a completely orthogonal use of functions. For example, most other Lisps do not allow calls like


: (mapcar if '(T NIL T NIL) '(1 2 3 4) '(5 6 7 8))
-> (1 6 3 8)

Pico Lisp has no such restrictions. It favors the principle of "Least Astonishment".

Object System

The OOP system is most powerful, because it is fully dynamic, yet extremely simple:

Pragmatism

Pico Lisp has many practical features not found in other Lisp dialects. Among them are:

Persistent Symbols

Database objects ("external" symbols) are a primary data type in Pico Lisp. They look like normal symbols to the programmer, but are managed (fetched from and stored to the data base) automatically by the system. Symbol manipulation functions like set, put or get, the garbage collector, and other parts of the interpreter know about them.

Application Server

Stand-alone system: Does not depend on external programs like Apache or MySQL. Provides a "life" user interface on the client side, via a TCP connection between each Java applet and its corresponding application server session. The GUI layout and behavior (both in plain HTML and in Java applets) is described with s-expressions, generated dynamically at runtime, and maps directly to the database structures.

Localization

Internal exclusive and full use of UTF-8 encoding, and self-translating transient symbols (strings), make it easy to write country- and language-independent applications.


How is the performace compared to other Lisp systems?

Despite the fact that Pico Lisp is an interpreted-only system, the performance is quite good. Typical Lisp programs, operating on list data structures, execute in (interpreted) Pico Lisp at about the same speed as in (compiled) CMUCL, and about two or three times faster than in CLisp or Scheme48. Programs with lots of numeric calculations, however, are several times slower, probably due to Pico Lisp's somewhat inefficient implementation of bignums.

But in practice, speed was never a problem, even with the first versions of Pico Lisp in 1988 on a Mac II with a 12 MHz CPU. And certain things are cleaner and easier to do in plain C anyway. It is very easy to write C functions in Pico Lisp, either in the kernel, as shared object libraries, or even inline in the Lisp code.

Pico Lisp is very space-effective. Other Lisp systems reserve heap space twice as much as needed, or use rather large internal structures to store cells and symbols. The cells (and also the minimal symbols) in Pico Lisp use only 8 bytes each. No additional tags are stored, because they are implied in the pointer encodings. No gaps remain in the heap during allocation, as there are only objects of a single size. As a result, consing and garbage collection are very fast, and overall performace benefits from a better cache efficiency. Heap and stack grow automatically, and are limited only by hardware and operating system constraints.


What means "interpreted"?

It means to directly execute Lisp data as program code. No transformation to some other representation of code (e.g. compilation), and no structural modifications of these data, takes place.

Lisp data are the "real" things, like numbers, symbols and lists, which can be directly handled by the system. They are not the textual representation of these structures (which is outside the Lisp realm and taken care of the reading and printing interfaces).

The following example builds a function and immediately calls it with two arguments:


: ((list (list 'X 'Y) (list '* 'X 'Y)) 3 4)
-> 12

Note that no time is wasted to build up a lexical environment. Variable bindings take place dynamically during interpretation.

A Pico Lisp function is able to inspect or modify itself while it is running (though this is rarely done in application programming). The following function modifies itself by incrementing the '0' in its body:


(de incMe ()
   (do 8
      (printsp 0)
      (inc (cdadr (cdadr incMe))) ) )

: (incMe)
0 1 2 3 4 5 6 7 -> 8
: (incMe)
8 9 10 11 12 13 14 15 -> 16

Only an interpreted Lisp can fully support such "Equivalence of Code and Data". If executable pieces of data are used frequently, like in Pico Lisp's dynamically generated GUI, a fast interpreter is preferable over any compiler.


Is there (or will be in the future) a compiler available?

No. That would contradict the idea of Pico's simple virtual machine structure. A compiler transforms it to another (physical) machine, with the result that many assumptions about the machine's behavior won't hold any more. Besides that, Pico Lisp primitive functions evaluate their arguments independently and are not very much suited for being called from compiled code. Finally, the gain in execution speed would probably not be worth the effort. Typical Pico Lisp applications often use single-pass code which is loaded, executed and thrown away; a process that would be considerably slowed down by compilation.


Is it portable?

Yes and No. Though we wrote and tested Pico Lisp originally only on Linux, it now also runs on FreeBSD, Mac OS X (Darwin), Cygwin/Win32, and probably other POSIX systems. The first versions were even fully portable between DOS, SCO-Unix and Macintosh systems. But today we have Linux. Linux itself is very portable, and you can get access to a Linux system almost everywhere. So why bother?

The GUI is completely platform independent (Browser/Applet), and in the times of Internet an application server does not really need to be portable.


Is Pico Lisp a web server?

Not really, but it evolved a great deal into that direction.

Historically it was the other way round: We had a plain X11 GUI for our applications, and needed something platform independent. The solution was obvious: Browsers with Java are installed virtually everywhere. So we developed a protocol which persuades a browser to function as a GUI frontend to our applications. This is much simpler than to develop a full-blown web server.

In a sense, Pico Lisp is a "pure" application server, not a web server handling "web applications".


I cannot find the LAMBDA keyword in Pico Lisp

Because it isn't there. The reason is that it is redundant; it is equivalent to the quote function in all practical aspects, because there's no distinction between code and data in Pico Lisp, and quote returns the whole (unevaluated) argument list. If you insist on it, you can define your own lambda:


: (setq lambda quote)
-> 67293272
: ((lambda (X Y) (+ X Y)) 3 4)
-> 7
: (mapcar (lambda (X) (+ 1 X)) '(1 2 3 4 5))
-> (2 3 4 5 6)


Why do you use dynamic variable binding?

Dynamic binding is very powerful, because there is only one single, dynamically changing environment active all the time. This makes it possible (e.g. for program sniplets, interspersed with application data and/or passed over the network) to access the whole application context, freely, yet in a dynamically controlled manner. And (shallow) dynamic binding is the fastest method for a Lisp interpreter.

Lexical binding is more limited by definition, because each environment is deliberately restricted to the visible (textual) static scope within its establishing form. Therefore, most Lisps with lexical binding introduce "special variables" to support dynamic binding as well, and constructs like labels to extend the scope of variables beyond a single function.

In Pico Lisp, function definitions are normal symbol values. They can be dynamically rebound like other variables. As a useful real-world example, take this litte gem:


(de recur recurse
   (run (cdr recurse)) )

It implements anonymous recursion, by defining recur statically and recurse dynamically. Usually it is very cumbersome to think up a name for a function (like the following one) which is used only in a single place. But with recur and recurse you can simply write:


: (mapcar
   '((N)
      (recur (N)
         (if (=0 N)
            1
            (* N (recurse (- N 1))) ) ) )
   (1 2 3 4 5 6 7 8) )
-> (1 2 6 24 120 720 5040 40320)

Needless to say, the call to recurse does not have to reside in the same function as recur. Can you implement anonymous recursion so elegantly with lexical binding?


Are there no problems caused by dynamic binding?

You mean the funarg problem, or problems that arise when a variable might be bound to itself? For that reason we have a convention in Pico Lisp to use transient (instead of internal) symbols

  1. for all parameters and locals, when functional arguments or executable lists are passed through the current dynamic bindings
  2. for a parameter or local, when that symbol might possibly be (directly or indirectly) bound to itself, and the bound symbol's value is accessed in the dynamic context

This is a form of lexical scoping - though we still have dynamic binding - of symbols, similar to the static keyword in C.

In fact, these problems are a real threat, and may lead to mysterious bugs (other Lisps have similar problems, e.g. with symbol capture in macros). They can be avoided, however, when the above conventions are observed. As an example, consider a function which doubles the value in a variable:


(de double (Var)
   (set Var (* 2 (val Var))) )

This works fine, as long as we call it as (double 'X), but will break if we call it as (double 'Var). Therefore, the correnct implementation of double should be:


(de double ("Var")
   (set "Var" (* 2 (val "Var"))) )

If double is defined that way in a separate source file, and/or isolated via the ==== function, then the symbol "Var" is locked into a private lexical context and cannot conflict with other symbols.

Admittedly, there are two disadvantages with this solution:

  1. The rules for when to use transient symbols are a bit complicated. Though it is safe to use them even when not necessary, it will take more space then and be more difficult to debug.
  2. The string-like syntax of transient symbols as variables may look strange to alumni of other languages.
Fortunately, these pitfalls do not occur so very often, and seem more likely in utilities than in production code, so that they can be easily encapsulated.


But with dynamic binding I cannot implement closures!

This is not true. Closures are a matter of scope, not of binding.

For a closure it is necessary to build and maintain an environment. For lexical bindings, this has always to be done, and in case of compiled code it is the most efficient strategy anyway, because it is done once by the compiler, and can then be accessed as stack frames at runtime.

For an interpreter, however, this is quite an overhead. So it should not be done automatically at each and every function invocation, but only if needed.

You have several options in Pico Lisp. For simple cases, you can take advantage of the static scope of transient symbols. For the general case, Pico Lisp has built-in functions like bind or job, which dynamically manage statically scoped environments.

As an example, consider a currying function:


(de curry Args
   (list (car Args)
      (list 'list
         (lit (cadr Args))
         (list 'cons ''job
            (list 'cons
               (list 'lit (list 'env (lit (car Args))))
               (lit (cddr Args)) ) ) ) ) )

When called, it returns a function-building function which may be applied to some argument:


: ((curry (X) (N) (* X N)) 3)
-> ((N) (job '((X . 3)) (* X N)))

or used as:


: (((curry (X) (N) (* X N)) 3) 4)
-> 12

In other cases, you are free to choose a shorter and faster solution. If (as in the example above) the curried argument is known to be immutable:


(de curry Args
   (list
      (cadr Args)
      (list 'fill
         (lit (cons (car Args) (cddr Args)))
         (lit (cadr Args)) ) ) )

Then the function built above will just be:


: ((curry (X) (N) (* X N)) 3)
-> ((X) (* X 3))

In that case, the "environment build-up" is reduced by a simple (lexical) constant substitution with zero runtime overhead.


Do you have macros?

Yes, there is a macro mechanism in Pico Lisp, to build and immediately execute a list of expressions. But it is seldom used. Macros are a kludge. Most things where you need macros in other Lisps are directly expressible as functions in Pico Lisp, which in turn can be applied, passed around, and debugged.


What happens when I locally bind a symbol which has a function definition?

That's not a good idea. The next time that function gets executed within the dynamic context the system may crash. Therefore we have a convention to use an upper case first letter for locally bound symbols:


(de findCar (Car List)
   (when (member Car (cdr List))
      (list Car (car List)) ) )
;-)


Would it make sense to build Pico Lisp in hardware?

At least it should be interesting. It would be a machine executing list (tree) structures instead of linear instruction sequences. "Instruction prefetch" would look down the CAR- and CDR-chains, and perhaps need only a single cache for both data and instructions.

Primitive functions like set, val, if and while, which are written in C now, would be implemented in microcode. Plus a few I/O functions for hardware access. EVAL itself would be a microcode subroutine.

Only a single heap and a single stack is needed. They grow towards each other, and cause garbage collection if they get too close. Heap compaction is trivial due to the single cell size.

There is no assembly-language. The lowest level (above the hardware and microcode levels) are s-expressions: The machine language is Lisp.


Why are there only a few questions in this FAQ?

Because I did not receive many questions so far. Please don't hesitate to discuss with me, abu@software-lab.de. As long as I don't get any feedback, I'll assume that everything is fine and clear, and that there's no need for further documentation :-)